[2026-07-02 06:18:50] Starting L2T batch32 Qwen3-4B GRPO/RLSD_TR dataset-first queue [2026-07-02 06:18:50] Commit: ee6a667 [2026-07-02 06:18:50] Settings: model=Qwen/Qwen3-4B steps=100 train_batch=32 rlsd_tr_batch=32 rollout=8 mini_batch=8 tr_mix=0.1 max_model_len=10240 vllm=0.8 [2026-07-02 06:18:50] Skipping chemistry grpo; completed by previous queue [2026-07-02 06:18:50] Starting chemistry rlsd_tr Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: chemistry Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet Ray tmp: /tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B 2026-07-02 06:18:52,310 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/chemistry Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 06:19:01,985 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2026458) TaskRunner hostname: mole-workspace-rw, PID: 2026458 (TaskRunner pid=2026458) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2026458) 'calculate_entropy': False, (TaskRunner pid=2026458) 'calculate_sum_pi_squared': False, (TaskRunner pid=2026458) 'checkpoint': {'_target_': (TaskRunner pid=2026458) 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2026458) 'async_save': False, (TaskRunner pid=2026458) 'load_contents': ['model', (TaskRunner pid=2026458) 'optimizer', (TaskRunner pid=2026458) 'extra'], (TaskRunner pid=2026458) 'save_contents': ['model', (TaskRunner pid=2026458) 'optimizer', (TaskRunner pid=2026458) 'extra']}, (TaskRunner pid=2026458) 'clip_ratio': 0.2, (TaskRunner pid=2026458) 'clip_ratio_c': 3.0, (TaskRunner pid=2026458) 'clip_ratio_high': 0.28, (TaskRunner pid=2026458) 'clip_ratio_low': 0.2, (TaskRunner pid=2026458) 'data_loader_seed': 42, (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_coeff': 0, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'freeze_vision_tower': False, (TaskRunner pid=2026458) 'fsdp_config': (TaskRunner pid=2026458) {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': False, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'grad_clip': 1.0, (TaskRunner pid=2026458) 'kl_loss_coef': 0.001, (TaskRunner pid=2026458) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2026458) 'loss_scale_factor': None, (TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2026458) 'betas': [0.9, 0.999], (TaskRunner pid=2026458) 'clip_grad': 1.0, (TaskRunner pid=2026458) 'lr': 1e-06, (TaskRunner pid=2026458) 'lr_scheduler_type': 'constant', (TaskRunner pid=2026458) 'lr_warmup_steps': 10, (TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2026458) 'min_lr_ratio': 0.0, (TaskRunner pid=2026458) 'num_cycles': 0.5, (TaskRunner pid=2026458) 'optimizer': 'AdamW', (TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2026458) 'override_optimizer_config': None, (TaskRunner pid=2026458) 'total_training_steps': -1, (TaskRunner pid=2026458) 'warmup_style': None, (TaskRunner pid=2026458) 'weight_decay': 0.01}, (TaskRunner pid=2026458) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2026458) 'clip_cov_lb': 1.0, (TaskRunner pid=2026458) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2026458) 'clip_cov_ub': 5.0, (TaskRunner pid=2026458) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2026458) 'loss_mode': 'rlsd', (TaskRunner pid=2026458) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2026458) 'ppo_epochs': 1, (TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'ppo_micro_batch_size': None, (TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'ppo_mini_batch_size': 8, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2026458) 'mode': 'disabled', (TaskRunner pid=2026458) 'record_file': None, (TaskRunner pid=2026458) 'replay_file': None}, (TaskRunner pid=2026458) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2026458) 'alpha': 0.5, (TaskRunner pid=2026458) 'distillation_add_tail': True, (TaskRunner pid=2026458) 'distillation_topk': 100, (TaskRunner pid=2026458) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2026458) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2026458) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'loss_weight': 0.0, (TaskRunner pid=2026458) 'mask': 'all'}, (TaskRunner pid=2026458) 'feedback_template': '\n' (TaskRunner pid=2026458) 'The ' (TaskRunner pid=2026458) 'following ' (TaskRunner pid=2026458) 'is ' (TaskRunner pid=2026458) 'feedback ' (TaskRunner pid=2026458) 'from ' (TaskRunner pid=2026458) 'your ' (TaskRunner pid=2026458) 'unsuccessful ' (TaskRunner pid=2026458) 'earlier ' (TaskRunner pid=2026458) 'attempt:\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) '{feedback_raw}', (TaskRunner pid=2026458) 'full_logit_distillation': True, (TaskRunner pid=2026458) 'include_environment_feedback': True, (TaskRunner pid=2026458) 'is_clip': 2, (TaskRunner pid=2026458) 'max_reprompt_len': 10240, (TaskRunner pid=2026458) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2026458) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) 'Correctly ' (TaskRunner pid=2026458) 'solve ' (TaskRunner pid=2026458) 'the ' (TaskRunner pid=2026458) 'original ' (TaskRunner pid=2026458) 'question.', (TaskRunner pid=2026458) 'reprompt_truncation': 'right', (TaskRunner pid=2026458) 'solution_template': '\n' (TaskRunner pid=2026458) 'Correct ' (TaskRunner pid=2026458) 'solution:\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) '{successful_previous_attempt}', (TaskRunner pid=2026458) 'srpo_dynamic_weighting': False, (TaskRunner pid=2026458) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2026458) 'success_reward_threshold': 0.5, (TaskRunner pid=2026458) 'teacher_regularization': 'trust-region', (TaskRunner pid=2026458) 'teacher_update_rate': 0.1, (TaskRunner pid=2026458) 'token_reweight_decay_steps': 0, (TaskRunner pid=2026458) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2026458) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2026458) 'shuffle': False, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2026458) 'tau_neg': 1.05, (TaskRunner pid=2026458) 'tau_pos': 1.0, (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_kl_loss': False, (TaskRunner pid=2026458) 'use_prefix_grouper': False, (TaskRunner pid=2026458) 'use_remove_padding': True, (TaskRunner pid=2026458) 'use_torch_compile': True}, (TaskRunner pid=2026458) 'hybrid_engine': True, (TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2026458) 'custom_chat_template': None, (TaskRunner pid=2026458) 'enable_activation_offload': False, (TaskRunner pid=2026458) 'enable_gradient_checkpointing': True, (TaskRunner pid=2026458) 'exclude_modules': None, (TaskRunner pid=2026458) 'external_lib': None, (TaskRunner pid=2026458) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2026458) 'hf_config_path': None, (TaskRunner pid=2026458) 'lora_adapter_path': None, (TaskRunner pid=2026458) 'lora_alpha': 16, (TaskRunner pid=2026458) 'lora_rank': 0, (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'target_modules': 'all-linear', (TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2026458) 'num_shards': 4}, (TaskRunner pid=2026458) 'tokenizer_path': None, (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_liger': False, (TaskRunner pid=2026458) 'use_remove_padding': True, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'nccl_timeout': 600, (TaskRunner pid=2026458) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': True, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'log_prob_micro_batch_size': None, (TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2026458) 'mode': 'disabled', (TaskRunner pid=2026458) 'record_file': None, (TaskRunner pid=2026458) 'replay_file': None}, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_torch_compile': True}, (TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2026458) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2026458) 'agent_loop_config_path': None, (TaskRunner pid=2026458) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2026458) 'name': None, (TaskRunner pid=2026458) 'path': None}, (TaskRunner pid=2026458) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2026458) 'num_workers': 8}, (TaskRunner pid=2026458) 'calculate_log_probs': True, (TaskRunner pid=2026458) 'cudagraph_capture_sizes': None, (TaskRunner pid=2026458) 'data_parallel_size': 1, (TaskRunner pid=2026458) 'disable_log_stats': True, (TaskRunner pid=2026458) 'do_sample': True, (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'enable_chunked_prefill': True, (TaskRunner pid=2026458) 'enable_prefix_caching': True, (TaskRunner pid=2026458) 'enable_rollout_routing_replay': False, (TaskRunner pid=2026458) 'enforce_eager': False, (TaskRunner pid=2026458) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2026458) 'expert_parallel_size': 1, (TaskRunner pid=2026458) 'free_cache_engine': True, (TaskRunner pid=2026458) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2026458) 'ignore_eos': False, (TaskRunner pid=2026458) 'layered_summon': False, (TaskRunner pid=2026458) 'load_format': 'dummy', (TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'log_prob_micro_batch_size': None, (TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2026458) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2026458) 'max_model_len': 10240, (TaskRunner pid=2026458) 'max_num_batched_tokens': 10240, (TaskRunner pid=2026458) 'max_num_seqs': 1024, (TaskRunner pid=2026458) 'mode': 'async', (TaskRunner pid=2026458) 'multi_stage_wake_up': False, (TaskRunner pid=2026458) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'format': 'hermes', (TaskRunner pid=2026458) 'interaction_config_path': None, (TaskRunner pid=2026458) 'max_assistant_turns': None, (TaskRunner pid=2026458) 'max_parallel_calls': 1, (TaskRunner pid=2026458) 'max_tool_response_length': 256, (TaskRunner pid=2026458) 'max_user_turns': None, (TaskRunner pid=2026458) 'num_repeat_rollouts': None, (TaskRunner pid=2026458) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2026458) 'tool_config_path': None, (TaskRunner pid=2026458) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2026458) 'use_inference_chat_template': False}, (TaskRunner pid=2026458) 'n': 8, (TaskRunner pid=2026458) 'name': 'vllm', (TaskRunner pid=2026458) 'over_sample_rate': 0, (TaskRunner pid=2026458) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2026458) 'port': 9090, (TaskRunner pid=2026458) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2026458) 'prompt_length': 2048, (TaskRunner pid=2026458) 'quantization': None, (TaskRunner pid=2026458) 'quantization_config_file': None, (TaskRunner pid=2026458) 'response_length': 8192, (TaskRunner pid=2026458) 'scheduling_policy': 'fcfs', (TaskRunner pid=2026458) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2026458) 'skip_rollout': False, (TaskRunner pid=2026458) 'skip_tokenizer_init': True, (TaskRunner pid=2026458) 'temperature': 1.0, (TaskRunner pid=2026458) 'tensor_model_parallel_size': 2, (TaskRunner pid=2026458) 'top_k': -1, (TaskRunner pid=2026458) 'top_p': 1.0, (TaskRunner pid=2026458) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2026458) 'backend': None, (TaskRunner pid=2026458) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2026458) 'token2text': False}, (TaskRunner pid=2026458) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2026458) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2026458) 'do_sample': True, (TaskRunner pid=2026458) 'n': 16, (TaskRunner pid=2026458) 'temperature': 0.6, (TaskRunner pid=2026458) 'top_k': -1, (TaskRunner pid=2026458) 'top_p': 0.95}}}, (TaskRunner pid=2026458) 'algorithm': (TaskRunner pid=2026458) {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2026458) 'adv_estimator': 'grpo', (TaskRunner pid=2026458) 'gamma': 1.0, (TaskRunner pid=2026458) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2026458) 'horizon': 10000, (TaskRunner pid=2026458) 'kl_coef': 0.001, (TaskRunner pid=2026458) 'target_kl': 0.1, (TaskRunner pid=2026458) 'type': 'fixed'}, (TaskRunner pid=2026458) 'kl_penalty': 'kl', (TaskRunner pid=2026458) 'lam': 1.0, (TaskRunner pid=2026458) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2026458) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2026458) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2026458) 'loss_type': 'ppo_clip', (TaskRunner pid=2026458) 'rollout_is': 'token', (TaskRunner pid=2026458) 'rollout_is_batch_normalize': False, (TaskRunner pid=2026458) 'rollout_is_threshold': 2.0, (TaskRunner pid=2026458) 'rollout_rs': None, (TaskRunner pid=2026458) 'rollout_rs_threshold': None}, (TaskRunner pid=2026458) 'use_kl_in_reward': False, (TaskRunner pid=2026458) 'use_pf_ppo': False}, (TaskRunner pid=2026458) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2026458) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2026458) 'async_save': False, (TaskRunner pid=2026458) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2026458) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2026458) 'cliprange_value': 0.5, (TaskRunner pid=2026458) 'data_loader_seed': 42, (TaskRunner pid=2026458) 'enable': None, (TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'forward_micro_batch_size': None, (TaskRunner pid=2026458) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'grad_clip': 1.0, (TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2026458) 'enable_activation_offload': False, (TaskRunner pid=2026458) 'enable_gradient_checkpointing': True, (TaskRunner pid=2026458) 'external_lib': None, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': False, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'lora_alpha': 16, (TaskRunner pid=2026458) 'lora_rank': 0, (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2026458) 'target_modules': 'all-linear', (TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2026458) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_remove_padding': False, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2026458) 'betas': [0.9, 0.999], (TaskRunner pid=2026458) 'clip_grad': 1.0, (TaskRunner pid=2026458) 'lr': 1e-05, (TaskRunner pid=2026458) 'lr_scheduler_type': 'constant', (TaskRunner pid=2026458) 'lr_warmup_steps': -1, (TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2026458) 'min_lr_ratio': 0.0, (TaskRunner pid=2026458) 'num_cycles': 0.5, (TaskRunner pid=2026458) 'optimizer': 'AdamW', (TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2026458) 'override_optimizer_config': None, (TaskRunner pid=2026458) 'total_training_steps': -1, (TaskRunner pid=2026458) 'warmup_style': None, (TaskRunner pid=2026458) 'weight_decay': 0.01}, (TaskRunner pid=2026458) 'ppo_epochs': 1, (TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'ppo_micro_batch_size': None, (TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'ppo_mini_batch_size': 8, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'shuffle': False, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False}, (TaskRunner pid=2026458) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2026458) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2026458) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2026458) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2026458) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2026458) 'dataloader_num_workers': 8, (TaskRunner pid=2026458) 'filter_overlong_prompts': True, (TaskRunner pid=2026458) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2026458) 'image_key': 'images', (TaskRunner pid=2026458) 'image_patch_size': 14, (TaskRunner pid=2026458) 'max_prompt_length': 2048, (TaskRunner pid=2026458) 'max_response_length': 8192, (TaskRunner pid=2026458) 'prompt_key': 'prompt', (TaskRunner pid=2026458) 'return_full_prompt': False, (TaskRunner pid=2026458) 'return_multi_modal_inputs': True, (TaskRunner pid=2026458) 'return_raw_chat': True, (TaskRunner pid=2026458) 'return_raw_input_ids': False, (TaskRunner pid=2026458) 'reward_fn_key': 'data_source', (TaskRunner pid=2026458) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2026458) 'seed': None, (TaskRunner pid=2026458) 'shuffle': True, (TaskRunner pid=2026458) 'tokenizer': None, (TaskRunner pid=2026458) 'tool_config_path': None, (TaskRunner pid=2026458) 'train_batch_size': 32, (TaskRunner pid=2026458) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'], (TaskRunner pid=2026458) 'train_max_samples': 3200, (TaskRunner pid=2026458) 'truncation': 'error', (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_shm': False, (TaskRunner pid=2026458) 'val_batch_size': None, (TaskRunner pid=2026458) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'], (TaskRunner pid=2026458) 'val_max_samples': -1, (TaskRunner pid=2026458) 'validation_shuffle': False, (TaskRunner pid=2026458) 'video_key': 'videos'}, (TaskRunner pid=2026458) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2026458) 'cuda-memory-usage': 'true', (TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2026458) 'capture-range-end': None, (TaskRunner pid=2026458) 'cuda-graph-trace': 'graph', (TaskRunner pid=2026458) 'cuda-memory-usage': 'true', (TaskRunner pid=2026458) 'kill': 'none', (TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2026458) 'torch_memory': {'context': 'all', (TaskRunner pid=2026458) 'kw_args': {}, (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'stacks': 'all', (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2026458) 'profile_continuous_steps': False, (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'steps': None, (TaskRunner pid=2026458) 'tool': None}, (TaskRunner pid=2026458) 'max_model_len': 10240, (TaskRunner pid=2026458) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B', (TaskRunner pid=2026458) 'include_dashboard': False, (TaskRunner pid=2026458) 'num_cpus': None, (TaskRunner pid=2026458) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'TASK': 'datasets/sciknoweval/chemistry', (TaskRunner pid=2026458) 'USER': 'root'}}}, (TaskRunner pid=2026458) 'timeline_json_file': None}, (TaskRunner pid=2026458) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2026458) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2026458) 'name': 'custom_reward_manager', (TaskRunner pid=2026458) 'path': None}, (TaskRunner pid=2026458) 'name': 'naive', (TaskRunner pid=2026458) 'source': 'register'}, (TaskRunner pid=2026458) 'reward_model': {'enable': False, (TaskRunner pid=2026458) 'enable_resource_pool': False, (TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'launch_reward_fn_async': False, (TaskRunner pid=2026458) 'max_length': None, (TaskRunner pid=2026458) 'micro_batch_size': None, (TaskRunner pid=2026458) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'model': {'external_lib': None, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2026458) 'trust_remote_code': False, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_remove_padding': False, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'n_gpus_per_node': 8, (TaskRunner pid=2026458) 'nnodes': 0, (TaskRunner pid=2026458) 'num_workers': 1, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'reward_loop_class_name': None, (TaskRunner pid=2026458) 'reward_loop_module_path': None, (TaskRunner pid=2026458) 'reward_loop_source': 'register', (TaskRunner pid=2026458) 'reward_manager': 'naive', (TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2026458) 'cudagraph_capture_sizes': None, (TaskRunner pid=2026458) 'data_parallel_size': 1, (TaskRunner pid=2026458) 'disable_log_stats': True, (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'enable_chunked_prefill': True, (TaskRunner pid=2026458) 'enable_prefix_caching': True, (TaskRunner pid=2026458) 'enforce_eager': True, (TaskRunner pid=2026458) 'engine_kwargs': {}, (TaskRunner pid=2026458) 'expert_parallel_size': 1, (TaskRunner pid=2026458) 'free_cache_engine': True, (TaskRunner pid=2026458) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2026458) 'limit_images': None, (TaskRunner pid=2026458) 'load_format': 'auto', (TaskRunner pid=2026458) 'max_model_len': None, (TaskRunner pid=2026458) 'max_num_batched_tokens': 8192, (TaskRunner pid=2026458) 'max_num_seqs': 1024, (TaskRunner pid=2026458) 'name': '???', (TaskRunner pid=2026458) 'prompt_length': 2048, (TaskRunner pid=2026458) 'response_length': 2048, (TaskRunner pid=2026458) 'skip_tokenizer_init': False, (TaskRunner pid=2026458) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2026458) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2026458) 'memory_limit_mb': 1024, (TaskRunner pid=2026458) 'url': None}, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False, (TaskRunner pid=2026458) 'use_reward_loop': False}, (TaskRunner pid=2026458) 'trainer': {'balance_batch': True, (TaskRunner pid=2026458) 'critic_warmup': 0, (TaskRunner pid=2026458) 'default_hdfs_dir': None, (TaskRunner pid=2026458) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'del_local_ckpt_after_load': False, (TaskRunner pid=2026458) 'device': 'cuda', (TaskRunner pid=2026458) 'esi_redundant_time': 0, (TaskRunner pid=2026458) 'experiment_name': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2026458) 'log_val_generations': 0, (TaskRunner pid=2026458) 'logger': ['console', 'wandb'], (TaskRunner pid=2026458) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2026458) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2026458) 'n_gpus_per_node': 8, (TaskRunner pid=2026458) 'nnodes': 1, (TaskRunner pid=2026458) 'project_name': 'SDPO-root', (TaskRunner pid=2026458) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2026458) 'resume_from_path': None, (TaskRunner pid=2026458) 'resume_mode': 'auto', (TaskRunner pid=2026458) 'rollout_data_dir': None, (TaskRunner pid=2026458) 'save_freq': 10, (TaskRunner pid=2026458) 'test_freq': 10, (TaskRunner pid=2026458) 'total_epochs': 30, (TaskRunner pid=2026458) 'total_training_steps': 100, (TaskRunner pid=2026458) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2026458) 'val_before_train': False, (TaskRunner pid=2026458) 'val_only': False, (TaskRunner pid=2026458) 'validation_data_dir': None}, (TaskRunner pid=2026458) 'transfer_queue': {'enable': False}, (TaskRunner pid=2026458) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry', (TaskRunner pid=2026458) 'dir': '/users/root/SDPO', (TaskRunner pid=2026458) 'log_dir': '/users/root/output', (TaskRunner pid=2026458) 'task': 'datasets/sciknoweval/chemistry'}} (TaskRunner pid=2026458) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2026458) use_critic=need_critic(config), (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2026458) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,289:'(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (TaskRunner pid=2026458) Retrying in 1s [Retry 1/5]. (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,290:Retrying in 1s [Retry 1/5]. (TaskRunner pid=2026458) Using dataset class: RLHFDataset (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2026458) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2026458) dataset len: 1890 (TaskRunner pid=2026458) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:12,124:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00 (TaskRunner pid=2026458) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 301.55 examples/s] (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2026458) self.use_critic = need_critic(self.config) (WorkerDict pid=2026872) [W702 06:19:21.430329774 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2026871) [Gloo] Rank 3 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2026873) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2026873) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2026869) [W702 06:19:22.501122826 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2026868) Model config after override: Qwen3Config { (WorkerDict pid=2026868) "architectures": [ (WorkerDict pid=2026868) "Qwen3ForCausalLM" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "attention_bias": false, (WorkerDict pid=2026868) "attention_dropout": 0.0, (WorkerDict pid=2026868) "dtype": "bfloat16", (WorkerDict pid=2026868) "eos_token_id": 151645, (WorkerDict pid=2026868) "head_dim": 128, (WorkerDict pid=2026868) "hidden_act": "silu", (WorkerDict pid=2026868) "hidden_size": 2560, (WorkerDict pid=2026868) "initializer_range": 0.02, (WorkerDict pid=2026868) "intermediate_size": 9728, (WorkerDict pid=2026868) "layer_types": [ (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "max_position_embeddings": 40960, (WorkerDict pid=2026868) "max_window_layers": 36, (WorkerDict pid=2026868) "model_type": "qwen3", (WorkerDict pid=2026868) "num_attention_heads": 32, (WorkerDict pid=2026868) "num_hidden_layers": 36, (WorkerDict pid=2026868) "num_key_value_heads": 8, (WorkerDict pid=2026868) "pad_token_id": 151643, (WorkerDict pid=2026868) "rms_norm_eps": 1e-06, (WorkerDict pid=2026868) "rope_scaling": null, (WorkerDict pid=2026868) "rope_theta": 1000000, (WorkerDict pid=2026868) "sliding_window": null, (WorkerDict pid=2026868) "tie_word_embeddings": true, (WorkerDict pid=2026868) "transformers_version": "4.57.1", (WorkerDict pid=2026868) "use_cache": true, (WorkerDict pid=2026868) "use_sliding_window": false, (WorkerDict pid=2026868) "vocab_size": 151936 (WorkerDict pid=2026868) } (WorkerDict pid=2026868) (WorkerDict pid=2026873) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2026873) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2026868) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2026871) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2026870) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.94s/it] [repeated 12x across cluster] (WorkerDict pid=2026870) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster] (WorkerDict pid=2026868) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 3.00s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 2x across cluster] (WorkerDict pid=2026868) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2026868) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2026868) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2026868) Actor use_remove_padding=True (WorkerDict pid=2026868) Actor use_fused_kernels=False (WorkerDict pid=2026868) Actor use_prefix_grouper=False (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 1 peer ranks. Expected number of connected peer ranks is : 1 (WorkerDict pid=2026871) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2026871) warnings.warn( (WorkerDict pid=2026869) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2026869) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2026868) reference model: Qwen/Qwen3-4B (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2026868) Model config after override: Qwen3Config { (WorkerDict pid=2026868) "architectures": [ (WorkerDict pid=2026868) "Qwen3ForCausalLM" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "attention_bias": false, (WorkerDict pid=2026868) "attention_dropout": 0.0, (WorkerDict pid=2026868) "dtype": "bfloat16", (WorkerDict pid=2026868) "eos_token_id": 151645, (WorkerDict pid=2026868) "head_dim": 128, (WorkerDict pid=2026868) "hidden_act": "silu", (WorkerDict pid=2026868) "hidden_size": 2560, (WorkerDict pid=2026868) "initializer_range": 0.02, (WorkerDict pid=2026868) "intermediate_size": 9728, (WorkerDict pid=2026868) "layer_types": [ (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "max_position_embeddings": 40960, (WorkerDict pid=2026868) "max_window_layers": 36, (WorkerDict pid=2026868) "model_type": "qwen3", (WorkerDict pid=2026868) "num_attention_heads": 32, (WorkerDict pid=2026868) "num_hidden_layers": 36, (WorkerDict pid=2026868) "num_key_value_heads": 8, (WorkerDict pid=2026868) "pad_token_id": 151643, (WorkerDict pid=2026868) "rms_norm_eps": 1e-06, (WorkerDict pid=2026868) "rope_scaling": null, (WorkerDict pid=2026868) "rope_theta": 1000000, (WorkerDict pid=2026868) "sliding_window": null, (WorkerDict pid=2026868) "tie_word_embeddings": true, (WorkerDict pid=2026868) "transformers_version": "4.57.1", (WorkerDict pid=2026868) "use_cache": true, (WorkerDict pid=2026868) "use_sliding_window": false, (WorkerDict pid=2026868) "vocab_size": 151936 (WorkerDict pid=2026868) } (WorkerDict pid=2026868) (WorkerDict pid=2026871) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2026868) Ref use_remove_padding=True (WorkerDict pid=2026868) Ref use_fused_kernels=False (WorkerDict pid=2026868) Ref use_prefix_grouper=False (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2026874) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] [repeated 12x across cluster] (WorkerDict pid=2026873) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.89s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster] (WorkerDict pid=2026874) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 4x across cluster] (vLLMHttpServer pid=2027750) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2027750) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2027750) ['serve', (vLLMHttpServer pid=2027750) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2027750) '--dtype', (vLLMHttpServer pid=2027750) 'bfloat16', (vLLMHttpServer pid=2027750) '--load_format', (vLLMHttpServer pid=2027750) 'dummy', (vLLMHttpServer pid=2027750) '--trust_remote_code', (vLLMHttpServer pid=2027750) '--max_model_len', (vLLMHttpServer pid=2027750) '40960', (vLLMHttpServer pid=2027750) '--max_num_seqs', (vLLMHttpServer pid=2027750) '1024', (vLLMHttpServer pid=2027750) '--enable_chunked_prefill', (vLLMHttpServer pid=2027750) '--max_num_batched_tokens', (vLLMHttpServer pid=2027750) '10240', (vLLMHttpServer pid=2027750) '--enable_prefix_caching', (vLLMHttpServer pid=2027750) '--enable_sleep_mode', (vLLMHttpServer pid=2027750) '--logprobs_mode', (vLLMHttpServer pid=2027750) 'processed_logprobs', (vLLMHttpServer pid=2027750) '--disable_custom_all_reduce', (vLLMHttpServer pid=2027750) '--gpu_memory_utilization', (vLLMHttpServer pid=2027750) '0.8', (vLLMHttpServer pid=2027750) '--disable_log_stats', (vLLMHttpServer pid=2027750) '--tensor_parallel_size', (vLLMHttpServer pid=2027750) '2', (vLLMHttpServer pid=2027750) '--seed', (vLLMHttpServer pid=2027750) '0', (vLLMHttpServer pid=2027750) '--override_generation_config', (vLLMHttpServer pid=2027750) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2027750) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2027750) '--hf_overrides', (vLLMHttpServer pid=2027750) '{}', (vLLMHttpServer pid=2027750) '--scheduling_policy', (vLLMHttpServer pid=2027750) 'fcfs'] (WorkerDict pid=2026872) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2026872) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (vLLMHttpServer pid=2027750) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2027750) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2027750) WARNING 07-02 06:20:18 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2026868) WARNING 07-02 06:20:26 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2027751) WARNING 07-02 06:20:19 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2026872) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2026868) 2026-07-02 06:20:40,724 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2026868) 2026-07-02 06:20:40,743 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2027753) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2027753) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2027753) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2027753) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a complex structure with a benzene ring substituted with a chlorine atom and a trifluoromethyl group (C(F)(F)F). The key part of the product is the "Cc2ccc(Cl)cc2C(F)(F)F" fragment, which suggests that the reactant must contain a similar structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A contains a ketone group (O=C) and a nitrogen-containing ring, but it does not match the specific substituents in the product. (TaskRunner pid=2026458) - Option B contains a chlorine atom and a nitrogen, but it does not include the trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) - Option C contains a trifluoromethyl group (C(F)(F)F) and a chlorine atom, and it includes a structure that matches the "Cc2ccc(Cl)cc2" part of the product. (TaskRunner pid=2026458) - Option D contains a trifluoromethyl group but lacks the chlorine atom in the correct position. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C is the only one that includes both the chlorine and the trifluoromethyl group in the correct positions, matching the structure of the product. Therefore, it is the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) C (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 0.0 (TaskRunner pid=2026458) [acc] 0.0 (TaskRunner pid=2026458) [pred] C (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2026458) step:10 - global_seqlen/min:17490 - global_seqlen/max:20901 - global_seqlen/minmax_diff:3411 - global_seqlen/balanced_min:19357 - global_seqlen/balanced_max:19367 - global_seqlen/mean:19364.75 - actor/entropy:0.24537719786167145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744035601615906 - training/rollout_probs_diff_mean:0.005785205401480198 - training/rollout_probs_diff_std:0.014700246043503284 - training/rollout_actor_probs_pearson_corr:0.9979208707809448 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9998391270637512 - rollout_corr/rollout_is_ratio_fraction_high:1.1633588655968197e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.980152829783037e-05 - rollout_corr/rollout_is_max:2.010491132736206 - rollout_corr/rollout_is_min:0.23784410953521729 - rollout_corr/rollout_is_std:0.04395073652267456 - rollout_corr/rollout_is_eff_sample_size:0.9980714630669532 - rollout_corr/rollout_is_seq_mean:0.9998910427093506 - rollout_corr/rollout_is_seq_std:0.0027966259513050318 - rollout_corr/rollout_is_seq_max:1.0104008913040161 - rollout_corr/rollout_is_seq_min:0.9919784069061279 - rollout_corr/rollout_is_seq_max_deviation:0.010400891304016113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026330736465752) - rollout_corr/training_log_ppl:np.float64(0.26078601760673337) - rollout_corr/kl:np.float64(0.0013049291076470126) - rollout_corr/k3_kl:np.float64(0.0011964667813799679) - rollout_corr/rollout_ppl:np.float64(1.3009595507755876) - rollout_corr/rollout_log_ppl:np.float64(0.25948108817101456) - rollout_corr/log_ppl_diff:np.float64(0.0013049294357188046) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024894755915738642) - rollout_corr/log_ppl_diff_max:np.float64(0.017253637313842773) - rollout_corr/log_ppl_diff_min:np.float64(-0.008967161178588867) - rollout_corr/ppl_ratio:np.float64(1.0013103527016938) - rollout_corr/chi2_token:np.float64(0.002327824244275689) - rollout_corr/chi2_seq:np.float64(0.852935308823362) - actor/pg_loss:np.float64(0.00642693554982543) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010866172510759498) - actor/ppo_kl:np.float64(0.00015771930741692586) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(177288.61759452336) - self_distillation/token_reweight_w_std:np.float64(2583470.6754472964) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025011864490807) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1041747338604182) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.5560944229364395) - perf/mfu/actor:np.float64(0.03574857500362935) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(141.2105484008789) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16058361739063698) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7451095238095238) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04477536267518606) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.46405714285714283) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11915383713937476) - val-aux/sciknoweval/score/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/score/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.16058361739063698) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7451095238095238) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04477536267518606) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.46405714285714283) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11915383713937476) - val-core/sciknoweval/acc/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/acc/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.16058361739063698) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7451095238095238) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04477536267518606) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.46405714285714283) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11915383713937476) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994571428571428) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015133869749850571) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997476190476191) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0010668260935279934) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9989428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0019790054535129544) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999190476190476) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0006155763481184213) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9981333333333333) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023247461032216924) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9970238095238095) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0023053472298853674) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001270969514735043 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001270969514735043 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.7734375 - response_length/max:1122.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.7734375 - response_length_non_aborted/max:1122.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.375 - prompt_length/max:701.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.7655776143074036e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4987823646515608) - timing_s/agent_loop/generate_sequences/max:np.float64(7.95271722227335) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.580353087221738) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.95271722227335) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1122 - timing_s/gen:13.539362616837025 - timing_s/reward:0.05532005988061428 - timing_s/old_log_prob:2.3085584342479706 - timing_s/adv:0.48644337616860867 - timing_s/update_actor:15.44553491845727 - timing_s/step:31.924703411757946 - timing_s/testing:117.72085026092827 - timing_s/save_checkpoint:6.645242588594556 - timing_s/stop_profile:0.0001191999763250351 - timing_per_token_ms/adv:0.0031400055265921887 - timing_per_token_ms/gen:0.1575113731919894 - timing_per_token_ms/update_actor:0.09970135761149299 - perf/total_num_tokens:154918 - perf/time_per_step:31.924703411757946 - perf/throughput:606.5757213226894 (TaskRunner pid=2026458) Training Progress: 10%|█ | 10/100 [08:48<1:56:19, 77.55s/it] (TaskRunner pid=2026458) step:11 - global_seqlen/min:17717 - global_seqlen/max:22105 - global_seqlen/minmax_diff:4388 - global_seqlen/balanced_min:20139 - global_seqlen/balanced_max:20427 - global_seqlen/mean:20176.875 - actor/entropy:0.28003615140914917 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2986578345298767 - training/rollout_probs_diff_mean:0.006229304242879152 - training/rollout_probs_diff_std:0.014922214671969414 - training/rollout_actor_probs_pearson_corr:0.9980378746986389 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9996895790100098 - rollout_corr/rollout_is_ratio_fraction_high:2.3571841666125692e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.071552681736648e-05 - rollout_corr/rollout_is_max:2.091331720352173 - rollout_corr/rollout_is_min:0.056948471814394 - rollout_corr/rollout_is_std:0.04615355655550957 - rollout_corr/rollout_is_eff_sample_size:0.9978729526678736 - rollout_corr/rollout_is_seq_mean:0.9996609091758728 - rollout_corr/rollout_is_seq_std:0.002769676735624671 - rollout_corr/rollout_is_seq_max:1.0097713470458984 - rollout_corr/rollout_is_seq_min:0.9914740920066833 - rollout_corr/rollout_is_seq_max_deviation:0.009771347045898438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3445568550378084) - rollout_corr/training_log_ppl:np.float64(0.2928216397704091) - rollout_corr/kl:np.float64(0.0013540790551616055) - rollout_corr/k3_kl:np.float64(0.001210083299042708) - rollout_corr/rollout_ppl:np.float64(1.3426842936314642) - rollout_corr/rollout_log_ppl:np.float64(0.2914675589709077) - rollout_corr/log_ppl_diff:np.float64(0.0013540807995013893) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025300010456703603) - rollout_corr/log_ppl_diff_max:np.float64(0.011778771877288818) - rollout_corr/log_ppl_diff_min:np.float64(-0.006693422794342041) - rollout_corr/ppl_ratio:np.float64(1.001359449699521) - rollout_corr/chi2_token:np.float64(0.0021713071037083864) - rollout_corr/chi2_seq:np.float64(0.5860715808812529) - actor/pg_loss:np.float64(0.009074105182662606) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010309692956980143) - actor/ppo_kl:np.float64(-0.00014117327255291912) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(239610.312335568) - self_distillation/token_reweight_w_std:np.float64(3316290.997419097) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019973907619715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14387543653720059) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.6877006888389587) - perf/mfu/actor:np.float64(0.037142691887104826) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9859390258789) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006922754924744368 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006922754924744368 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:331.43359375 - response_length/max:1206.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.43359375 - response_length_non_aborted/max:1206.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012538395822048187 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7413379289209843) - timing_s/agent_loop/generate_sequences/max:np.float64(8.170563256368041) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.67358700458135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.170563256368041) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:1206 - timing_s/gen:14.099589532241225 - timing_s/reward:0.05785256810486317 - timing_s/old_log_prob:2.514467492699623 - timing_s/adv:0.5351101234555244 - timing_s/update_actor:15.359056254848838 - timing_s/step:32.6669915933162 - timing_s/stop_profile:0.0001175031065940857 - timing_per_token_ms/adv:0.003315120177527023 - timing_per_token_ms/gen:0.16617664186407563 - timing_per_token_ms/update_actor:0.09515259582349124 - perf/total_num_tokens:161415 - perf/time_per_step:32.6669915933162 - perf/throughput:617.6532951423746 (TaskRunner pid=2026458) Training Progress: 11%|█ | 11/100 [09:20<1:34:40, 63.83s/it] (TaskRunner pid=2026458) step:12 - global_seqlen/min:17937 - global_seqlen/max:22358 - global_seqlen/minmax_diff:4421 - global_seqlen/balanced_min:19749 - global_seqlen/balanced_max:19763 - global_seqlen/mean:19757.25 - actor/entropy:0.2599850594997406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29290565848350525 - training/rollout_probs_diff_mean:0.00601433590054512 - training/rollout_probs_diff_std:0.014930167235434055 - training/rollout_actor_probs_pearson_corr:0.9979494214057922 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.4609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999984085559845 - rollout_corr/rollout_is_ratio_fraction_high:2.223754199803807e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.33563111780677e-05 - rollout_corr/rollout_is_max:2.084258556365967 - rollout_corr/rollout_is_min:0.48210135102272034 - rollout_corr/rollout_is_std:0.04539177566766739 - rollout_corr/rollout_is_eff_sample_size:0.9979434670549656 - rollout_corr/rollout_is_seq_mean:0.9999362230300903 - rollout_corr/rollout_is_seq_std:0.002637566300109029 - rollout_corr/rollout_is_seq_max:1.0081006288528442 - rollout_corr/rollout_is_seq_min:0.9917522072792053 - rollout_corr/rollout_is_seq_max_deviation:0.008247792720794678 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3254084084182978) - rollout_corr/training_log_ppl:np.float64(0.2779389991774224) - rollout_corr/kl:np.float64(0.0013704395118168122) - rollout_corr/k3_kl:np.float64(0.0011261843454803966) - rollout_corr/rollout_ppl:np.float64(1.3235342632979155) - rollout_corr/rollout_log_ppl:np.float64(0.27656855969689786) - rollout_corr/log_ppl_diff:np.float64(0.0013704394805245101) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024474224192090333) - rollout_corr/log_ppl_diff_max:np.float64(0.011240929365158081) - rollout_corr/log_ppl_diff_min:np.float64(-0.010027527809143066) - rollout_corr/ppl_ratio:np.float64(1.001375619089231) - rollout_corr/chi2_token:np.float64(0.001807245658710599) - rollout_corr/chi2_seq:np.float64(1.2864351340103894) - actor/pg_loss:np.float64(0.0068493077997118235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010652919690983254) - actor/ppo_kl:np.float64(0.00020145679512317827) - actor/pg_clipfrac_lower:np.float64(1.669337689236272e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4609375) - self_distillation/token_reweight_w_mean:np.float64(145999.17218339257) - self_distillation/token_reweight_w_std:np.float64(2374372.460695313) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004515508189797) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1246801737579517) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.5914773046970367) - perf/mfu/actor:np.float64(0.03625096042724267) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.91592407226562) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004239031113684177 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004239031113684177 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:351.3203125 - response_length/max:1148.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:351.3203125 - response_length_non_aborted/max:1148.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.013755116611719131 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6791318338364363) - timing_s/agent_loop/generate_sequences/max:np.float64(8.12350033968687) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.846359725270304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.12350033968687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1148 - timing_s/gen:13.899831658229232 - timing_s/reward:0.05922466889023781 - timing_s/old_log_prob:2.4094871655106544 - timing_s/adv:0.5697984360158443 - timing_s/update_actor:15.474863145500422 - timing_s/step:32.48048596084118 - timing_s/stop_profile:0.0001192968338727951 - timing_per_token_ms/adv:0.003604995862378648 - timing_per_token_ms/gen:0.15454904109752532 - timing_per_token_ms/update_actor:0.09790623154475206 - perf/total_num_tokens:158058 - perf/time_per_step:32.48048596084118 - perf/throughput:608.2806157463147 (TaskRunner pid=2026458) Training Progress: 12%|█▏ | 12/100 [09:53<1:19:39, 54.31s/it] (TaskRunner pid=2026458) step:13 - global_seqlen/min:17460 - global_seqlen/max:22387 - global_seqlen/minmax_diff:4927 - global_seqlen/balanced_min:19538 - global_seqlen/balanced_max:19656 - global_seqlen/mean:19562.0 - actor/entropy:0.28330478072166443 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932348847389221 - training/rollout_probs_diff_mean:0.006090556271374226 - training/rollout_probs_diff_std:0.014429626986384392 - training/rollout_actor_probs_pearson_corr:0.998181939125061 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9998684525489807 - rollout_corr/rollout_is_ratio_fraction_high:2.3373224394163117e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.3373224394163117e-05 - rollout_corr/rollout_is_max:2.1884407997131348 - rollout_corr/rollout_is_min:0.4281519055366516 - rollout_corr/rollout_is_std:0.04431807994842529 - rollout_corr/rollout_is_eff_sample_size:0.9980391641302073 - rollout_corr/rollout_is_seq_mean:0.9998126029968262 - rollout_corr/rollout_is_seq_std:0.0026776245795190334 - rollout_corr/rollout_is_seq_max:1.009365439414978 - rollout_corr/rollout_is_seq_min:0.9913782477378845 - rollout_corr/rollout_is_seq_max_deviation:0.009365439414978027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3591533433645964) - rollout_corr/training_log_ppl:np.float64(0.30308022926328704) - rollout_corr/kl:np.float64(0.0012214116412607723) - rollout_corr/k3_kl:np.float64(0.0011643693818541578) - rollout_corr/rollout_ppl:np.float64(1.3574783634394407) - rollout_corr/rollout_log_ppl:np.float64(0.3018588193517644) - rollout_corr/log_ppl_diff:np.float64(0.0012214099115226418) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024031201901379973) - rollout_corr/log_ppl_diff_max:np.float64(0.011142253875732422) - rollout_corr/log_ppl_diff_min:np.float64(-0.00809323787689209) - rollout_corr/ppl_ratio:np.float64(1.0012265297118574) - rollout_corr/chi2_token:np.float64(0.0022032533306628466) - rollout_corr/chi2_seq:np.float64(1.1880777331534773) - actor/pg_loss:np.float64(0.008368437876924872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011567205356186605) - actor/ppo_kl:np.float64(-1.6140308314760432e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(93498.56042674859) - self_distillation/token_reweight_w_std:np.float64(1510537.690864286) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999996354104951) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12338341507711448) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.620448037981987) - perf/mfu/actor:np.float64(0.036300476348807696) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.69003295898438) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003617006354033947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003617006354033947 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.25 - response_length/max:1159.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.25 - response_length_non_aborted/max:1159.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014793872833251953 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3433514721691608) - timing_s/agent_loop/generate_sequences/max:np.float64(7.973012773320079) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6053173123218585) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.973012773320079) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:1159 - timing_s/gen:13.718764901161194 - timing_s/reward:0.05917247012257576 - timing_s/old_log_prob:2.3764582853764296 - timing_s/adv:0.49192045256495476 - timing_s/update_actor:15.186074122786522 - timing_s/step:31.921324100345373 - timing_s/stop_profile:0.00019265897572040558 - timing_per_token_ms/adv:0.003143342018741404 - timing_per_token_ms/gen:0.1603258800154403 - timing_per_token_ms/update_actor:0.09703809760496448 - perf/total_num_tokens:156496 - perf/time_per_step:31.921324100345373 - perf/throughput:612.8191906609648 (TaskRunner pid=2026458) Training Progress: 13%|█▎ | 13/100 [10:25<1:08:56, 47.54s/it] (TaskRunner pid=2026458) step:14 - global_seqlen/min:16941 - global_seqlen/max:22560 - global_seqlen/minmax_diff:5619 - global_seqlen/balanced_min:18841 - global_seqlen/balanced_max:18844 - global_seqlen/mean:18842.75 - actor/entropy:0.2922348976135254 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5466010570526123 - training/rollout_probs_diff_mean:0.006264000199735165 - training/rollout_probs_diff_std:0.014866359531879425 - training/rollout_actor_probs_pearson_corr:0.9981194138526917 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0002233982086182 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.292350346688181e-05 - rollout_corr/rollout_is_max:1.9629987478256226 - rollout_corr/rollout_is_min:0.12186871469020844 - rollout_corr/rollout_is_std:0.04516667500138283 - rollout_corr/rollout_is_eff_sample_size:0.9979653119171001 - rollout_corr/rollout_is_seq_mean:1.0002156496047974 - rollout_corr/rollout_is_seq_std:0.00266577722504735 - rollout_corr/rollout_is_seq_max:1.0086764097213745 - rollout_corr/rollout_is_seq_min:0.9924325942993164 - rollout_corr/rollout_is_seq_max_deviation:0.008676409721374512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.356894207186997) - rollout_corr/training_log_ppl:np.float64(0.30215946782846004) - rollout_corr/kl:np.float64(0.0010239662146345752) - rollout_corr/k3_kl:np.float64(0.0012049657070747344) - rollout_corr/rollout_ppl:np.float64(1.3555219103582203) - rollout_corr/rollout_log_ppl:np.float64(0.3011355001362972) - rollout_corr/log_ppl_diff:np.float64(0.0010239676921628416) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023661727900616825) - rollout_corr/log_ppl_diff_max:np.float64(0.017396926879882812) - rollout_corr/log_ppl_diff_min:np.float64(-0.007369279861450195) - rollout_corr/ppl_ratio:np.float64(1.0010292823426425) - rollout_corr/chi2_token:np.float64(0.0027081489097326994) - rollout_corr/chi2_seq:np.float64(1.2070772459264845) - actor/pg_loss:np.float64(0.00797818333376199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010150298585358541) - actor/ppo_kl:np.float64(0.00015036190526984683) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.453125) - self_distillation/token_reweight_w_mean:np.float64(119982.02733938815) - self_distillation/token_reweight_w_std:np.float64(1933813.0637409594) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014771707355976) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1388165877724532) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.6455883979797363) - perf/mfu/actor:np.float64(0.034641753809947065) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.4944953918457) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026480345986783504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026480345986783504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.3984375 - response_length/max:682.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:321.3984375 - response_length_non_aborted/max:682.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012210384011268616 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7070676069706678) - timing_s/agent_loop/generate_sequences/max:np.float64(5.696014670655131) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4708188238437288) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.696014670655131) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:413 - timing_s/agent_loop/slowest/response_length:682 - timing_s/gen:11.448512280359864 - timing_s/reward:0.05713308043777943 - timing_s/old_log_prob:2.4212917368859053 - timing_s/adv:0.4922778233885765 - timing_s/update_actor:15.44989874586463 - timing_s/step:29.956744581460953 - timing_s/stop_profile:0.00011506862938404083 - timing_per_token_ms/adv:0.003265697837288722 - timing_per_token_ms/gen:0.1391442704047238 - timing_per_token_ms/update_actor:0.10249232958209809 - perf/total_num_tokens:150742 - perf/time_per_step:29.956744581460953 - perf/throughput:628.9985865707529 (TaskRunner pid=2026458) Training Progress: 14%|█▍ | 14/100 [10:55<1:00:33, 42.25s/it] (TaskRunner pid=2026458) step:15 - global_seqlen/min:17798 - global_seqlen/max:24465 - global_seqlen/minmax_diff:6667 - global_seqlen/balanced_min:20570 - global_seqlen/balanced_max:20574 - global_seqlen/mean:20572.75 - actor/entropy:0.28748324513435364 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564783573150635 - training/rollout_probs_diff_mean:0.005869686137884855 - training/rollout_probs_diff_std:0.014366100542247295 - training/rollout_actor_probs_pearson_corr:0.9982949495315552 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001139640808105 - rollout_corr/rollout_is_ratio_fraction_high:5.4864267440279946e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3891413952223957e-05 - rollout_corr/rollout_is_max:3.0497446060180664 - rollout_corr/rollout_is_min:0.38640856742858887 - rollout_corr/rollout_is_std:0.045057009905576706 - rollout_corr/rollout_is_eff_sample_size:0.9979744538008674 - rollout_corr/rollout_is_seq_mean:1.0000189542770386 - rollout_corr/rollout_is_seq_std:0.002783029805868864 - rollout_corr/rollout_is_seq_max:1.0093011856079102 - rollout_corr/rollout_is_seq_min:0.9919822812080383 - rollout_corr/rollout_is_seq_max_deviation:0.009301185607910156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3641661452129483) - rollout_corr/training_log_ppl:np.float64(0.3061258476227522) - rollout_corr/kl:np.float64(0.0008984394618103408) - rollout_corr/k3_kl:np.float64(0.0010934443635051139) - rollout_corr/rollout_ppl:np.float64(1.3629517373628914) - rollout_corr/rollout_log_ppl:np.float64(0.3052274066139944) - rollout_corr/log_ppl_diff:np.float64(0.0008984410087577999) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023149027838371694) - rollout_corr/log_ppl_diff_max:np.float64(0.015007197856903076) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058860182762146) - rollout_corr/ppl_ratio:np.float64(1.00090327905491) - rollout_corr/chi2_token:np.float64(0.0026275489944964647) - rollout_corr/chi2_seq:np.float64(1.554575996240601) - actor/pg_loss:np.float64(0.006172901834361255) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010921142766164849) - actor/ppo_kl:np.float64(-0.00014767027690254508) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(168825.68510642461) - self_distillation/token_reweight_w_std:np.float64(2411026.6848952193) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999142882879823) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13169058857602067) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5341654643416405) - perf/mfu/actor:np.float64(0.03786336781112096) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.43387985229492) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036114403046667576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036114403046667576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.9921875 - response_length/max:841.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.9921875 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000149507075548172 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8174101784825325) - timing_s/agent_loop/generate_sequences/max:np.float64(6.822324391454458) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8521247571407002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.822324391454458) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:187 - timing_s/agent_loop/slowest/response_length:841 - timing_s/gen:12.53348901681602 - timing_s/reward:0.06065330095589161 - timing_s/old_log_prob:2.2750260774046183 - timing_s/adv:0.5170386601239443 - timing_s/update_actor:15.529155055060983 - timing_s/step:31.002764070406556 - timing_s/stop_profile:0.00011553429067134857 - timing_per_token_ms/adv:0.003141526170079014 - timing_per_token_ms/gen:0.13752813458002525 - timing_per_token_ms/update_actor:0.09435512422416172 - perf/total_num_tokens:164582 - perf/time_per_step:31.002764070406556 - perf/throughput:663.5779298026383 (TaskRunner pid=2026458) Training Progress: 15%|█▌ | 15/100 [11:26<55:04, 38.87s/it] (TaskRunner pid=2026458) step:16 - global_seqlen/min:17436 - global_seqlen/max:21462 - global_seqlen/minmax_diff:4026 - global_seqlen/balanced_min:19472 - global_seqlen/balanced_max:19474 - global_seqlen/mean:19473.375 - actor/entropy:0.29451751708984375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34674251079559326 - training/rollout_probs_diff_mean:0.006398364901542664 - training/rollout_probs_diff_std:0.014944251626729965 - training/rollout_actor_probs_pearson_corr:0.9980317950248718 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.64453125 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.64453125 - rollout_corr/rollout_is_mean:1.000139832496643 - rollout_corr/rollout_is_ratio_fraction_high:3.57470526068937e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001429882104275748 - rollout_corr/rollout_is_max:2.075308322906494 - rollout_corr/rollout_is_min:0.33335795998573303 - rollout_corr/rollout_is_std:0.04721960052847862 - rollout_corr/rollout_is_eff_sample_size:0.9977759819851924 - rollout_corr/rollout_is_seq_mean:1.0001648664474487 - rollout_corr/rollout_is_seq_std:0.002871463308110833 - rollout_corr/rollout_is_seq_max:1.0102380514144897 - rollout_corr/rollout_is_seq_min:0.9917946457862854 - rollout_corr/rollout_is_seq_max_deviation:0.010238051414489746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36143590323627) - rollout_corr/training_log_ppl:np.float64(0.3041792264266405) - rollout_corr/kl:np.float64(0.0010337699030245062) - rollout_corr/k3_kl:np.float64(0.0012385786424147227) - rollout_corr/rollout_ppl:np.float64(1.3600022280588746) - rollout_corr/rollout_log_ppl:np.float64(0.30314545703004114) - rollout_corr/log_ppl_diff:np.float64(0.0010337693965993822) - rollout_corr/log_ppl_abs_diff:np.float64(0.002399776072707027) - rollout_corr/log_ppl_diff_max:np.float64(0.011399328708648682) - rollout_corr/log_ppl_diff_min:np.float64(-0.006810277700424194) - rollout_corr/ppl_ratio:np.float64(1.0010383201297373) - rollout_corr/chi2_token:np.float64(0.0029240259900689125) - rollout_corr/chi2_seq:np.float64(1.0353804978076369) - actor/pg_loss:np.float64(0.007192879682406783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010274524142914743) - actor/ppo_kl:np.float64(4.0873823671461196e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.64453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(90371.76209130394) - self_distillation/token_reweight_w_std:np.float64(1389709.4737052019) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000855000689626) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11968303663888946) - self_distillation/empty_target_batch:np.float64(0.35546875) - actor/grad_norm:np.float64(0.6043514311313629) - perf/mfu/actor:np.float64(0.03566211221390109) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.67253494262695) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006435959134250879 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006435959134250879 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:327.82421875 - response_length/max:698.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.82421875 - response_length_non_aborted/max:698.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010755844414234161 - timing_s/agent_loop/generate_sequences/min:np.float64(1.786530127748847) - timing_s/agent_loop/generate_sequences/max:np.float64(5.909618901088834) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.657506644296518) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.909618901088834) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:670 - timing_s/gen:11.721529113128781 - timing_s/reward:0.05861745588481426 - timing_s/old_log_prob:2.29558939486742 - timing_s/adv:0.5112862680107355 - timing_s/update_actor:15.580748109146953 - timing_s/step:30.256891636177897 - timing_s/stop_profile:0.00016321614384651184 - timing_per_token_ms/adv:0.003281957210875975 - timing_per_token_ms/gen:0.13967004412531464 - timing_per_token_ms/update_actor:0.10001314685530213 - perf/total_num_tokens:155787 - perf/time_per_step:30.256891636177897 - perf/throughput:643.6013069074107 (TaskRunner pid=2026458) Training Progress: 16%|█▌ | 16/100 [11:56<50:48, 36.29s/it] (TaskRunner pid=2026458) step:17 - global_seqlen/min:19903 - global_seqlen/max:25273 - global_seqlen/minmax_diff:5370 - global_seqlen/balanced_min:22184 - global_seqlen/balanced_max:22190 - global_seqlen/mean:22187.375 - actor/entropy:0.28030383586883545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4096241295337677 - training/rollout_probs_diff_mean:0.00557515537366271 - training/rollout_probs_diff_std:0.014061576686799526 - training/rollout_actor_probs_pearson_corr:0.9982938766479492 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.640625 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.640625 - rollout_corr/rollout_is_mean:1.000001311302185 - rollout_corr/rollout_is_ratio_fraction_high:2.1693151211366057e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.4232874390436336e-05 - rollout_corr/rollout_is_max:3.007925271987915 - rollout_corr/rollout_is_min:0.3316154181957245 - rollout_corr/rollout_is_std:0.0431625172495842 - rollout_corr/rollout_is_eff_sample_size:0.9981403427818611 - rollout_corr/rollout_is_seq_mean:1.0001276731491089 - rollout_corr/rollout_is_seq_std:0.002614374505355954 - rollout_corr/rollout_is_seq_max:1.0077310800552368 - rollout_corr/rollout_is_seq_min:0.9923279881477356 - rollout_corr/rollout_is_seq_max_deviation:0.007731080055236816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.381326341535896) - rollout_corr/training_log_ppl:np.float64(0.3175459819030948) - rollout_corr/kl:np.float64(0.0007249800520758853) - rollout_corr/k3_kl:np.float64(0.0011385135364321286) - rollout_corr/rollout_ppl:np.float64(1.3803315898403525) - rollout_corr/rollout_log_ppl:np.float64(0.316821001295466) - rollout_corr/log_ppl_diff:np.float64(0.0007249806076288223) - rollout_corr/log_ppl_abs_diff:np.float64(0.00215923844370991) - rollout_corr/log_ppl_diff_max:np.float64(0.00994020700454712) - rollout_corr/log_ppl_diff_min:np.float64(-0.007171809673309326) - rollout_corr/ppl_ratio:np.float64(1.0007290567737073) - rollout_corr/chi2_token:np.float64(0.003144999034702778) - rollout_corr/chi2_seq:np.float64(0.9032414378598332) - actor/pg_loss:np.float64(0.0055801779963076115) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008305092796945246) - actor/ppo_kl:np.float64(-0.00022399483597723702) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.640625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(213885.15984977316) - self_distillation/token_reweight_w_std:np.float64(3187382.02780142) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009852752555162) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11256717666401528) - self_distillation/empty_target_batch:np.float64(0.359375) - actor/grad_norm:np.float64(0.5271423608064651) - perf/mfu/actor:np.float64(0.04032452755142399) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.51458740234375) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036620749160647392 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036620749160647392 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.13671875 - response_length/max:1140.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.13671875 - response_length_non_aborted/max:1140.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014656782150268555 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6168197989463806) - timing_s/agent_loop/generate_sequences/max:np.float64(8.074994288384914) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7389801555254962) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.074994288384914) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1140 - timing_s/gen:13.71630746126175 - timing_s/reward:0.06177904084324837 - timing_s/old_log_prob:2.311936927959323 - timing_s/adv:0.5291431322693825 - timing_s/update_actor:15.853100046515465 - timing_s/step:32.560671512037516 - timing_s/stop_profile:0.00012019649147987366 - timing_per_token_ms/adv:0.0029811048640802622 - timing_per_token_ms/gen:0.14877496026098758 - timing_per_token_ms/update_actor:0.08931374287469487 - perf/total_num_tokens:177499 - perf/time_per_step:32.560671512037516 - perf/throughput:681.4163826995226 (TaskRunner pid=2026458) Training Progress: 17%|█▋ | 17/100 [12:29<48:40, 35.18s/it] (TaskRunner pid=2026458) step:18 - global_seqlen/min:17147 - global_seqlen/max:20375 - global_seqlen/minmax_diff:3228 - global_seqlen/balanced_min:18455 - global_seqlen/balanced_max:18458 - global_seqlen/mean:18457.0 - actor/entropy:0.3130648136138916 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772509455680847 - training/rollout_probs_diff_mean:0.0063330200500786304 - training/rollout_probs_diff_std:0.014608096331357956 - training/rollout_actor_probs_pearson_corr:0.9982773065567017 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9997703433036804 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.43533785478212e-05 - rollout_corr/rollout_is_max:1.9900730848312378 - rollout_corr/rollout_is_min:0.4134674072265625 - rollout_corr/rollout_is_std:0.04487474635243416 - rollout_corr/rollout_is_eff_sample_size:0.9979895323387393 - rollout_corr/rollout_is_seq_mean:0.9998025298118591 - rollout_corr/rollout_is_seq_std:0.002652442781254649 - rollout_corr/rollout_is_seq_max:1.0070677995681763 - rollout_corr/rollout_is_seq_min:0.9913365244865417 - rollout_corr/rollout_is_seq_max_deviation:0.008663475513458252 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3969450276345015) - rollout_corr/training_log_ppl:np.float64(0.3299181046313606) - rollout_corr/kl:np.float64(0.001080212333377517) - rollout_corr/k3_kl:np.float64(0.0011240256234259505) - rollout_corr/rollout_ppl:np.float64(1.3953958731144667) - rollout_corr/rollout_log_ppl:np.float64(0.32883789180777967) - rollout_corr/log_ppl_diff:np.float64(0.0010802128235809505) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023538178065791726) - rollout_corr/log_ppl_diff_max:np.float64(0.011510014533996582) - rollout_corr/log_ppl_diff_min:np.float64(-0.009586125612258911) - rollout_corr/ppl_ratio:np.float64(1.0010849123355001) - rollout_corr/chi2_token:np.float64(0.0023459065705537796) - rollout_corr/chi2_seq:np.float64(0.5731152268126607) - actor/pg_loss:np.float64(0.007650406565517187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011541137814674585) - actor/ppo_kl:np.float64(-0.00018551408525979696) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(258180.61296664365) - self_distillation/token_reweight_w_std:np.float64(3445719.439609512) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022555205505341) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15918678505113348) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.6539853662252426) - perf/mfu/actor:np.float64(0.033737344150768206) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.57009506225586) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005579437594860792 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005579437594860792 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:303.5 - response_length/max:713.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:303.5 - response_length_non_aborted/max:713.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013168901205062866 - timing_s/agent_loop/generate_sequences/min:np.float64(1.151022594422102) - timing_s/agent_loop/generate_sequences/max:np.float64(5.789204815402627) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2287484776461497) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.789204815402627) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:713 - timing_s/gen:11.655897682532668 - timing_s/reward:0.05685456842184067 - timing_s/old_log_prob:2.3437488954514265 - timing_s/adv:0.4995372127741575 - timing_s/update_actor:15.640153270214796 - timing_s/step:30.285941103473306 - timing_s/stop_profile:0.00014618411660194397 - timing_per_token_ms/adv:0.0033831148939031095 - timing_per_token_ms/gen:0.15001927618580968 - timing_per_token_ms/update_actor:0.10592291048257298 - perf/total_num_tokens:147656 - perf/time_per_step:30.285941103473306 - perf/throughput:609.4246811397015 (TaskRunner pid=2026458) Training Progress: 18%|█▊ | 18/100 [12:59<46:05, 33.73s/it] (TaskRunner pid=2026458) step:19 - global_seqlen/min:16712 - global_seqlen/max:22903 - global_seqlen/minmax_diff:6191 - global_seqlen/balanced_min:19739 - global_seqlen/balanced_max:19742 - global_seqlen/mean:19740.875 - actor/entropy:0.30067509412765503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2716989815235138 - training/rollout_probs_diff_mean:0.006011276505887508 - training/rollout_probs_diff_std:0.014068786054849625 - training/rollout_actor_probs_pearson_corr:0.9983283281326294 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0000412464141846 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.48191299760947e-05 - rollout_corr/rollout_is_max:1.967125415802002 - rollout_corr/rollout_is_min:0.3074234426021576 - rollout_corr/rollout_is_std:0.04442286118865013 - rollout_corr/rollout_is_eff_sample_size:0.9980306147615733 - rollout_corr/rollout_is_seq_mean:1.0000803470611572 - rollout_corr/rollout_is_seq_std:0.0029041871894150972 - rollout_corr/rollout_is_seq_max:1.0110079050064087 - rollout_corr/rollout_is_seq_min:0.9903981685638428 - rollout_corr/rollout_is_seq_max_deviation:0.011007905006408691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.375831812620163) - rollout_corr/training_log_ppl:np.float64(0.31493956671329215) - rollout_corr/kl:np.float64(0.0009361723424987645) - rollout_corr/k3_kl:np.float64(0.00108607468507671) - rollout_corr/rollout_ppl:np.float64(1.3745251446962357) - rollout_corr/rollout_log_ppl:np.float64(0.31400339302490465) - rollout_corr/log_ppl_diff:np.float64(0.0009361736883874983) - rollout_corr/log_ppl_abs_diff:np.float64(0.002310327283339575) - rollout_corr/log_ppl_diff_max:np.float64(0.008226662874221802) - rollout_corr/log_ppl_diff_min:np.float64(-0.010857582092285156) - rollout_corr/ppl_ratio:np.float64(1.000940806698054) - rollout_corr/chi2_token:np.float64(0.002460781019181013) - rollout_corr/chi2_seq:np.float64(1.263379932148382) - actor/pg_loss:np.float64(0.005378377623856068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007215913469735824) - actor/ppo_kl:np.float64(-2.377988029422795e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(92620.61167126987) - self_distillation/token_reweight_w_std:np.float64(1450653.645121337) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001238340511918) - self_distillation/token_reweight_w_clip_frac:np.float64(0.096618483774364) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.5270146429538727) - perf/mfu/actor:np.float64(0.03604524222111994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.61618423461914) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024989761877804995 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024989761877804995 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:314.77734375 - response_length/max:828.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.77734375 - response_length_non_aborted/max:828.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014412589371204376 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8398964125663042) - timing_s/agent_loop/generate_sequences/max:np.float64(6.303931085392833) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4352486536299693) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.303931085392833) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:828 - timing_s/gen:12.11710455827415 - timing_s/reward:0.05806134268641472 - timing_s/old_log_prob:2.3140441831201315 - timing_s/adv:0.5250083282589912 - timing_s/update_actor:15.580280201509595 - timing_s/step:30.68251054175198 - timing_s/stop_profile:0.00012233667075634003 - timing_per_token_ms/adv:0.0033243734653288625 - timing_per_token_ms/gen:0.150368000177136 - timing_per_token_ms/update_actor:0.09865494944822352 - perf/total_num_tokens:157927 - perf/time_per_step:30.68251054175198 - perf/throughput:643.3917776427427 (TaskRunner pid=2026458) Training Progress: 19%|█▉ | 19/100 [13:30<44:19, 32.83s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a structure with a substituted benzene ring (with a Cl group), a trifluoromethyl group (C(F)(F)F), and a nitrogen-containing amine group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A contains a structure with a substituted benzene ring, a nitrogen atom, and a carbonyl group. It also includes the Cl and trifluoromethyl groups, which match the product's structure. However, it has an extra nitrogen and a different arrangement that may not be the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option B has a similar structure but with a different arrangement of substituents and a different nitrogen configuration, which does not match the product's structure exactly. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option C includes a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a hydroxyl group and a different type of substitution, which does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option D has a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a different type of substitution and does not match the product's structure exactly. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A is the only one that matches the structure of the product, including the correct substituents and arrangement of atoms. Therefore, it is the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2026458) step:20 - global_seqlen/min:17486 - global_seqlen/max:22655 - global_seqlen/minmax_diff:5169 - global_seqlen/balanced_min:20031 - global_seqlen/balanced_max:20228 - global_seqlen/mean:20058.625 - actor/entropy:0.297139972448349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3528301417827606 - training/rollout_probs_diff_mean:0.006166450679302216 - training/rollout_probs_diff_std:0.014436931349337101 - training/rollout_actor_probs_pearson_corr:0.9982419013977051 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000038743019104 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.690288890036754e-05 - rollout_corr/rollout_is_max:1.9574382305145264 - rollout_corr/rollout_is_min:0.3535598814487457 - rollout_corr/rollout_is_std:0.04514159634709358 - rollout_corr/rollout_is_eff_sample_size:0.9979664991652689 - rollout_corr/rollout_is_seq_mean:1.0000238418579102 - rollout_corr/rollout_is_seq_std:0.0029090906027704477 - rollout_corr/rollout_is_seq_max:1.0110317468643188 - rollout_corr/rollout_is_seq_min:0.9888052940368652 - rollout_corr/rollout_is_seq_max_deviation:0.011194705963134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3683372386731207) - rollout_corr/training_log_ppl:np.float64(0.31030508255935274) - rollout_corr/kl:np.float64(0.0011023225397535086) - rollout_corr/k3_kl:np.float64(0.001434143366623175) - rollout_corr/rollout_ppl:np.float64(1.3668043981306255) - rollout_corr/rollout_log_ppl:np.float64(0.30920275836251676) - rollout_corr/log_ppl_diff:np.float64(0.0011023241968359798) - rollout_corr/log_ppl_abs_diff:np.float64(0.002514752763090655) - rollout_corr/log_ppl_diff_max:np.float64(0.0131492018699646) - rollout_corr/log_ppl_diff_min:np.float64(-0.008862733840942383) - rollout_corr/ppl_ratio:np.float64(1.0011082254350185) - rollout_corr/chi2_token:np.float64(0.0035093913320451975) - rollout_corr/chi2_seq:np.float64(1.2149556565564126) - actor/pg_loss:np.float64(0.009395288652740419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015388868646368792) - actor/ppo_kl:np.float64(4.407900779046514e-05) - actor/pg_clipfrac_lower:np.float64(3.007869418070186e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(178377.74242206803) - self_distillation/token_reweight_w_std:np.float64(2601166.713610011) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996962018776685) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1573005250247661) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.68055609613657) - perf/mfu/actor:np.float64(0.03693197062950951) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.50461959838867) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1512467176955187) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7826571428571428) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06785839628022725) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5356190476190476) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11104188480484187) - val-aux/sciknoweval/score/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/score/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1512467176955187) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7826571428571428) - val-aux/sciknoweval/score/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06785839628022725) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5356190476190476) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11104188480484187) - val-core/sciknoweval/acc/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1512467176955187) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7826571428571428) - val-core/sciknoweval/acc/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06785839628022725) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5356190476190476) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11104188480484187) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999809523809524) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.000300566358820181) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9992904761904762) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0016956593912030376) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9996047619047619) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0013137248630794105) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.002042357581276089) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999285714285715) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0005788212276734866) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980380952380952) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023437861108329258) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9968761904761905) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002262100242179584) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011333633214235306 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011333633214235306 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.23828125 - response_length/max:1220.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.23828125 - response_length_non_aborted/max:1220.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:392.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000131312757730484 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3023900520056486) - timing_s/agent_loop/generate_sequences/max:np.float64(8.421228867024183) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7250610580740613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.421228867024183) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1220 - timing_s/gen:14.119003845378757 - timing_s/reward:0.058908019214868546 - timing_s/old_log_prob:2.3049094192683697 - timing_s/adv:0.5378871113061905 - timing_s/update_actor:15.319367559626698 - timing_s/step:32.428459821268916 - timing_s/testing:112.78920979425311 - timing_s/save_checkpoint:6.872485619038343 - timing_s/stop_profile:0.00016718730330467224 - timing_per_token_ms/adv:0.003351968986571802 - timing_per_token_ms/gen:0.16068242321386106 - timing_per_token_ms/update_actor:0.09546621191399397 - perf/total_num_tokens:160469 - perf/time_per_step:32.428459821268916 - perf/throughput:618.5500363123663 (TaskRunner pid=2026458) Training Progress: 20%|██ | 20/100 [16:02<1:31:32, 68.66s/it] (TaskRunner pid=2026458) step:21 - global_seqlen/min:17387 - global_seqlen/max:23009 - global_seqlen/minmax_diff:5622 - global_seqlen/balanced_min:19296 - global_seqlen/balanced_max:19303 - global_seqlen/mean:19300.625 - actor/entropy:0.2956748902797699 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42548468708992004 - training/rollout_probs_diff_mean:0.005995167884975672 - training/rollout_probs_diff_std:0.014325084164738655 - training/rollout_actor_probs_pearson_corr:0.9982481002807617 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:0.999878466129303 - rollout_corr/rollout_is_ratio_fraction_high:1.2208670341351535e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.325202022911981e-05 - rollout_corr/rollout_is_max:2.6692886352539062 - rollout_corr/rollout_is_min:0.4042225778102875 - rollout_corr/rollout_is_std:0.044611647725105286 - rollout_corr/rollout_is_eff_sample_size:0.9980132789914753 - rollout_corr/rollout_is_seq_mean:0.9997992515563965 - rollout_corr/rollout_is_seq_std:0.0029755644500255585 - rollout_corr/rollout_is_seq_max:1.0099884271621704 - rollout_corr/rollout_is_seq_min:0.9911137819290161 - rollout_corr/rollout_is_seq_max_deviation:0.00998842716217041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3673860565759242) - rollout_corr/training_log_ppl:np.float64(0.3087902419210877) - rollout_corr/kl:np.float64(0.002253606636003269) - rollout_corr/k3_kl:np.float64(0.0017760526020538236) - rollout_corr/rollout_ppl:np.float64(1.36424843640998) - rollout_corr/rollout_log_ppl:np.float64(0.30653663334669545) - rollout_corr/log_ppl_diff:np.float64(0.002253608574392274) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032565151050221175) - rollout_corr/log_ppl_diff_max:np.float64(0.021895110607147217) - rollout_corr/log_ppl_diff_min:np.float64(-0.006019324064254761) - rollout_corr/ppl_ratio:np.float64(1.002263783942908) - rollout_corr/chi2_token:np.float64(0.002287261188030243) - rollout_corr/chi2_seq:np.float64(0.5934691540896893) - actor/pg_loss:np.float64(0.010032993159256876) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001869809015715873) - actor/ppo_kl:np.float64(0.000984730247550747) - actor/pg_clipfrac_lower:np.float64(2.9235126021376345e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(353075.78875684226) - self_distillation/token_reweight_w_std:np.float64(4698832.619528098) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0033803705591708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17246504218201153) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.7826535254716873) - perf/mfu/actor:np.float64(0.03582340340916603) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.28646850585938) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004340182524174452 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004340182524174452 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:319.95703125 - response_length/max:825.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:319.95703125 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.8999285101890564e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3953026235103607) - timing_s/agent_loop/generate_sequences/max:np.float64(6.321475323289633) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4004087638677447) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.321475323289633) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:348 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:12.068734215572476 - timing_s/reward:0.05136189982295036 - timing_s/old_log_prob:2.455154914408922 - timing_s/adv:0.532898997887969 - timing_s/update_actor:15.235047852620482 - timing_s/step:30.448000945150852 - timing_s/stop_profile:0.00011887960135936737 - timing_per_token_ms/adv:0.003451306614992837 - timing_per_token_ms/gen:0.14734320057102976 - timing_per_token_ms/update_actor:0.09866939446663309 - perf/total_num_tokens:154405 - perf/time_per_step:30.448000945150852 - perf/throughput:633.8880846321642 (TaskRunner pid=2026458) Training Progress: 21%|██ | 21/100 [16:33<1:15:19, 57.20s/it] (TaskRunner pid=2026458) step:22 - global_seqlen/min:15143 - global_seqlen/max:18708 - global_seqlen/minmax_diff:3565 - global_seqlen/balanced_min:16965 - global_seqlen/balanced_max:16966 - global_seqlen/mean:16965.375 - actor/entropy:0.31038597226142883 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2530360817909241 - training/rollout_probs_diff_mean:0.0064188591204583645 - training/rollout_probs_diff_std:0.014548459090292454 - training/rollout_actor_probs_pearson_corr:0.9982183575630188 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:1.0001835823059082 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.267485928721726e-05 - rollout_corr/rollout_is_max:1.9917649030685425 - rollout_corr/rollout_is_min:0.3914262652397156 - rollout_corr/rollout_is_std:0.0448860339820385 - rollout_corr/rollout_is_eff_sample_size:0.9979898885301821 - rollout_corr/rollout_is_seq_mean:1.0002237558364868 - rollout_corr/rollout_is_seq_std:0.0030402897391468287 - rollout_corr/rollout_is_seq_max:1.0082043409347534 - rollout_corr/rollout_is_seq_min:0.9909875988960266 - rollout_corr/rollout_is_seq_max_deviation:0.009012401103973389 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3948297398164868) - rollout_corr/training_log_ppl:np.float64(0.32873041287530214) - rollout_corr/kl:np.float64(0.0012066373147963105) - rollout_corr/k3_kl:np.float64(0.0013965993947522293) - rollout_corr/rollout_ppl:np.float64(1.3931263061240315) - rollout_corr/rollout_log_ppl:np.float64(0.32752377312863246) - rollout_corr/log_ppl_diff:np.float64(0.0012066397466696799) - rollout_corr/log_ppl_abs_diff:np.float64(0.002761501877103001) - rollout_corr/log_ppl_diff_max:np.float64(0.011306077241897583) - rollout_corr/log_ppl_diff_min:np.float64(-0.009509235620498657) - rollout_corr/ppl_ratio:np.float64(1.0012132688425481) - rollout_corr/chi2_token:np.float64(0.003218145342543721) - rollout_corr/chi2_seq:np.float64(1.0662864169571549) - actor/pg_loss:np.float64(0.006237241672351956) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020408303180374787) - actor/ppo_kl:np.float64(0.0003451099556370796) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(326844.6732725387) - self_distillation/token_reweight_w_std:np.float64(4298094.462517299) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025643010158092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1402466203435324) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.6192770302295685) - perf/mfu/actor:np.float64(0.03161979996245398) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.7820701599121) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003102818038314581 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003102818038314581 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.60546875 - response_length/max:646.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.60546875 - response_length_non_aborted/max:646.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012409500777721405 - timing_s/agent_loop/generate_sequences/min:np.float64(1.460020076483488) - timing_s/agent_loop/generate_sequences/max:np.float64(5.160831518471241) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.023891647448181) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.160831518471241) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:646 - timing_s/gen:10.945641493424773 - timing_s/reward:0.05562561750411987 - timing_s/old_log_prob:2.356655828654766 - timing_s/adv:0.4728220421820879 - timing_s/update_actor:15.27205784060061 - timing_s/step:29.190949207171798 - timing_s/stop_profile:0.0001829788088798523 - timing_per_token_ms/adv:0.003483728197741635 - timing_per_token_ms/gen:0.15570124032240534 - timing_per_token_ms/update_actor:0.11252372730193563 - perf/total_num_tokens:135723 - perf/time_per_step:29.190949207171798 - perf/throughput:581.1861368259944 (TaskRunner pid=2026458) Training Progress: 22%|██▏ | 22/100 [17:02<1:03:27, 48.81s/it] (TaskRunner pid=2026458) step:23 - global_seqlen/min:15206 - global_seqlen/max:25332 - global_seqlen/minmax_diff:10126 - global_seqlen/balanced_min:18411 - global_seqlen/balanced_max:25520 - global_seqlen/mean:19300.375 - actor/entropy:0.264564573764801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4770219326019287 - training/rollout_probs_diff_mean:0.0052871485240757465 - training/rollout_probs_diff_std:0.013520004227757454 - training/rollout_actor_probs_pearson_corr:0.998356282711029 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999374747276306 - rollout_corr/rollout_is_ratio_fraction_high:3.601138087105937e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.0018966905772686e-05 - rollout_corr/rollout_is_max:2.1046066284179688 - rollout_corr/rollout_is_min:0.36882883310317993 - rollout_corr/rollout_is_std:0.041905514895915985 - rollout_corr/rollout_is_eff_sample_size:0.9982467687554272 - rollout_corr/rollout_is_seq_mean:0.9998911023139954 - rollout_corr/rollout_is_seq_std:0.0030532306991517544 - rollout_corr/rollout_is_seq_max:1.0095295906066895 - rollout_corr/rollout_is_seq_min:0.9908995628356934 - rollout_corr/rollout_is_seq_max_deviation:0.009529590606689453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3668543784879148) - rollout_corr/training_log_ppl:np.float64(0.30833037884258374) - rollout_corr/kl:np.float64(0.0014712483040568713) - rollout_corr/k3_kl:np.float64(0.0014252641368557306) - rollout_corr/rollout_ppl:np.float64(1.364856572356075) - rollout_corr/rollout_log_ppl:np.float64(0.30685912866283616) - rollout_corr/log_ppl_diff:np.float64(0.0014712501797475852) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029776765513815917) - rollout_corr/log_ppl_diff_max:np.float64(0.01450282335281372) - rollout_corr/log_ppl_diff_min:np.float64(-0.008856803178787231) - rollout_corr/ppl_ratio:np.float64(1.0014787362888455) - rollout_corr/chi2_token:np.float64(0.003022435586899519) - rollout_corr/chi2_seq:np.float64(1.3533644466660917) - actor/pg_loss:np.float64(0.007422412978485227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020335467470431468) - actor/ppo_kl:np.float64(0.00035265562636155323) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(345985.51773530385) - self_distillation/token_reweight_w_std:np.float64(4731856.136310147) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988346283789724) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16046884056413546) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.7161731868982315) - perf/mfu/actor:np.float64(0.03389860046243484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1311912536621) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008347137831151485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008347137831151485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.41796875 - response_length/max:8192.0 - response_length/min:126.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:325.41796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010981783270835876 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5618780367076397) - timing_s/agent_loop/generate_sequences/max:np.float64(49.69344619102776) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4613353605309385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.69344619102776) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:230 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.576194578781724 - timing_s/reward:0.05657547526061535 - timing_s/old_log_prob:2.56242498382926 - timing_s/adv:0.5329509787261486 - timing_s/update_actor:16.533158656209707 - timing_s/step:75.34719325602055 - timing_s/stop_profile:0.00011706724762916565 - timing_per_token_ms/adv:0.0034516879770869 - timing_per_token_ms/gen:0.6671251464916721 - timing_per_token_ms/update_actor:0.1070779625798055 - perf/total_num_tokens:154403 - perf/time_per_step:75.34719325602055 - perf/throughput:256.1525408706292 (TaskRunner pid=2026458) Training Progress: 23%|██▎ | 23/100 [18:17<1:12:52, 56.79s/it] (TaskRunner pid=2026458) step:24 - global_seqlen/min:16231 - global_seqlen/max:19060 - global_seqlen/minmax_diff:2829 - global_seqlen/balanced_min:17704 - global_seqlen/balanced_max:17708 - global_seqlen/mean:17706.25 - actor/entropy:0.29770269989967346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5271099209785461 - training/rollout_probs_diff_mean:0.0061988383531570435 - training/rollout_probs_diff_std:0.014574181288480759 - training/rollout_actor_probs_pearson_corr:0.9981999397277832 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.000082015991211 - rollout_corr/rollout_is_ratio_fraction_high:2.8622949685086496e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011449179874034598 - rollout_corr/rollout_is_max:2.555720567703247 - rollout_corr/rollout_is_min:0.15318159759044647 - rollout_corr/rollout_is_std:0.04580353945493698 - rollout_corr/rollout_is_eff_sample_size:0.997906784084707 - rollout_corr/rollout_is_seq_mean:1.000064492225647 - rollout_corr/rollout_is_seq_std:0.0029369699768722057 - rollout_corr/rollout_is_seq_max:1.0110825300216675 - rollout_corr/rollout_is_seq_min:0.9918853640556335 - rollout_corr/rollout_is_seq_max_deviation:0.01108253002166748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36986547941342) - rollout_corr/training_log_ppl:np.float64(0.31092395351151936) - rollout_corr/kl:np.float64(0.0015822704315340275) - rollout_corr/k3_kl:np.float64(0.0015203862882344765) - rollout_corr/rollout_ppl:np.float64(1.3676151768304408) - rollout_corr/rollout_log_ppl:np.float64(0.3093416829360649) - rollout_corr/log_ppl_diff:np.float64(0.0015822705754544586) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029158030229154974) - rollout_corr/log_ppl_diff_max:np.float64(0.013341128826141357) - rollout_corr/log_ppl_diff_min:np.float64(-0.009624361991882324) - rollout_corr/ppl_ratio:np.float64(1.0015899618156254) - rollout_corr/chi2_token:np.float64(0.0028670839965343475) - rollout_corr/chi2_seq:np.float64(0.6717999817337841) - actor/pg_loss:np.float64(0.0065518481424078345) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001698189102626202) - actor/ppo_kl:np.float64(0.000539918119613958) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(259653.62712082197) - self_distillation/token_reweight_w_std:np.float64(3592355.994360844) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997991549316794) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1357375134830363) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.6166152656078339) - perf/mfu/actor:np.float64(0.03283040904184733) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1153450012207) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003980379085987806 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003980379085987806 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.9453125 - response_length/max:645.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.9453125 - response_length_non_aborted/max:645.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011000782251358032 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4089452624320984) - timing_s/agent_loop/generate_sequences/max:np.float64(5.263506431132555) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0320224383394816) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.263506431132555) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:11.14209609106183 - timing_s/reward:0.05296533182263374 - timing_s/old_log_prob:2.2817067373543978 - timing_s/adv:0.506876740604639 - timing_s/update_actor:15.04843096062541 - timing_s/step:29.098722390830517 - timing_s/stop_profile:0.00011877715587615967 - timing_per_token_ms/adv:0.003578374448320784 - timing_per_token_ms/gen:0.15945982899307082 - timing_per_token_ms/update_actor:0.10623671698288323 - perf/total_num_tokens:141650 - perf/time_per_step:29.098722390830517 - perf/throughput:608.4889144679262 (TaskRunner pid=2026458) Training Progress: 24%|██▍ | 24/100 [18:46<1:01:25, 48.49s/it] (TaskRunner pid=2026458) step:25 - global_seqlen/min:15795 - global_seqlen/max:22433 - global_seqlen/minmax_diff:6638 - global_seqlen/balanced_min:18523 - global_seqlen/balanced_max:18527 - global_seqlen/mean:18525.875 - actor/entropy:0.2758452296257019 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25533485412597656 - training/rollout_probs_diff_mean:0.005769852548837662 - training/rollout_probs_diff_std:0.013884530402719975 - training/rollout_actor_probs_pearson_corr:0.9983004331588745 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.78515625 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78515625 - rollout_corr/rollout_is_mean:1.0001115798950195 - rollout_corr/rollout_is_ratio_fraction_high:1.414367125107674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.828734250215348e-05 - rollout_corr/rollout_is_max:2.6026298999786377 - rollout_corr/rollout_is_min:0.3732539415359497 - rollout_corr/rollout_is_std:0.04336366429924965 - rollout_corr/rollout_is_eff_sample_size:0.9981233594863631 - rollout_corr/rollout_is_seq_mean:1.000120759010315 - rollout_corr/rollout_is_seq_std:0.003118946449831128 - rollout_corr/rollout_is_seq_max:1.0095044374465942 - rollout_corr/rollout_is_seq_min:0.990989625453949 - rollout_corr/rollout_is_seq_max_deviation:0.009504437446594238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3535170801915228) - rollout_corr/training_log_ppl:np.float64(0.2983562831650488) - rollout_corr/kl:np.float64(0.000989430469498842) - rollout_corr/k3_kl:np.float64(0.0011073322453967194) - rollout_corr/rollout_ppl:np.float64(1.3521247510798275) - rollout_corr/rollout_log_ppl:np.float64(0.29736685164971277) - rollout_corr/log_ppl_diff:np.float64(0.0009894315153360367) - rollout_corr/log_ppl_abs_diff:np.float64(0.002447564620524645) - rollout_corr/log_ppl_diff_max:np.float64(0.01045176386833191) - rollout_corr/log_ppl_diff_min:np.float64(-0.008554220199584961) - rollout_corr/ppl_ratio:np.float64(1.0009945430792868) - rollout_corr/chi2_token:np.float64(0.002472628140822053) - rollout_corr/chi2_seq:np.float64(0.6732058441266418) - actor/pg_loss:np.float64(0.005739172571338713) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010865399485737726) - actor/ppo_kl:np.float64(0.0001121951680964628) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(393425.2932063611) - self_distillation/token_reweight_w_std:np.float64(5274741.3204780845) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018663464579731) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12906738638412207) - self_distillation/empty_target_batch:np.float64(0.21484375) - actor/grad_norm:np.float64(0.6542618125677109) - perf/mfu/actor:np.float64(0.034142301102186606) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1241111755371) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005874927621334791 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005874927621334791 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:276.18359375 - response_length/max:807.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:276.18359375 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014953315258026123 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3853862024843693) - timing_s/agent_loop/generate_sequences/max:np.float64(5.845100851729512) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.959879736532457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.845100851729512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:807 - timing_s/gen:11.540503876283765 - timing_s/reward:0.05447870306670666 - timing_s/old_log_prob:2.3400896415114403 - timing_s/adv:0.5113725643604994 - timing_s/update_actor:15.447554072365165 - timing_s/step:29.938440999016166 - timing_s/stop_profile:0.00012017786502838135 - timing_per_token_ms/adv:0.0034503941403611123 - timing_per_token_ms/gen:0.16322509478075561 - timing_per_token_ms/update_actor:0.10422958478590866 - perf/total_num_tokens:148207 - perf/time_per_step:29.938440999016166 - perf/throughput:618.7989214471387 (TaskRunner pid=2026458) Training Progress: 25%|██▌ | 25/100 [19:16<53:40, 42.94s/it] (TaskRunner pid=2026458) step:26 - global_seqlen/min:16232 - global_seqlen/max:20091 - global_seqlen/minmax_diff:3859 - global_seqlen/balanced_min:18014 - global_seqlen/balanced_max:18018 - global_seqlen/mean:18016.5 - actor/entropy:0.28236204385757446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797455430030823 - training/rollout_probs_diff_mean:0.005817539524286985 - training/rollout_probs_diff_std:0.014095457270741463 - training/rollout_actor_probs_pearson_corr:0.9982751607894897 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000920295715332 - rollout_corr/rollout_is_ratio_fraction_high:1.4651584933744743e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.790950960246846e-05 - rollout_corr/rollout_is_max:2.0633111000061035 - rollout_corr/rollout_is_min:0.2806716561317444 - rollout_corr/rollout_is_std:0.044501952826976776 - rollout_corr/rollout_is_eff_sample_size:0.9980238466152811 - rollout_corr/rollout_is_seq_mean:1.0001037120819092 - rollout_corr/rollout_is_seq_std:0.0033811419270932674 - rollout_corr/rollout_is_seq_max:1.010282039642334 - rollout_corr/rollout_is_seq_min:0.9834394454956055 - rollout_corr/rollout_is_seq_max_deviation:0.01656055450439453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3481695111840963) - rollout_corr/training_log_ppl:np.float64(0.294715576863382) - rollout_corr/kl:np.float64(0.00122398861193318) - rollout_corr/k3_kl:np.float64(0.0011840371912512637) - rollout_corr/rollout_ppl:np.float64(1.3464941019192338) - rollout_corr/rollout_log_ppl:np.float64(0.2934915872756392) - rollout_corr/log_ppl_diff:np.float64(0.0012239895877428353) - rollout_corr/log_ppl_abs_diff:np.float64(0.002889484982006252) - rollout_corr/log_ppl_diff_max:np.float64(0.026334106922149658) - rollout_corr/log_ppl_diff_min:np.float64(-0.00859677791595459) - rollout_corr/ppl_ratio:np.float64(1.0012316501233727) - rollout_corr/chi2_token:np.float64(0.002297403523698449) - rollout_corr/chi2_seq:np.float64(1.2100839058402926) - actor/pg_loss:np.float64(0.004832616308704019) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006622324299314641) - actor/ppo_kl:np.float64(0.00027824501646955824) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(57253.49309117114) - self_distillation/token_reweight_w_std:np.float64(780711.6470303052) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9960764090064913) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10869685839861631) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.5899917781352997) - perf/mfu/actor:np.float64(0.03340075709542584) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.18029022216797) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0030969788786023855 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0030969788786023855 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:266.609375 - response_length/max:611.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.609375 - response_length_non_aborted/max:611.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014692358672618866 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2525565940886736) - timing_s/agent_loop/generate_sequences/max:np.float64(4.980492563918233) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.909446031990228) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.980492563918233) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:406 - timing_s/agent_loop/slowest/response_length:611 - timing_s/gen:10.703484870493412 - timing_s/reward:0.056105511263012886 - timing_s/old_log_prob:2.315383156761527 - timing_s/adv:0.49405837804079056 - timing_s/update_actor:15.426462525501847 - timing_s/step:29.082023203372955 - timing_s/stop_profile:0.0001284964382648468 - timing_per_token_ms/adv:0.003427818791391159 - timing_per_token_ms/gen:0.156823021603666 - timing_per_token_ms/update_actor:0.10703010105668309 - perf/total_num_tokens:144132 - perf/time_per_step:29.082023203372955 - perf/throughput:619.5064172120746 (TaskRunner pid=2026458) Training Progress: 26%|██▌ | 26/100 [19:46<47:51, 38.80s/it] (TaskRunner pid=2026458) step:27 - global_seqlen/min:14932 - global_seqlen/max:19899 - global_seqlen/minmax_diff:4967 - global_seqlen/balanced_min:17014 - global_seqlen/balanced_max:17065 - global_seqlen/mean:17022.125 - actor/entropy:0.27002352476119995 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583366632461548 - training/rollout_probs_diff_mean:0.005672288592904806 - training/rollout_probs_diff_std:0.013904700987040997 - training/rollout_actor_probs_pearson_corr:0.9982785582542419 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:1.0000253915786743 - rollout_corr/rollout_is_ratio_fraction_high:5.697275264537893e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4243188161344733e-05 - rollout_corr/rollout_is_max:2.935737371444702 - rollout_corr/rollout_is_min:0.4851767420768738 - rollout_corr/rollout_is_std:0.04337328299880028 - rollout_corr/rollout_is_eff_sample_size:0.9981222906269569 - rollout_corr/rollout_is_seq_mean:0.9999536275863647 - rollout_corr/rollout_is_seq_std:0.002799020381644368 - rollout_corr/rollout_is_seq_max:1.0089412927627563 - rollout_corr/rollout_is_seq_min:0.991137683391571 - rollout_corr/rollout_is_seq_max_deviation:0.008941292762756348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3306819731369615) - rollout_corr/training_log_ppl:np.float64(0.28223902196623385) - rollout_corr/kl:np.float64(0.0010320211754617503) - rollout_corr/k3_kl:np.float64(0.0012100190330102123) - rollout_corr/rollout_ppl:np.float64(1.3292772802524269) - rollout_corr/rollout_log_ppl:np.float64(0.28120700089493766) - rollout_corr/log_ppl_diff:np.float64(0.0010320210712961853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025203333934769034) - rollout_corr/log_ppl_diff_max:np.float64(0.011266171932220459) - rollout_corr/log_ppl_diff_min:np.float64(-0.00728076696395874) - rollout_corr/ppl_ratio:np.float64(1.0010370928794146) - rollout_corr/chi2_token:np.float64(0.002755629364401102) - rollout_corr/chi2_seq:np.float64(0.8209603114519268) - actor/pg_loss:np.float64(0.0054146344773471355) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001441872691430035) - actor/ppo_kl:np.float64(-3.021463783170475e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(149585.24241724727) - self_distillation/token_reweight_w_std:np.float64(2245182.637069527) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022315112873912) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10743028059368953) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.5845905467867851) - perf/mfu/actor:np.float64(0.031846668272466384) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.01116561889648) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008261048933491111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008261048933491111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.25390625 - response_length/max:877.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.25390625 - response_length_non_aborted/max:877.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001650247722864151 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4051890634000301) - timing_s/agent_loop/generate_sequences/max:np.float64(6.235670423135161) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9710693403030746) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.235670423135161) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:877 - timing_s/gen:12.108512347564101 - timing_s/reward:0.05487024039030075 - timing_s/old_log_prob:2.2881970033049583 - timing_s/adv:0.46778740733861923 - timing_s/update_actor:15.23558795452118 - timing_s/step:30.242075925692916 - timing_s/stop_profile:0.0001263841986656189 - timing_per_token_ms/adv:0.003435142552256396 - timing_per_token_ms/gen:0.17246382013081088 - timing_per_token_ms/update_actor:0.11188077248376142 - perf/total_num_tokens:136177 - perf/time_per_step:30.242075925692916 - perf/throughput:562.8623194328543 (TaskRunner pid=2026458) Training Progress: 27%|██▋ | 27/100 [20:16<44:06, 36.25s/it] (TaskRunner pid=2026458) step:28 - global_seqlen/min:15064 - global_seqlen/max:18912 - global_seqlen/minmax_diff:3848 - global_seqlen/balanced_min:17172 - global_seqlen/balanced_max:17302 - global_seqlen/mean:17191.25 - actor/entropy:0.26276373863220215 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491259455680847 - training/rollout_probs_diff_mean:0.005732021760195494 - training/rollout_probs_diff_std:0.014186778105795383 - training/rollout_actor_probs_pearson_corr:0.9981420636177063 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999222159385681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4750567970622797e-05 - rollout_corr/rollout_is_max:1.8682423830032349 - rollout_corr/rollout_is_min:0.4175392985343933 - rollout_corr/rollout_is_std:0.043118301779031754 - rollout_corr/rollout_is_eff_sample_size:0.9981441433187315 - rollout_corr/rollout_is_seq_mean:0.9998924732208252 - rollout_corr/rollout_is_seq_std:0.0026770406402647495 - rollout_corr/rollout_is_seq_max:1.0065722465515137 - rollout_corr/rollout_is_seq_min:0.9909303188323975 - rollout_corr/rollout_is_seq_max_deviation:0.009069681167602539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3176847058348358) - rollout_corr/training_log_ppl:np.float64(0.2727040337776998) - rollout_corr/kl:np.float64(0.0012852028201848498) - rollout_corr/k3_kl:np.float64(0.001238845826378565) - rollout_corr/rollout_ppl:np.float64(1.315975757315755) - rollout_corr/rollout_log_ppl:np.float64(0.27141882907017134) - rollout_corr/log_ppl_diff:np.float64(0.0012852047075284645) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026650355284800753) - rollout_corr/log_ppl_diff_max:np.float64(0.011441320180892944) - rollout_corr/log_ppl_diff_min:np.float64(-0.011876940727233887) - rollout_corr/ppl_ratio:np.float64(1.0012910023797303) - rollout_corr/chi2_token:np.float64(0.002501809736713767) - rollout_corr/chi2_seq:np.float64(1.4115001251921058) - actor/pg_loss:np.float64(0.005721935303881764) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001080054133581143) - actor/ppo_kl:np.float64(0.00020560735394870022) - actor/pg_clipfrac_lower:np.float64(3.0048076951061375e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(263679.60727620986) - self_distillation/token_reweight_w_std:np.float64(3103030.912776855) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007451251149178) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10346215419122018) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6124091595411301) - perf/mfu/actor:np.float64(0.03192732787611963) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.95586776733398) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00606801500543952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00606801500543952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:264.8203125 - response_length/max:1012.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:264.8203125 - response_length_non_aborted/max:1012.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001549404114484787 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3163175880908966) - timing_s/agent_loop/generate_sequences/max:np.float64(6.851215856149793) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.895652609608078) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.851215856149793) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:255 - timing_s/agent_loop/slowest/response_length:1012 - timing_s/gen:12.757756907492876 - timing_s/reward:0.05424255505204201 - timing_s/old_log_prob:2.304752092808485 - timing_s/adv:0.5271155349910259 - timing_s/update_actor:15.178545750677586 - timing_s/step:30.909547282382846 - timing_s/stop_profile:0.00012030825018882751 - timing_per_token_ms/adv:0.0038327312949249324 - timing_per_token_ms/gen:0.18818415947565972 - timing_per_token_ms/update_actor:0.11036534392988864 - perf/total_num_tokens:137530 - perf/time_per_step:30.909547282382846 - perf/throughput:556.179288002652 (TaskRunner pid=2026458) Training Progress: 28%|██▊ | 28/100 [20:47<41:35, 34.66s/it] (TaskRunner pid=2026458) step:29 - global_seqlen/min:14445 - global_seqlen/max:22059 - global_seqlen/minmax_diff:7614 - global_seqlen/balanced_min:19100 - global_seqlen/balanced_max:20098 - global_seqlen/mean:19231.25 - actor/entropy:0.2399301379919052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5590871572494507 - training/rollout_probs_diff_mean:0.005236997734755278 - training/rollout_probs_diff_std:0.014018451794981956 - training/rollout_actor_probs_pearson_corr:0.9981542229652405 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.34375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.0000776052474976 - rollout_corr/rollout_is_ratio_fraction_high:2.4095805201795883e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.819161040359177e-05 - rollout_corr/rollout_is_max:3.816781997680664 - rollout_corr/rollout_is_min:0.17573890089988708 - rollout_corr/rollout_is_std:0.04291044920682907 - rollout_corr/rollout_is_eff_sample_size:0.9981623150356519 - rollout_corr/rollout_is_seq_mean:0.9999245405197144 - rollout_corr/rollout_is_seq_std:0.0027032301295548677 - rollout_corr/rollout_is_seq_max:1.008720874786377 - rollout_corr/rollout_is_seq_min:0.992788553237915 - rollout_corr/rollout_is_seq_max_deviation:0.008720874786376953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3173197288997471) - rollout_corr/training_log_ppl:np.float64(0.27068548381794244) - rollout_corr/kl:np.float64(0.0009471329129642214) - rollout_corr/k3_kl:np.float64(0.001118057121630045) - rollout_corr/rollout_ppl:np.float64(1.3160036713816226) - rollout_corr/rollout_log_ppl:np.float64(0.2697383483755402) - rollout_corr/log_ppl_diff:np.float64(0.0009471354424022138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024268297711387277) - rollout_corr/log_ppl_diff_max:np.float64(0.010676145553588867) - rollout_corr/log_ppl_diff_min:np.float64(-0.008322864770889282) - rollout_corr/ppl_ratio:np.float64(1.0009521523024887) - rollout_corr/chi2_token:np.float64(0.0025950868148356676) - rollout_corr/chi2_seq:np.float64(1.0168882303405553) - actor/pg_loss:np.float64(0.004738333867862821) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009898098714984371) - actor/ppo_kl:np.float64(-0.0001235502958607526) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.34375) - self_distillation/token_reweight_w_mean:np.float64(148981.53039407637) - self_distillation/token_reweight_w_std:np.float64(2396815.2922347095) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019751288928092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.088682441302808) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5395381674170494) - perf/mfu/actor:np.float64(0.0350041185371077) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9663429260254) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014469530433416367 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014469530433416367 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:324.2265625 - response_length/max:2192.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.2265625 - response_length_non_aborted/max:2192.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000128256157040596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1863264609128237) - timing_s/agent_loop/generate_sequences/max:np.float64(12.72119532711804) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.387789061780495) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.72119532711804) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:2192 - timing_s/gen:18.591661918908358 - timing_s/reward:0.05764943175017834 - timing_s/old_log_prob:2.3708045054227114 - timing_s/adv:0.6288922913372517 - timing_s/update_actor:15.632321024313569 - timing_s/step:37.36768543533981 - timing_s/stop_profile:0.00012124888598918915 - timing_per_token_ms/adv:0.004087697701249605 - timing_per_token_ms/gen:0.2239905293716821 - timing_per_token_ms/update_actor:0.10160754646937646 - perf/total_num_tokens:153850 - perf/time_per_step:37.36768543533981 - perf/throughput:514.6492156512428 (TaskRunner pid=2026458) Training Progress: 29%|██▉ | 29/100 [21:24<41:59, 35.49s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a key fragment: "Cc2ccc(Cl)cc2C(F)(F)F", which is a benzene ring with a chlorine substituent and a trifluoromethyl group. This fragment is likely derived from a reactant that contains a similar structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A includes a structure with a benzene ring, a chlorine substituent, and a carboxylic acid group, but it does not contain the trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a structure with a benzene ring and a chlorine substituent, but it lacks the trifluoromethyl group and has a different functional group arrangement. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a structure with a benzene ring, a chlorine substituent, and a trifluoromethyl group, and it matches the required fragment in the product. It also includes a carboxylic acid group, which is consistent with the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a structure with a benzene ring and a trifluoromethyl group, but it lacks the chlorine substituent and has a different functional group arrangement. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, Option C is the only correct reactant that matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) C (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 0.0 (TaskRunner pid=2026458) [acc] 0.0 (TaskRunner pid=2026458) [pred] C (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2026458) step:30 - global_seqlen/min:15398 - global_seqlen/max:27317 - global_seqlen/minmax_diff:11919 - global_seqlen/balanced_min:18528 - global_seqlen/balanced_max:25696 - global_seqlen/mean:20322.875 - actor/entropy:0.21429221332073212 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29036805033683777 - training/rollout_probs_diff_mean:0.0045964838936924934 - training/rollout_probs_diff_std:0.012965062633156776 - training/rollout_actor_probs_pearson_corr:0.9982911944389343 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:0.9999114871025085 - rollout_corr/rollout_is_ratio_fraction_high:1.198825157189276e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.99412560404744e-05 - rollout_corr/rollout_is_max:2.0840299129486084 - rollout_corr/rollout_is_min:0.03206612169742584 - rollout_corr/rollout_is_std:0.04015055671334267 - rollout_corr/rollout_is_eff_sample_size:0.9983904085296447 - rollout_corr/rollout_is_seq_mean:0.9999212622642517 - rollout_corr/rollout_is_seq_std:0.003086200449615717 - rollout_corr/rollout_is_seq_max:1.0128618478775024 - rollout_corr/rollout_is_seq_min:0.9903483986854553 - rollout_corr/rollout_is_seq_max_deviation:0.012861847877502441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3321797889657319) - rollout_corr/training_log_ppl:np.float64(0.28194430993880815) - rollout_corr/kl:np.float64(0.0011635422050204625) - rollout_corr/k3_kl:np.float64(0.0012278429346110897) - rollout_corr/rollout_ppl:np.float64(1.3305623242631555) - rollout_corr/rollout_log_ppl:np.float64(0.2807807675008007) - rollout_corr/log_ppl_diff:np.float64(0.0011635424380074255) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028164471932541346) - rollout_corr/log_ppl_diff_max:np.float64(0.01170569658279419) - rollout_corr/log_ppl_diff_min:np.float64(-0.010286390781402588) - rollout_corr/ppl_ratio:np.float64(1.0011705257929862) - rollout_corr/chi2_token:np.float64(0.0025684768334031105) - rollout_corr/chi2_seq:np.float64(0.56981481006369) - actor/pg_loss:np.float64(0.004291482153348625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010325129969714908) - actor/ppo_kl:np.float64(2.040402626590776e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(97715.03854049812) - self_distillation/token_reweight_w_std:np.float64(1362849.7468459418) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998477473855019) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08768687405972742) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5216539651155472) - perf/mfu/actor:np.float64(0.03586380732978027) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9196662902832) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/reward/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10296365255394938) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7833380952380953) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0895080232788665) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6183523809523809) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07343617415290264) - val-aux/sciknoweval/score/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/score/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10296365255394938) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7833380952380953) - val-aux/sciknoweval/score/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0895080232788665) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6183523809523809) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07343617415290264) - val-core/sciknoweval/acc/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/acc/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10296365255394938) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7833380952380953) - val-core/sciknoweval/acc/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0895080232788665) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6183523809523809) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07343617415290264) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.08896331489086151 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.08896331489086151 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.83984375 - response_length/max:8192.0 - response_length/min:95.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:325.83984375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:309.25 - prompt_length/max:516.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012153573334217072 - timing_s/agent_loop/generate_sequences/min:np.float64(0.691241292282939) - timing_s/agent_loop/generate_sequences/max:np.float64(49.47414601966739) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9101952763448935) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.47414601966739) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:218 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.4048219807446 - timing_s/reward:0.05735262483358383 - timing_s/old_log_prob:2.4137737192213535 - timing_s/adv:0.4771828204393387 - timing_s/update_actor:16.52942644432187 - timing_s/step:74.96756593696773 - timing_s/testing:65.94198310934007 - timing_s/save_checkpoint:6.872431704774499 - timing_s/stop_profile:9.614601731300354e-05 - timing_per_token_ms/adv:0.002935010551160568 - timing_per_token_ms/gen:0.6642069409667878 - timing_per_token_ms/update_actor:0.10166761865829681 - perf/total_num_tokens:162583 - perf/time_per_step:74.96756593696773 - perf/throughput:271.0889001930161 (TaskRunner pid=2026458) Training Progress: 30%|███ | 30/100 [23:52<1:20:43, 69.19s/it] (TaskRunner pid=2026458) step:31 - global_seqlen/min:15697 - global_seqlen/max:20497 - global_seqlen/minmax_diff:4800 - global_seqlen/balanced_min:18142 - global_seqlen/balanced_max:18440 - global_seqlen/mean:18179.75 - actor/entropy:0.24378487467765808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4056262969970703 - training/rollout_probs_diff_mean:0.005305483937263489 - training/rollout_probs_diff_std:0.013869675807654858 - training/rollout_actor_probs_pearson_corr:0.9981986880302429 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0002444982528687 - rollout_corr/rollout_is_ratio_fraction_high:5.420200977823697e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.485351620242e-05 - rollout_corr/rollout_is_max:2.6529436111450195 - rollout_corr/rollout_is_min:0.2698281407356262 - rollout_corr/rollout_is_std:0.04314593970775604 - rollout_corr/rollout_is_eff_sample_size:0.9981427181140134 - rollout_corr/rollout_is_seq_mean:1.0003163814544678 - rollout_corr/rollout_is_seq_std:0.0029239451978355646 - rollout_corr/rollout_is_seq_max:1.0140212774276733 - rollout_corr/rollout_is_seq_min:0.9939373135566711 - rollout_corr/rollout_is_seq_max_deviation:0.01402127742767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3019663002341986) - rollout_corr/training_log_ppl:np.float64(0.2597778052149806) - rollout_corr/kl:np.float64(0.001051263411028458) - rollout_corr/k3_kl:np.float64(0.0012008980110067569) - rollout_corr/rollout_ppl:np.float64(1.3005597270093858) - rollout_corr/rollout_log_ppl:np.float64(0.2587265418842435) - rollout_corr/log_ppl_diff:np.float64(0.0010512633307371289) - rollout_corr/log_ppl_abs_diff:np.float64(0.002555768267484382) - rollout_corr/log_ppl_diff_max:np.float64(0.010198384523391724) - rollout_corr/log_ppl_diff_min:np.float64(-0.010389238595962524) - rollout_corr/ppl_ratio:np.float64(1.00105693587102) - rollout_corr/chi2_token:np.float64(0.002714824862778187) - rollout_corr/chi2_seq:np.float64(1.5057584547903389) - actor/pg_loss:np.float64(0.004452571272850037) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820382315912866) - actor/ppo_kl:np.float64(0.00035900160938950876) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(99177.65471042739) - self_distillation/token_reweight_w_std:np.float64(1529254.068720376) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981596393045038) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09003465421847068) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.5145964473485947) - perf/mfu/actor:np.float64(0.0335625674385555) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.96116638183594) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012466462794691324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012466462794691324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.2734375 - response_length/max:769.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:288.2734375 - response_length_non_aborted/max:769.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.449354648590088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2310489993542433) - timing_s/agent_loop/generate_sequences/max:np.float64(5.724285513162613) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1068192758539226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.724285513162613) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:635 - timing_s/agent_loop/slowest/response_length:769 - timing_s/gen:11.310221493244171 - timing_s/reward:0.053344203159213066 - timing_s/old_log_prob:2.4446758944541216 - timing_s/adv:0.550066627562046 - timing_s/update_actor:15.255971036851406 - timing_s/step:29.709052259102464 - timing_s/stop_profile:0.0001292899250984192 - timing_per_token_ms/adv:0.003782138282718726 - timing_per_token_ms/gen:0.15325918714930176 - timing_per_token_ms/update_actor:0.10489673288171872 - perf/total_num_tokens:145438 - perf/time_per_step:29.709052259102464 - perf/throughput:611.9262856804853 (TaskRunner pid=2026458) Training Progress: 31%|███ | 31/100 [24:22<1:05:57, 57.36s/it] (TaskRunner pid=2026458) step:32 - global_seqlen/min:13741 - global_seqlen/max:20093 - global_seqlen/minmax_diff:6352 - global_seqlen/balanced_min:17020 - global_seqlen/balanced_max:17022 - global_seqlen/mean:17021.5 - actor/entropy:0.2510213851928711 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174631893634796 - training/rollout_probs_diff_mean:0.005626973696053028 - training/rollout_probs_diff_std:0.014290885999798775 - training/rollout_actor_probs_pearson_corr:0.9980513453483582 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0003660917282104 - rollout_corr/rollout_is_ratio_fraction_high:4.35514775745105e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.35514775745105e-05 - rollout_corr/rollout_is_max:2.588045358657837 - rollout_corr/rollout_is_min:0.49156951904296875 - rollout_corr/rollout_is_std:0.04363907128572464 - rollout_corr/rollout_is_eff_sample_size:0.9981006764057321 - rollout_corr/rollout_is_seq_mean:1.0004558563232422 - rollout_corr/rollout_is_seq_std:0.0026327052619308233 - rollout_corr/rollout_is_seq_max:1.0082557201385498 - rollout_corr/rollout_is_seq_min:0.9930630326271057 - rollout_corr/rollout_is_seq_max_deviation:0.008255720138549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3111304240301251) - rollout_corr/training_log_ppl:np.float64(0.2656050497171236) - rollout_corr/kl:np.float64(0.0011632751664905072) - rollout_corr/k3_kl:np.float64(0.0012394887605751137) - rollout_corr/rollout_ppl:np.float64(1.3095909063704312) - rollout_corr/rollout_log_ppl:np.float64(0.2644417754927417) - rollout_corr/log_ppl_diff:np.float64(0.0011632742243818939) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027760078373830765) - rollout_corr/log_ppl_diff_max:np.float64(0.027847379446029663) - rollout_corr/log_ppl_diff_min:np.float64(-0.00688682496547699) - rollout_corr/ppl_ratio:np.float64(1.0011707202065736) - rollout_corr/chi2_token:np.float64(0.00264995195902884) - rollout_corr/chi2_seq:np.float64(1.4071137343998998) - actor/pg_loss:np.float64(0.003960146103054285) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013795204131383798) - actor/ppo_kl:np.float64(0.0005964390661219454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(250012.3793359641) - self_distillation/token_reweight_w_std:np.float64(2860040.7311532954) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007816257420927) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07969121757196262) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.522880308330059) - perf/mfu/actor:np.float64(0.031496540810405246) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.58367156982422) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019834069535136223 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019834069535136223 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.078125 - response_length/max:628.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.078125 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002002175897359848 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3522796910256147) - timing_s/agent_loop/generate_sequences/max:np.float64(4.971057282760739) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.862827809440205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.971057282760739) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:10.714683732017875 - timing_s/reward:0.05403785966336727 - timing_s/old_log_prob:2.382686661556363 - timing_s/adv:0.4786151312291622 - timing_s/update_actor:15.277625290676951 - timing_s/step:28.9955370221287 - timing_s/stop_profile:0.00011957064270973206 - timing_per_token_ms/adv:0.003514783738427593 - timing_per_token_ms/gen:0.1555467703968683 - timing_per_token_ms/update_actor:0.1121935881875639 - perf/total_num_tokens:136172 - perf/time_per_step:28.9955370221287 - perf/throughput:587.0386186332606 (TaskRunner pid=2026458) Training Progress: 32%|███▏ | 32/100 [24:51<55:22, 48.86s/it] (TaskRunner pid=2026458) step:33 - global_seqlen/min:15641 - global_seqlen/max:20620 - global_seqlen/minmax_diff:4979 - global_seqlen/balanced_min:18180 - global_seqlen/balanced_max:18182 - global_seqlen/mean:18181.0 - actor/entropy:0.28735455870628357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9959163069725037 - training/rollout_probs_diff_mean:0.00580747751519084 - training/rollout_probs_diff_std:0.015400327742099762 - training/rollout_actor_probs_pearson_corr:0.9980352520942688 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9997233152389526 - rollout_corr/rollout_is_ratio_fraction_high:1.423528101440752e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014235280104912817 - rollout_corr/rollout_is_max:2.051370859146118 - rollout_corr/rollout_is_min:0.0007158105727285147 - rollout_corr/rollout_is_std:0.04281798005104065 - rollout_corr/rollout_is_eff_sample_size:0.998169085060141 - rollout_corr/rollout_is_seq_mean:0.9997956156730652 - rollout_corr/rollout_is_seq_std:0.003073018742725253 - rollout_corr/rollout_is_seq_max:1.0114701986312866 - rollout_corr/rollout_is_seq_min:0.9903202056884766 - rollout_corr/rollout_is_seq_max_deviation:0.011470198631286621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3598110149614513) - rollout_corr/training_log_ppl:np.float64(0.30384603922721) - rollout_corr/kl:np.float64(0.0014454446327292203) - rollout_corr/k3_kl:np.float64(0.001423622587481077) - rollout_corr/rollout_ppl:np.float64(1.3577961758710444) - rollout_corr/rollout_log_ppl:np.float64(0.3024005940533243) - rollout_corr/log_ppl_diff:np.float64(0.0014454451738856733) - rollout_corr/log_ppl_abs_diff:np.float64(0.002951021713670343) - rollout_corr/log_ppl_diff_max:np.float64(0.02602332830429077) - rollout_corr/log_ppl_diff_min:np.float64(-0.010346651077270508) - rollout_corr/ppl_ratio:np.float64(1.001454961951822) - rollout_corr/chi2_token:np.float64(0.002324617700651288) - rollout_corr/chi2_seq:np.float64(0.6557224565185606) - actor/pg_loss:np.float64(0.0037985126255080104) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009114227987083723) - actor/ppo_kl:np.float64(7.80257138011109e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(186856.52135437424) - self_distillation/token_reweight_w_std:np.float64(2469605.76739274) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999388714786619) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09890055964933708) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5402719005942345) - perf/mfu/actor:np.float64(0.03298787544887611) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.31770038604736) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003960967063903809 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003960967063903809 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.40625 - response_length/max:556.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.40625 - response_length_non_aborted/max:556.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001493934541940689 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3925712388008833) - timing_s/agent_loop/generate_sequences/max:np.float64(4.813172671943903) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.01079661748372) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.813172671943903) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:556 - timing_s/gen:10.570713378489017 - timing_s/reward:0.056401949375867844 - timing_s/old_log_prob:2.26583200879395 - timing_s/adv:0.4962475970387459 - timing_s/update_actor:15.83989292383194 - timing_s/step:29.316704677417874 - timing_s/stop_profile:3.923475742340088e-05 - timing_per_token_ms/adv:0.00341185576315072 - timing_per_token_ms/gen:0.1504770723506579 - timing_per_token_ms/update_actor:0.10890416453874883 - perf/total_num_tokens:145448 - perf/time_per_step:29.316704677417874 - perf/throughput:620.1583772818947 (TaskRunner pid=2026458) Training Progress: 33%|███▎ | 33/100 [25:20<48:01, 43.01s/it] (TaskRunner pid=2026458) step:34 - global_seqlen/min:15048 - global_seqlen/max:22716 - global_seqlen/minmax_diff:7668 - global_seqlen/balanced_min:18406 - global_seqlen/balanced_max:18407 - global_seqlen/mean:18406.75 - actor/entropy:0.2822253406047821 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3892279863357544 - training/rollout_probs_diff_mean:0.00599341094493866 - training/rollout_probs_diff_std:0.014615834690630436 - training/rollout_actor_probs_pearson_corr:0.9981156587600708 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9999809265136719 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012492018868215382 - rollout_corr/rollout_is_max:1.9517436027526855 - rollout_corr/rollout_is_min:0.10854519158601761 - rollout_corr/rollout_is_std:0.04540359228849411 - rollout_corr/rollout_is_eff_sample_size:0.9979426360193112 - rollout_corr/rollout_is_seq_mean:0.9999065399169922 - rollout_corr/rollout_is_seq_std:0.003091080579906702 - rollout_corr/rollout_is_seq_max:1.0100407600402832 - rollout_corr/rollout_is_seq_min:0.9879905581474304 - rollout_corr/rollout_is_seq_max_deviation:0.01200944185256958 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3448055190965533) - rollout_corr/training_log_ppl:np.float64(0.29254834007588215) - rollout_corr/kl:np.float64(0.001122740535322464) - rollout_corr/k3_kl:np.float64(0.0011786174198959998) - rollout_corr/rollout_ppl:np.float64(1.3432867326773703) - rollout_corr/rollout_log_ppl:np.float64(0.2914255977375433) - rollout_corr/log_ppl_diff:np.float64(0.0011227423383388668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025718869001138955) - rollout_corr/log_ppl_diff_max:np.float64(0.014177501201629639) - rollout_corr/log_ppl_diff_min:np.float64(-0.006468132138252258) - rollout_corr/ppl_ratio:np.float64(1.0011283229105175) - rollout_corr/chi2_token:np.float64(0.002466450445353985) - rollout_corr/chi2_seq:np.float64(0.9479083714541048) - actor/pg_loss:np.float64(0.005841488018631935) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011660458308142552) - actor/ppo_kl:np.float64(-8.996075499190237e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(205396.02536904486) - self_distillation/token_reweight_w_std:np.float64(2610298.4065771694) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991585344541818) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12581025002873503) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.6120489835739136) - perf/mfu/actor:np.float64(0.03423697654860308) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.21736526489258) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012471199035644531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012471199035644531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.4296875 - response_length/max:743.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.4296875 - response_length_non_aborted/max:743.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.58306884765625e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4199188239872456) - timing_s/agent_loop/generate_sequences/max:np.float64(5.984945947304368) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2054700024527847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.984945947304368) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:207 - timing_s/agent_loop/slowest/response_length:743 - timing_s/gen:11.809425836429 - timing_s/reward:0.05546233803033829 - timing_s/old_log_prob:2.252874342724681 - timing_s/adv:0.49194580502808094 - timing_s/update_actor:15.391067944467068 - timing_s/step:30.08788885548711 - timing_s/stop_profile:0.00011718645691871643 - timing_per_token_ms/adv:0.003340797567659153 - timing_per_token_ms/gen:0.16391507976055575 - timing_per_token_ms/update_actor:0.10452054235855779 - perf/total_num_tokens:147254 - perf/time_per_step:30.08788885548711 - perf/throughput:611.7660859626305 (TaskRunner pid=2026458) Training Progress: 34%|███▍ | 34/100 [25:50<43:03, 39.14s/it] (TaskRunner pid=2026458) step:35 - global_seqlen/min:14987 - global_seqlen/max:19906 - global_seqlen/minmax_diff:4919 - global_seqlen/balanced_min:16938 - global_seqlen/balanced_max:16939 - global_seqlen/mean:16938.25 - actor/entropy:0.294697642326355 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22079068422317505 - training/rollout_probs_diff_mean:0.005864571314305067 - training/rollout_probs_diff_std:0.013605127111077309 - training/rollout_actor_probs_pearson_corr:0.9984296560287476 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.00005042552948 - rollout_corr/rollout_is_ratio_fraction_high:3.1109037081478164e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.1109037081478164e-05 - rollout_corr/rollout_is_max:2.7191102504730225 - rollout_corr/rollout_is_min:0.37482571601867676 - rollout_corr/rollout_is_std:0.04332791268825531 - rollout_corr/rollout_is_eff_sample_size:0.9981263285522803 - rollout_corr/rollout_is_seq_mean:1.0001163482666016 - rollout_corr/rollout_is_seq_std:0.0030649646650999784 - rollout_corr/rollout_is_seq_max:1.0084450244903564 - rollout_corr/rollout_is_seq_min:0.989280641078949 - rollout_corr/rollout_is_seq_max_deviation:0.010719358921051025 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3609930002130568) - rollout_corr/training_log_ppl:np.float64(0.3051015918608755) - rollout_corr/kl:np.float64(0.0011746971918284999) - rollout_corr/k3_kl:np.float64(0.0011032617354658214) - rollout_corr/rollout_ppl:np.float64(1.3593582352623343) - rollout_corr/rollout_log_ppl:np.float64(0.30392689420841634) - rollout_corr/log_ppl_diff:np.float64(0.0011746976524591446) - rollout_corr/log_ppl_abs_diff:np.float64(0.002850949880667031) - rollout_corr/log_ppl_diff_max:np.float64(0.012322753667831421) - rollout_corr/log_ppl_diff_min:np.float64(-0.010998740792274475) - rollout_corr/ppl_ratio:np.float64(1.0011811451986432) - rollout_corr/chi2_token:np.float64(0.0021009566262364388) - rollout_corr/chi2_seq:np.float64(1.5435644236858934) - actor/pg_loss:np.float64(0.0014815301401540637) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020538957323879004) - actor/ppo_kl:np.float64(0.00030627488916046275) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(94700.63521376462) - self_distillation/token_reweight_w_std:np.float64(1075744.331360741) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004025450907648) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03615615499438718) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.3742603175342083) - perf/mfu/actor:np.float64(0.031170659142369998) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1671371459961) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029106393922120333 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0029106393922120333 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:251.1328125 - response_length/max:502.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:251.1328125 - response_length_non_aborted/max:502.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001547783613204956 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1277420744299889) - timing_s/agent_loop/generate_sequences/max:np.float64(4.495345920324326) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.789818469223974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.495345920324326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:355 - timing_s/agent_loop/slowest/response_length:502 - timing_s/gen:10.292290568351746 - timing_s/reward:0.054091084748506546 - timing_s/old_log_prob:2.2543541602790356 - timing_s/adv:0.4955630674958229 - timing_s/update_actor:15.42586961016059 - timing_s/step:28.60832902789116 - timing_s/stop_profile:3.491714596748352e-05 - timing_per_token_ms/adv:0.003657130071700315 - timing_per_token_ms/gen:0.16009162495491905 - timing_per_token_ms/update_actor:0.11383901532153992 - perf/total_num_tokens:135506 - perf/time_per_step:28.60832902789116 - perf/throughput:592.0740768706333 (TaskRunner pid=2026458) Training Progress: 35%|███▌ | 35/100 [26:19<38:59, 35.99s/it] (TaskRunner pid=2026458) step:36 - global_seqlen/min:14431 - global_seqlen/max:20659 - global_seqlen/minmax_diff:6228 - global_seqlen/balanced_min:17408 - global_seqlen/balanced_max:17413 - global_seqlen/mean:17411.75 - actor/entropy:0.2818928360939026 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30500367283821106 - training/rollout_probs_diff_mean:0.005986991338431835 - training/rollout_probs_diff_std:0.014329608529806137 - training/rollout_actor_probs_pearson_corr:0.9982293248176575 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0000543594360352 - rollout_corr/rollout_is_ratio_fraction_high:1.4678683328384068e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.871473331353627e-05 - rollout_corr/rollout_is_max:3.0542242527008057 - rollout_corr/rollout_is_min:0.18286029994487762 - rollout_corr/rollout_is_std:0.044746387749910355 - rollout_corr/rollout_is_eff_sample_size:0.9980018804571298 - rollout_corr/rollout_is_seq_mean:1.0000989437103271 - rollout_corr/rollout_is_seq_std:0.0029557279776781797 - rollout_corr/rollout_is_seq_max:1.0128530263900757 - rollout_corr/rollout_is_seq_min:0.992638111114502 - rollout_corr/rollout_is_seq_max_deviation:0.012853026390075684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3384348950348794) - rollout_corr/training_log_ppl:np.float64(0.28769120943616144) - rollout_corr/kl:np.float64(0.0010638975770804748) - rollout_corr/k3_kl:np.float64(0.0012365762453327989) - rollout_corr/rollout_ppl:np.float64(1.3369859019294381) - rollout_corr/rollout_log_ppl:np.float64(0.28662730925134383) - rollout_corr/log_ppl_diff:np.float64(0.0010639001848176122) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025838720030151308) - rollout_corr/log_ppl_diff_max:np.float64(0.01416441798210144) - rollout_corr/log_ppl_diff_min:np.float64(-0.009993910789489746) - rollout_corr/ppl_ratio:np.float64(1.0010696118697524) - rollout_corr/chi2_token:np.float64(0.002987496554851532) - rollout_corr/chi2_seq:np.float64(0.8903114418499172) - actor/pg_loss:np.float64(0.0051360129145905375) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000932457276576315) - actor/ppo_kl:np.float64(0.00010139281163645819) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(95758.95235729823) - self_distillation/token_reweight_w_std:np.float64(1392223.179759777) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9973206941504031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10905869025737047) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5497298240661621) - perf/mfu/actor:np.float64(0.03256102036135432) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1156120300293) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007596218958497047 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007596218958497047 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:266.1171875 - response_length/max:662.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.1171875 - response_length_non_aborted/max:662.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2092596888542175e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3356867656111717) - timing_s/agent_loop/generate_sequences/max:np.float64(5.081710338592529) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.919884198134241) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.081710338592529) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:379 - timing_s/agent_loop/slowest/response_length:662 - timing_s/gen:10.837040143087506 - timing_s/reward:0.05424739792943001 - timing_s/old_log_prob:2.3121078684926033 - timing_s/adv:0.5094937365502119 - timing_s/update_actor:15.35438490100205 - timing_s/step:29.153722267597914 - timing_s/stop_profile:3.104284405708313e-05 - timing_per_token_ms/adv:0.0036576861641579098 - timing_per_token_ms/gen:0.1590734835905162 - timing_per_token_ms/update_actor:0.11023005227075143 - perf/total_num_tokens:139294 - perf/time_per_step:29.153722267597914 - perf/throughput:597.2393452945732 (TaskRunner pid=2026458) Training Progress: 36%|███▌ | 36/100 [26:48<36:12, 33.94s/it] (TaskRunner pid=2026458) step:37 - global_seqlen/min:15833 - global_seqlen/max:21178 - global_seqlen/minmax_diff:5345 - global_seqlen/balanced_min:18583 - global_seqlen/balanced_max:18587 - global_seqlen/mean:18585.625 - actor/entropy:0.285804808139801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4652300179004669 - training/rollout_probs_diff_mean:0.005965293850749731 - training/rollout_probs_diff_std:0.014233550056815147 - training/rollout_actor_probs_pearson_corr:0.9982559680938721 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0002163648605347 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9148995963623747e-05 - rollout_corr/rollout_is_max:1.695287823677063 - rollout_corr/rollout_is_min:0.16980920732021332 - rollout_corr/rollout_is_std:0.04377271980047226 - rollout_corr/rollout_is_eff_sample_size:0.9980885633504123 - rollout_corr/rollout_is_seq_mean:1.000288486480713 - rollout_corr/rollout_is_seq_std:0.0028211281169205904 - rollout_corr/rollout_is_seq_max:1.0087376832962036 - rollout_corr/rollout_is_seq_min:0.9919080138206482 - rollout_corr/rollout_is_seq_max_deviation:0.008737683296203613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3575056260451674) - rollout_corr/training_log_ppl:np.float64(0.3016296330606565) - rollout_corr/kl:np.float64(0.0007570147956474926) - rollout_corr/k3_kl:np.float64(0.0012164926273499077) - rollout_corr/rollout_ppl:np.float64(1.3564373790286481) - rollout_corr/rollout_log_ppl:np.float64(0.30087261548032984) - rollout_corr/log_ppl_diff:np.float64(0.0007570175803266466) - rollout_corr/log_ppl_abs_diff:np.float64(0.002560925087891519) - rollout_corr/log_ppl_diff_max:np.float64(0.011357903480529785) - rollout_corr/log_ppl_diff_min:np.float64(-0.008871495723724365) - rollout_corr/ppl_ratio:np.float64(1.000762677518651) - rollout_corr/chi2_token:np.float64(0.0033016859088093042) - rollout_corr/chi2_seq:np.float64(1.0105145464185625) - actor/pg_loss:np.float64(0.0031967939576134086) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006711441328661749) - actor/ppo_kl:np.float64(3.3348735911431504e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(214368.9940047427) - self_distillation/token_reweight_w_std:np.float64(2919033.6983316364) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007499558851123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07807663449784741) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4402663931250572) - perf/mfu/actor:np.float64(0.034623907048585915) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.02531814575195) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031590224243700504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031590224243700504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:268.01953125 - response_length/max:686.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:268.01953125 - response_length_non_aborted/max:686.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.923235297203064e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.457766680046916) - timing_s/agent_loop/generate_sequences/max:np.float64(5.311068296432495) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9142332015690044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.311068296432495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:686 - timing_s/gen:11.104413088411093 - timing_s/reward:0.05539845488965511 - timing_s/old_log_prob:2.2851091288030148 - timing_s/adv:0.5124162267893553 - timing_s/update_actor:15.311178967356682 - timing_s/step:29.35571401193738 - timing_s/stop_profile:0.00011339224874973297 - timing_per_token_ms/adv:0.003446320925374821 - timing_per_token_ms/gen:0.1618412412867983 - timing_per_token_ms/update_actor:0.10297729406030656 - perf/total_num_tokens:148685 - perf/time_per_step:29.35571401193738 - perf/throughput:633.1177975246056 (TaskRunner pid=2026458) Training Progress: 37%|███▋ | 37/100 [27:18<34:12, 32.58s/it] (TaskRunner pid=2026458) step:38 - global_seqlen/min:15292 - global_seqlen/max:19759 - global_seqlen/minmax_diff:4467 - global_seqlen/balanced_min:17036 - global_seqlen/balanced_max:17040 - global_seqlen/mean:17038.625 - actor/entropy:0.2831912040710449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4492228925228119 - training/rollout_probs_diff_mean:0.006005707662552595 - training/rollout_probs_diff_std:0.014447803609073162 - training/rollout_actor_probs_pearson_corr:0.9981924891471863 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998399019241333 - rollout_corr/rollout_is_ratio_fraction_high:2.9010312573518604e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3515468860277906e-05 - rollout_corr/rollout_is_max:4.976550579071045 - rollout_corr/rollout_is_min:0.1723451316356659 - rollout_corr/rollout_is_std:0.045169975608587265 - rollout_corr/rollout_is_eff_sample_size:0.997963056153362 - rollout_corr/rollout_is_seq_mean:0.999801754951477 - rollout_corr/rollout_is_seq_std:0.0029568150639533997 - rollout_corr/rollout_is_seq_max:1.01389741897583 - rollout_corr/rollout_is_seq_min:0.9917296171188354 - rollout_corr/rollout_is_seq_max_deviation:0.013897418975830078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3477132506668568) - rollout_corr/training_log_ppl:np.float64(0.29560438107000664) - rollout_corr/kl:np.float64(0.0015795158470695014) - rollout_corr/k3_kl:np.float64(0.0011915537743334426) - rollout_corr/rollout_ppl:np.float64(1.3455825243145227) - rollout_corr/rollout_log_ppl:np.float64(0.29402486264007166) - rollout_corr/log_ppl_diff:np.float64(0.0015795184299349785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002752322005107999) - rollout_corr/log_ppl_diff_max:np.float64(0.010404467582702637) - rollout_corr/log_ppl_diff_min:np.float64(-0.006978675723075867) - rollout_corr/ppl_ratio:np.float64(1.0015854325611144) - rollout_corr/chi2_token:np.float64(0.0016510849818587303) - rollout_corr/chi2_seq:np.float64(0.42648625490255654) - actor/pg_loss:np.float64(0.004350315313786268) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001112287538489909) - actor/ppo_kl:np.float64(0.0002975818272203945) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(277881.75766049954) - self_distillation/token_reweight_w_std:np.float64(3405953.521727615) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997533587738872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09852648590458557) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5438188463449478) - perf/mfu/actor:np.float64(0.03193194428398073) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.85803985595703) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00237884558737278 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00237884558737278 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.30078125 - response_length/max:549.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.30078125 - response_length_non_aborted/max:549.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013971328735351562 - timing_s/agent_loop/generate_sequences/min:np.float64(1.483020644634962) - timing_s/agent_loop/generate_sequences/max:np.float64(4.714750545099378) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0552332281222334) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.714750545099378) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:549 - timing_s/gen:10.464355397969484 - timing_s/reward:0.05412936769425869 - timing_s/old_log_prob:2.2855528369545937 - timing_s/adv:0.4730773251503706 - timing_s/update_actor:15.16877150721848 - timing_s/step:28.533766953274608 - timing_s/stop_profile:0.00012415647506713867 - timing_per_token_ms/adv:0.003470624281231398 - timing_per_token_ms/gen:0.15178711358943855 - timing_per_token_ms/update_actor:0.11128224480568767 - perf/total_num_tokens:136309 - perf/time_per_step:28.533766953274608 - perf/throughput:597.1389977321099 (TaskRunner pid=2026458) Training Progress: 38%|███▊ | 38/100 [27:46<32:25, 31.38s/it] (TaskRunner pid=2026458) step:39 - global_seqlen/min:15017 - global_seqlen/max:21775 - global_seqlen/minmax_diff:6758 - global_seqlen/balanced_min:17748 - global_seqlen/balanced_max:17751 - global_seqlen/mean:17750.375 - actor/entropy:0.2934376001358032 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.006076316349208355 - training/rollout_probs_diff_std:0.014462748542428017 - training/rollout_actor_probs_pearson_corr:0.9982326030731201 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001943111419678 - rollout_corr/rollout_is_ratio_fraction_high:2.8678357921307907e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.735671584261581e-05 - rollout_corr/rollout_is_max:2.3144755363464355 - rollout_corr/rollout_is_min:0.44314810633659363 - rollout_corr/rollout_is_std:0.04461565613746643 - rollout_corr/rollout_is_eff_sample_size:0.9980141101447896 - rollout_corr/rollout_is_seq_mean:1.0003681182861328 - rollout_corr/rollout_is_seq_std:0.0030234723817557096 - rollout_corr/rollout_is_seq_max:1.012935996055603 - rollout_corr/rollout_is_seq_min:0.991753876209259 - rollout_corr/rollout_is_seq_max_deviation:0.012935996055603027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3561590211465955) - rollout_corr/training_log_ppl:np.float64(0.30050787163781933) - rollout_corr/kl:np.float64(0.0008042539968293738) - rollout_corr/k3_kl:np.float64(0.0011359995904740572) - rollout_corr/rollout_ppl:np.float64(1.355069079902023) - rollout_corr/rollout_log_ppl:np.float64(0.2997036176966503) - rollout_corr/log_ppl_diff:np.float64(0.0008042539411690086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024447946052532643) - rollout_corr/log_ppl_diff_max:np.float64(0.010187238454818726) - rollout_corr/log_ppl_diff_min:np.float64(-0.010493099689483643) - rollout_corr/ppl_ratio:np.float64(1.0008093006908894) - rollout_corr/chi2_token:np.float64(0.0029845864046365023) - rollout_corr/chi2_seq:np.float64(0.9699735399335623) - actor/pg_loss:np.float64(0.003664600197225809) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006868128334645007) - actor/ppo_kl:np.float64(0.00012619273251779362) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(231536.99509470444) - self_distillation/token_reweight_w_std:np.float64(3103805.8869296443) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012708925642073) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08035396633204073) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.45319733768701553) - perf/mfu/actor:np.float64(0.03298738163573484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.00569915771484) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005079654045403004 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005079654045403004 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.41796875 - response_length/max:558.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.41796875 - response_length_non_aborted/max:558.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.198114275932312e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3789098169654608) - timing_s/agent_loop/generate_sequences/max:np.float64(4.869980722665787) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.020734860765515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.869980722665787) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:365 - timing_s/agent_loop/slowest/response_length:558 - timing_s/gen:10.739610413089395 - timing_s/reward:0.054600564762949944 - timing_s/old_log_prob:2.247015619650483 - timing_s/adv:0.5021204594522715 - timing_s/update_actor:15.248395329341292 - timing_s/step:28.878361692652106 - timing_s/stop_profile:0.00011930800974369049 - timing_per_token_ms/adv:0.0035359848697018477 - timing_per_token_ms/gen:0.15399719544429077 - timing_per_token_ms/update_actor:0.10738079709119731 - perf/total_num_tokens:142003 - perf/time_per_step:28.878361692652106 - perf/throughput:614.6600416226678 (TaskRunner pid=2026458) Training Progress: 39%|███▉ | 39/100 [28:15<31:09, 30.65s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a nitrogen atom bonded to a carbon chain, with a substituted aromatic ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. The nitrogen is bonded to a substituted aromatic ring with a chlorine atom and a trifluoromethyl group, matching the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it lacks the specific chlorine and trifluoromethyl groups. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes an oxygen group (Oc2) instead of the chlorine and trifluoromethyl groups. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes a double bond (CC=C) and a different trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Based on the structural analysis, Option A is the only one that matches the product's structure, making it the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2026458) step:40 - global_seqlen/min:16589 - global_seqlen/max:23076 - global_seqlen/minmax_diff:6487 - global_seqlen/balanced_min:19038 - global_seqlen/balanced_max:19042 - global_seqlen/mean:19040.5 - actor/entropy:0.3083128035068512 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48448076844215393 - training/rollout_probs_diff_mean:0.006092434749007225 - training/rollout_probs_diff_std:0.01408663671463728 - training/rollout_actor_probs_pearson_corr:0.9983260631561279 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0004887580871582 - rollout_corr/rollout_is_ratio_fraction_high:2.767170371953398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.383585185976699e-05 - rollout_corr/rollout_is_max:2.5501346588134766 - rollout_corr/rollout_is_min:0.2322635054588318 - rollout_corr/rollout_is_std:0.044838204979896545 - rollout_corr/rollout_is_eff_sample_size:0.9979954688959803 - rollout_corr/rollout_is_seq_mean:1.0004291534423828 - rollout_corr/rollout_is_seq_std:0.00280501926317811 - rollout_corr/rollout_is_seq_max:1.0100523233413696 - rollout_corr/rollout_is_seq_min:0.9925862550735474 - rollout_corr/rollout_is_seq_max_deviation:0.010052323341369629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3748237513937056) - rollout_corr/training_log_ppl:np.float64(0.3138403666380327) - rollout_corr/kl:np.float64(0.0008364543262393553) - rollout_corr/k3_kl:np.float64(0.0011391592736345046) - rollout_corr/rollout_ppl:np.float64(1.3736085868440568) - rollout_corr/rollout_log_ppl:np.float64(0.3130039106181357) - rollout_corr/log_ppl_diff:np.float64(0.0008364560198970139) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023778348113410175) - rollout_corr/log_ppl_diff_max:np.float64(0.009519308805465698) - rollout_corr/log_ppl_diff_min:np.float64(-0.0073702335357666016) - rollout_corr/ppl_ratio:np.float64(1.0008412639144808) - rollout_corr/chi2_token:np.float64(0.0029417627956718206) - rollout_corr/chi2_seq:np.float64(0.8329082692507654) - actor/pg_loss:np.float64(0.0021506232442334294) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047293603620346403) - actor/ppo_kl:np.float64(0.0002316894797989022) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(45010.86430421425) - self_distillation/token_reweight_w_std:np.float64(529273.9801185789) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998293393291533) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05143398354994133) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.38673160597682) - perf/mfu/actor:np.float64(0.03526754309862706) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.86711502075195) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/reward/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09064171674985216) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7905285714285714) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07673490521638164) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6577380952380952) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06656845162631347) - val-aux/sciknoweval/score/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/score/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/score/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09064171674985216) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7905285714285714) - val-aux/sciknoweval/score/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07673490521638164) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6577380952380952) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06656845162631347) - val-core/sciknoweval/acc/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/acc/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09064171674985216) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7905285714285714) - val-core/sciknoweval/acc/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07673490521638164) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6577380952380952) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06656845162631347) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002604287412166728) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994380952380952) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015362291495737217) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997095238095238) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011396681698700574) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9989476190476191) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.001975811782419187) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999333333333333) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0005594782149609339) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980619047619048) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023394041530661654) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00015050933932646775) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9969285714285715) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002278634869993062) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034849049989134073 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034849049989134073 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:282.328125 - response_length/max:680.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:282.328125 - response_length_non_aborted/max:680.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.6875 - prompt_length/max:501.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018454715609550476 - timing_s/agent_loop/generate_sequences/min:np.float64(1.287922067567706) - timing_s/agent_loop/generate_sequences/max:np.float64(5.481446422636509) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0874510004068725) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.481446422636509) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:680 - timing_s/gen:11.202703416347504 - timing_s/reward:0.05725381709635258 - timing_s/old_log_prob:2.3498482443392277 - timing_s/adv:0.48736315965652466 - timing_s/update_actor:15.279724219813943 - timing_s/step:29.463446816429496 - timing_s/testing:112.37399195507169 - timing_s/save_checkpoint:6.7144703436642885 - timing_s/stop_profile:0.00011608190834522247 - timing_per_token_ms/adv:0.003199516554558209 - timing_per_token_ms/gen:0.15499894039995993 - timing_per_token_ms/update_actor:0.10031068130966848 - perf/total_num_tokens:152324 - perf/time_per_step:29.463446816429496 - perf/throughput:646.2414298853378 (TaskRunner pid=2026458) Training Progress: 40%|████ | 40/100 [30:44<1:06:02, 66.04s/it] (TaskRunner pid=2026458) step:41 - global_seqlen/min:14161 - global_seqlen/max:22430 - global_seqlen/minmax_diff:8269 - global_seqlen/balanced_min:17529 - global_seqlen/balanced_max:17569 - global_seqlen/mean:17540.0 - actor/entropy:0.28421929478645325 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3167548179626465 - training/rollout_probs_diff_mean:0.005681131035089493 - training/rollout_probs_diff_std:0.013473561964929104 - training/rollout_actor_probs_pearson_corr:0.9984502792358398 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0002758502960205 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.967482328414917 - rollout_corr/rollout_is_min:0.5226783752441406 - rollout_corr/rollout_is_std:0.04252113774418831 - rollout_corr/rollout_is_eff_sample_size:0.998196166076475 - rollout_corr/rollout_is_seq_mean:1.000309944152832 - rollout_corr/rollout_is_seq_std:0.002816120395436883 - rollout_corr/rollout_is_seq_max:1.0100390911102295 - rollout_corr/rollout_is_seq_min:0.9916365146636963 - rollout_corr/rollout_is_seq_max_deviation:0.010039091110229492 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3460121774114668) - rollout_corr/training_log_ppl:np.float64(0.29434393605333753) - rollout_corr/kl:np.float64(0.0006344841518632904) - rollout_corr/k3_kl:np.float64(0.0010341189710061371) - rollout_corr/rollout_ppl:np.float64(1.3451492232270539) - rollout_corr/rollout_log_ppl:np.float64(0.2937094499648083) - rollout_corr/log_ppl_diff:np.float64(0.0006344860885292292) - rollout_corr/log_ppl_abs_diff:np.float64(0.002157465787604451) - rollout_corr/log_ppl_diff_max:np.float64(0.00896558165550232) - rollout_corr/log_ppl_diff_min:np.float64(-0.008629709482192993) - rollout_corr/ppl_ratio:np.float64(1.000638237921521) - rollout_corr/chi2_token:np.float64(0.0028806456830352545) - rollout_corr/chi2_seq:np.float64(0.6779269557446241) - actor/pg_loss:np.float64(0.004367844783701003) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004992729859623068) - actor/ppo_kl:np.float64(5.57020750591164e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(156417.83956303308) - self_distillation/token_reweight_w_std:np.float64(2357416.5479677515) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000363286351785) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0830252695013769) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.500577874481678) - perf/mfu/actor:np.float64(0.032587394156940674) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.82418060302734) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022402419708669186 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022402419708669186 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.21875 - response_length/max:788.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.21875 - response_length_non_aborted/max:788.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017307698726654053 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2802286799997091) - timing_s/agent_loop/generate_sequences/max:np.float64(5.627637119963765) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.104791782410757) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.627637119963765) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:788 - timing_s/gen:11.781217468902469 - timing_s/reward:0.05423154681921005 - timing_s/old_log_prob:2.465151973068714 - timing_s/adv:0.5479183085262775 - timing_s/update_actor:15.487478408962488 - timing_s/step:30.435431083664298 - timing_s/stop_profile:0.00011926889419555664 - timing_per_token_ms/adv:0.0039047769991895492 - timing_per_token_ms/gen:0.16191887670289265 - timing_per_token_ms/update_actor:0.11037256562829595 - perf/total_num_tokens:140320 - perf/time_per_step:30.435431083664298 - perf/throughput:576.3020064274463 (TaskRunner pid=2026458) Training Progress: 41%|████ | 41/100 [31:14<54:26, 55.37s/it] (TaskRunner pid=2026458) step:42 - global_seqlen/min:15278 - global_seqlen/max:20664 - global_seqlen/minmax_diff:5386 - global_seqlen/balanced_min:18018 - global_seqlen/balanced_max:18114 - global_seqlen/mean:18031.0 - actor/entropy:0.2842845022678375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.289776086807251 - training/rollout_probs_diff_mean:0.005445326212793589 - training/rollout_probs_diff_std:0.013068844564259052 - training/rollout_actor_probs_pearson_corr:0.9985550045967102 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997981786727905 - rollout_corr/rollout_is_ratio_fraction_high:1.2730420166917611e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5460840333835222e-05 - rollout_corr/rollout_is_max:2.036485433578491 - rollout_corr/rollout_is_min:0.43032172322273254 - rollout_corr/rollout_is_std:0.04114258661866188 - rollout_corr/rollout_is_eff_sample_size:0.998309494689457 - rollout_corr/rollout_is_seq_mean:0.999789297580719 - rollout_corr/rollout_is_seq_std:0.0025704929139465094 - rollout_corr/rollout_is_seq_max:1.008382797241211 - rollout_corr/rollout_is_seq_min:0.9924810528755188 - rollout_corr/rollout_is_seq_max_deviation:0.008382797241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3495313432067633) - rollout_corr/training_log_ppl:np.float64(0.2967612352222204) - rollout_corr/kl:np.float64(0.0012985346789022856) - rollout_corr/k3_kl:np.float64(0.0010939320847001) - rollout_corr/rollout_ppl:np.float64(1.3477259748615324) - rollout_corr/rollout_log_ppl:np.float64(0.29546269873389974) - rollout_corr/log_ppl_diff:np.float64(0.0012985364883206785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002426193736027926) - rollout_corr/log_ppl_diff_max:np.float64(0.00859573483467102) - rollout_corr/log_ppl_diff_min:np.float64(-0.006498366594314575) - rollout_corr/ppl_ratio:np.float64(1.0013033242430538) - rollout_corr/chi2_token:np.float64(0.001749418443068862) - rollout_corr/chi2_seq:np.float64(0.668122704140842) - actor/pg_loss:np.float64(0.0077214669436216354) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001590086207670538) - actor/ppo_kl:np.float64(0.0002024844293835315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(176662.73121298593) - self_distillation/token_reweight_w_std:np.float64(2621435.3882032083) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981021112762392) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14360627194400877) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.6128042787313461) - perf/mfu/actor:np.float64(0.03360615388418303) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.27114486694336) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003445170121267438 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003445170121267438 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:306.84375 - response_length/max:965.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:306.84375 - response_length_non_aborted/max:965.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013544969260692596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3969923835247755) - timing_s/agent_loop/generate_sequences/max:np.float64(6.683027235791087) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.339469947764883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.683027235791087) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:965 - timing_s/gen:13.290308959782124 - timing_s/reward:0.05642969720065594 - timing_s/old_log_prob:2.2229634914547205 - timing_s/adv:0.47765903919935226 - timing_s/update_actor:15.349972175434232 - timing_s/step:31.484555166214705 - timing_s/stop_profile:3.08305025100708e-05 - timing_per_token_ms/adv:0.003311373739666077 - timing_per_token_ms/gen:0.16919122313603885 - timing_per_token_ms/update_actor:0.10641376085238084 - perf/total_num_tokens:144248 - perf/time_per_step:31.484555166214705 - perf/throughput:572.6934970117862 (TaskRunner pid=2026458) Training Progress: 42%|████▏ | 42/100 [31:46<46:36, 48.21s/it] (TaskRunner pid=2026458) step:43 - global_seqlen/min:17746 - global_seqlen/max:21005 - global_seqlen/minmax_diff:3259 - global_seqlen/balanced_min:19023 - global_seqlen/balanced_max:19027 - global_seqlen/mean:19025.375 - actor/entropy:0.29123467206954956 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.277299165725708 - training/rollout_probs_diff_mean:0.0057848598808050156 - training/rollout_probs_diff_std:0.013805463910102844 - training/rollout_actor_probs_pearson_corr:0.9984029531478882 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998417496681213 - rollout_corr/rollout_is_ratio_fraction_high:1.3395668247540016e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.697834032820538e-05 - rollout_corr/rollout_is_max:2.0824544429779053 - rollout_corr/rollout_is_min:0.3074188232421875 - rollout_corr/rollout_is_std:0.0433402918279171 - rollout_corr/rollout_is_eff_sample_size:0.9981245471106103 - rollout_corr/rollout_is_seq_mean:0.9999098181724548 - rollout_corr/rollout_is_seq_std:0.0025561333168298006 - rollout_corr/rollout_is_seq_max:1.0072154998779297 - rollout_corr/rollout_is_seq_min:0.9910159111022949 - rollout_corr/rollout_is_seq_max_deviation:0.008984088897705078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3661411092616618) - rollout_corr/training_log_ppl:np.float64(0.30844287818763405) - rollout_corr/kl:np.float64(0.001555773359804391) - rollout_corr/k3_kl:np.float64(0.0014595502659631165) - rollout_corr/rollout_ppl:np.float64(1.3639800301752985) - rollout_corr/rollout_log_ppl:np.float64(0.30688710472895764) - rollout_corr/log_ppl_diff:np.float64(0.0015557734586764127) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026391229184810072) - rollout_corr/log_ppl_diff_max:np.float64(0.011147260665893555) - rollout_corr/log_ppl_diff_min:np.float64(-0.006583303213119507) - rollout_corr/ppl_ratio:np.float64(1.0015615709125996) - rollout_corr/chi2_token:np.float64(0.0026983728166669607) - rollout_corr/chi2_seq:np.float64(0.5271497289650142) - actor/pg_loss:np.float64(0.0038334736600518227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013588511892521637) - actor/ppo_kl:np.float64(0.00048035153239256445) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(120698.68936810177) - self_distillation/token_reweight_w_std:np.float64(1773706.5877801427) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000403418438509) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07702912960667163) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.46645358949899673) - perf/mfu/actor:np.float64(0.03541705134193752) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77776336669922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00016074800805654377 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00016074800805654377 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.60546875 - response_length/max:660.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.60546875 - response_length_non_aborted/max:660.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.972890019416809e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.429830538108945) - timing_s/agent_loop/generate_sequences/max:np.float64(5.387127343565226) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1587736944129574) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.387127343565226) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:660 - timing_s/gen:11.481718624010682 - timing_s/reward:0.056908562779426575 - timing_s/old_log_prob:2.2525762263685465 - timing_s/adv:0.5500077605247498 - timing_s/update_actor:15.172130836173892 - timing_s/step:29.60062885284424 - timing_s/stop_profile:0.00011467933654785156 - timing_per_token_ms/adv:0.0036136459894006673 - timing_per_token_ms/gen:0.15380528893130277 - timing_per_token_ms/update_actor:0.09968352027341046 - perf/total_num_tokens:152203 - perf/time_per_step:29.60062885284424 - perf/throughput:642.7355004713661 (TaskRunner pid=2026458) Training Progress: 43%|████▎ | 43/100 [32:15<40:30, 42.64s/it] (TaskRunner pid=2026458) step:44 - global_seqlen/min:16682 - global_seqlen/max:21465 - global_seqlen/minmax_diff:4783 - global_seqlen/balanced_min:19287 - global_seqlen/balanced_max:19289 - global_seqlen/mean:19287.75 - actor/entropy:0.2704760432243347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26803696155548096 - training/rollout_probs_diff_mean:0.005588939879089594 - training/rollout_probs_diff_std:0.013731644488871098 - training/rollout_actor_probs_pearson_corr:0.9983340501785278 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000927448272705 - rollout_corr/rollout_is_ratio_fraction_high:2.68305120698642e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.049153257161379e-05 - rollout_corr/rollout_is_max:2.0968687534332275 - rollout_corr/rollout_is_min:0.22089464962482452 - rollout_corr/rollout_is_std:0.04294377565383911 - rollout_corr/rollout_is_eff_sample_size:0.9981595832973885 - rollout_corr/rollout_is_seq_mean:1.0000972747802734 - rollout_corr/rollout_is_seq_std:0.00284771085716784 - rollout_corr/rollout_is_seq_max:1.0088731050491333 - rollout_corr/rollout_is_seq_min:0.9906178116798401 - rollout_corr/rollout_is_seq_max_deviation:0.009382188320159912 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3369123949669302) - rollout_corr/training_log_ppl:np.float64(0.2868449309025891) - rollout_corr/kl:np.float64(0.0009355866875218055) - rollout_corr/k3_kl:np.float64(0.0010745994904937106) - rollout_corr/rollout_ppl:np.float64(1.335645436309278) - rollout_corr/rollout_log_ppl:np.float64(0.28590934313251637) - rollout_corr/log_ppl_diff:np.float64(0.0009355877700727433) - rollout_corr/log_ppl_abs_diff:np.float64(0.002383651997661218) - rollout_corr/log_ppl_diff_max:np.float64(0.011576682329177856) - rollout_corr/log_ppl_diff_min:np.float64(-0.006330162286758423) - rollout_corr/ppl_ratio:np.float64(1.00094054476358) - rollout_corr/chi2_token:np.float64(0.00241070962511003) - rollout_corr/chi2_seq:np.float64(0.8048715838231146) - actor/pg_loss:np.float64(0.0025291290367022157) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006139230463304557) - actor/ppo_kl:np.float64(5.7461318307527876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(186553.62624621042) - self_distillation/token_reweight_w_std:np.float64(2401896.473150722) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013129995204508) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05587577508413233) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.40642886608839035) - perf/mfu/actor:np.float64(0.03559055533852201) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.5995101928711) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029178180266171694 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029178180266171694 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.1796875 - response_length/max:674.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.1796875 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001512337476015091 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5085251331329346) - timing_s/agent_loop/generate_sequences/max:np.float64(5.416358392685652) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1938710649119457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.416358392685652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:634 - timing_s/gen:11.832684468477964 - timing_s/reward:0.057033905759453773 - timing_s/old_log_prob:2.2766003496944904 - timing_s/adv:0.5443154312670231 - timing_s/update_actor:15.36143240518868 - timing_s/step:30.159771636128426 - timing_s/stop_profile:0.0001326482743024826 - timing_per_token_ms/adv:0.003527598030271954 - timing_per_token_ms/gen:0.1587384892876226 - timing_per_token_ms/update_actor:0.0995543311505274 - perf/total_num_tokens:154302 - perf/time_per_step:30.159771636128426 - perf/throughput:639.5190995708728 (TaskRunner pid=2026458) Training Progress: 44%|████▍ | 44/100 [32:46<36:19, 38.91s/it] (TaskRunner pid=2026458) step:45 - global_seqlen/min:15729 - global_seqlen/max:24872 - global_seqlen/minmax_diff:9143 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18456.25 - actor/entropy:0.28802311420440674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29810237884521484 - training/rollout_probs_diff_mean:0.005760597065091133 - training/rollout_probs_diff_std:0.013836598955094814 - training/rollout_actor_probs_pearson_corr:0.9983675479888916 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:1.000093698501587 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.471605279832147e-05 - rollout_corr/rollout_is_max:1.7223904132843018 - rollout_corr/rollout_is_min:0.3712316155433655 - rollout_corr/rollout_is_std:0.04281449690461159 - rollout_corr/rollout_is_eff_sample_size:0.9981707478872216 - rollout_corr/rollout_is_seq_mean:1.000179648399353 - rollout_corr/rollout_is_seq_std:0.002977126045152545 - rollout_corr/rollout_is_seq_max:1.008954644203186 - rollout_corr/rollout_is_seq_min:0.9924071431159973 - rollout_corr/rollout_is_seq_max_deviation:0.008954644203186035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3517238958738744) - rollout_corr/training_log_ppl:np.float64(0.2982309421640821) - rollout_corr/kl:np.float64(0.0008551144521504739) - rollout_corr/k3_kl:np.float64(0.0011088182236562716) - rollout_corr/rollout_ppl:np.float64(1.350550597999245) - rollout_corr/rollout_log_ppl:np.float64(0.29737582663074136) - rollout_corr/log_ppl_diff:np.float64(0.0008551155333407223) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025770837091840804) - rollout_corr/log_ppl_diff_max:np.float64(0.009607642889022827) - rollout_corr/log_ppl_diff_min:np.float64(-0.007143139839172363) - rollout_corr/ppl_ratio:np.float64(1.000860691536218) - rollout_corr/chi2_token:np.float64(0.002733499277383089) - rollout_corr/chi2_seq:np.float64(0.8130777420010418) - actor/pg_loss:np.float64(0.002860508277080953) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007193945029939641) - actor/ppo_kl:np.float64(8.01470097968604e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(108369.78586033895) - self_distillation/token_reweight_w_std:np.float64(1441947.6739233742) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9984894776716828) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07410955597879365) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.417113721370697) - perf/mfu/actor:np.float64(0.03440527561058198) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.48589324951172) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00041734983096830547 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00041734983096830547 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:262.0703125 - response_length/max:648.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.0703125 - response_length_non_aborted/max:648.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.201839566230774e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3249220605939627) - timing_s/agent_loop/generate_sequences/max:np.float64(5.273081086575985) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.846716286490846) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.273081086575985) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:418 - timing_s/agent_loop/slowest/response_length:648 - timing_s/gen:11.049067402258515 - timing_s/reward:0.0605863519012928 - timing_s/old_log_prob:2.241894433274865 - timing_s/adv:0.5028043333441019 - timing_s/update_actor:15.405420746654272 - timing_s/step:29.348894773051143 - timing_s/stop_profile:0.00011529400944709778 - timing_per_token_ms/adv:0.0034053798397839616 - timing_per_token_ms/gen:0.16469022808553457 - timing_per_token_ms/update_actor:0.1043374246302355 - perf/total_num_tokens:147650 - perf/time_per_step:29.348894773051143 - perf/throughput:628.8567301330532 (TaskRunner pid=2026458) Training Progress: 45%|████▌ | 45/100 [33:15<33:03, 36.06s/it] (TaskRunner pid=2026458) step:46 - global_seqlen/min:16485 - global_seqlen/max:20444 - global_seqlen/minmax_diff:3959 - global_seqlen/balanced_min:17819 - global_seqlen/balanced_max:17823 - global_seqlen/mean:17821.125 - actor/entropy:0.29167261719703674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032017827033997 - training/rollout_probs_diff_mean:0.005424427799880505 - training/rollout_probs_diff_std:0.013135196641087532 - training/rollout_actor_probs_pearson_corr:0.9985377788543701 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998041391372681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7731173759093508e-05 - rollout_corr/rollout_is_max:1.7920777797698975 - rollout_corr/rollout_is_min:0.19150957465171814 - rollout_corr/rollout_is_std:0.03981738165020943 - rollout_corr/rollout_is_eff_sample_size:0.9984166697314456 - rollout_corr/rollout_is_seq_mean:0.9997422695159912 - rollout_corr/rollout_is_seq_std:0.0028411736711859703 - rollout_corr/rollout_is_seq_max:1.0074349641799927 - rollout_corr/rollout_is_seq_min:0.9918615221977234 - rollout_corr/rollout_is_seq_max_deviation:0.008138477802276611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3577020564116538) - rollout_corr/training_log_ppl:np.float64(0.3013367626699619) - rollout_corr/kl:np.float64(0.0011281410026526828) - rollout_corr/k3_kl:np.float64(0.000916888057986398) - rollout_corr/rollout_ppl:np.float64(1.3561205281876028) - rollout_corr/rollout_log_ppl:np.float64(0.3002086222113576) - rollout_corr/log_ppl_diff:np.float64(0.0011281404586043209) - rollout_corr/log_ppl_abs_diff:np.float64(0.00256365115637891) - rollout_corr/log_ppl_diff_max:np.float64(0.008949041366577148) - rollout_corr/log_ppl_diff_min:np.float64(-0.009114667773246765) - rollout_corr/ppl_ratio:np.float64(1.0011336989700794) - rollout_corr/chi2_token:np.float64(0.0014092018827795982) - rollout_corr/chi2_seq:np.float64(0.8297181797679514) - actor/pg_loss:np.float64(0.005181095330044627) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006262393026190693) - actor/ppo_kl:np.float64(4.9855951789634645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(97563.27732459153) - self_distillation/token_reweight_w_std:np.float64(1470502.0369112268) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995444938540459) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10749436303740367) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.4946809858083725) - perf/mfu/actor:np.float64(0.03312182529882031) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.53104400634766) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002026108792051673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002026108792051673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.72265625 - response_length/max:593.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.72265625 - response_length_non_aborted/max:593.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.71662050485611e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3297640141099691) - timing_s/agent_loop/generate_sequences/max:np.float64(5.09482192248106) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.099804258738004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.09482192248106) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:593 - timing_s/gen:11.312629146501422 - timing_s/reward:0.05594059266149998 - timing_s/old_log_prob:2.290945826098323 - timing_s/adv:0.4917367957532406 - timing_s/update_actor:15.34849851578474 - timing_s/step:29.586808905005455 - timing_s/stop_profile:0.00011978484690189362 - timing_per_token_ms/adv:0.0034491144340862362 - timing_per_token_ms/gen:0.15685624362531608 - timing_per_token_ms/update_actor:0.10765663303933352 - perf/total_num_tokens:142569 - perf/time_per_step:29.586808905005455 - perf/throughput:602.3334607398315 (TaskRunner pid=2026458) Training Progress: 46%|████▌ | 46/100 [33:45<30:43, 34.13s/it] (TaskRunner pid=2026458) step:47 - global_seqlen/min:14094 - global_seqlen/max:21506 - global_seqlen/minmax_diff:7412 - global_seqlen/balanced_min:18641 - global_seqlen/balanced_max:18644 - global_seqlen/mean:18642.625 - actor/entropy:0.27096572518348694 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7320154905319214 - training/rollout_probs_diff_mean:0.005300942808389664 - training/rollout_probs_diff_std:0.015477053821086884 - training/rollout_actor_probs_pearson_corr:0.9979209303855896 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0002624988555908 - rollout_corr/rollout_is_ratio_fraction_high:1.350238289887784e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017553097859490663 - rollout_corr/rollout_is_max:5.250345230102539 - rollout_corr/rollout_is_min:0.1398765593767166 - rollout_corr/rollout_is_std:0.041101276874542236 - rollout_corr/rollout_is_eff_sample_size:0.9983143657831011 - rollout_corr/rollout_is_seq_mean:1.0002628564834595 - rollout_corr/rollout_is_seq_std:0.0028906052466481924 - rollout_corr/rollout_is_seq_max:1.0110986232757568 - rollout_corr/rollout_is_seq_min:0.9913266897201538 - rollout_corr/rollout_is_seq_max_deviation:0.011098623275756836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.33038782607764) - rollout_corr/training_log_ppl:np.float64(0.2808810631977394) - rollout_corr/kl:np.float64(0.0008357813613741882) - rollout_corr/k3_kl:np.float64(0.0010391420327664491) - rollout_corr/rollout_ppl:np.float64(1.329236305784434) - rollout_corr/rollout_log_ppl:np.float64(0.28004528186284006) - rollout_corr/log_ppl_diff:np.float64(0.0008357813348993659) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023171580396592617) - rollout_corr/log_ppl_diff_max:np.float64(0.011047214269638062) - rollout_corr/log_ppl_diff_min:np.float64(-0.006683439016342163) - rollout_corr/ppl_ratio:np.float64(1.0008402362000197) - rollout_corr/chi2_token:np.float64(0.002344296546652913) - rollout_corr/chi2_seq:np.float64(0.7856939071789384) - actor/pg_loss:np.float64(0.0035682206507772207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00080771961438586) - actor/ppo_kl:np.float64(0.000137977912149978) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(180196.61803952185) - self_distillation/token_reweight_w_std:np.float64(1850028.0704001258) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998111962340772) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09769563036388718) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4677669107913971) - perf/mfu/actor:np.float64(0.03410809941893227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.31404495239258) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003762101521715522 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003762101521715522 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:289.30078125 - response_length/max:738.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.30078125 - response_length_non_aborted/max:738.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.999237954616547e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5701810363680124) - timing_s/agent_loop/generate_sequences/max:np.float64(5.741097932681441) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.086432929841976) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.741097932681441) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:738 - timing_s/gen:12.44257121346891 - timing_s/reward:0.11020655557513237 - timing_s/old_log_prob:3.6139515712857246 - timing_s/adv:0.5418413802981377 - timing_s/update_actor:15.522931521758437 - timing_s/step:32.423666927963495 - timing_s/stop_profile:0.00011558458209037781 - timing_per_token_ms/adv:0.0036330813143142238 - timing_per_token_ms/gen:0.16800436415210313 - timing_per_token_ms/update_actor:0.10408225452262247 - perf/total_num_tokens:149141 - perf/time_per_step:32.423666927963495 - perf/throughput:574.9696677250851 (TaskRunner pid=2026458) Training Progress: 47%|████▋ | 47/100 [34:17<29:42, 33.63s/it] (TaskRunner pid=2026458) step:48 - global_seqlen/min:14172 - global_seqlen/max:18922 - global_seqlen/minmax_diff:4750 - global_seqlen/balanced_min:17273 - global_seqlen/balanced_max:17276 - global_seqlen/mean:17275.125 - actor/entropy:0.28335925936698914 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2923179566860199 - training/rollout_probs_diff_mean:0.005517815239727497 - training/rollout_probs_diff_std:0.0134055744856596 - training/rollout_actor_probs_pearson_corr:0.9984236359596252 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999392032623291 - rollout_corr/rollout_is_ratio_fraction_high:1.4249682863010094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.8499365726020187e-05 - rollout_corr/rollout_is_max:2.0768961906433105 - rollout_corr/rollout_is_min:0.30299994349479675 - rollout_corr/rollout_is_std:0.04144066944718361 - rollout_corr/rollout_is_eff_sample_size:0.9982854963367315 - rollout_corr/rollout_is_seq_mean:0.9998867511749268 - rollout_corr/rollout_is_seq_std:0.002759289462119341 - rollout_corr/rollout_is_seq_max:1.0119271278381348 - rollout_corr/rollout_is_seq_min:0.9927634000778198 - rollout_corr/rollout_is_seq_max_deviation:0.011927127838134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3371913330629468) - rollout_corr/training_log_ppl:np.float64(0.28653379750903696) - rollout_corr/kl:np.float64(0.0009971100477166317) - rollout_corr/k3_kl:np.float64(0.0009379864269476457) - rollout_corr/rollout_ppl:np.float64(1.3358645238913596) - rollout_corr/rollout_log_ppl:np.float64(0.28553668738459237) - rollout_corr/log_ppl_diff:np.float64(0.000997110124444589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023510434839408845) - rollout_corr/log_ppl_diff_max:np.float64(0.009961530566215515) - rollout_corr/log_ppl_diff_min:np.float64(-0.00725024938583374) - rollout_corr/ppl_ratio:np.float64(1.0010016579180956) - rollout_corr/chi2_token:np.float64(0.0017369601409882307) - rollout_corr/chi2_seq:np.float64(0.5906375367194414) - actor/pg_loss:np.float64(0.0036190334940329194) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005808295049973822) - actor/ppo_kl:np.float64(-3.3437734998642554e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(104527.51239224058) - self_distillation/token_reweight_w_std:np.float64(1541202.450584725) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000157124362886) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09064709232188761) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.42223525047302246) - perf/mfu/actor:np.float64(0.031950463769025786) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.0163230895996) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019717998802661896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019717998802661896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.12890625 - response_length/max:573.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.12890625 - response_length_non_aborted/max:573.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012047216296195984 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2815109174698591) - timing_s/agent_loop/generate_sequences/max:np.float64(4.910025550052524) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0799834246499813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.910025550052524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:573 - timing_s/gen:10.97082038037479 - timing_s/reward:0.05485686659812927 - timing_s/old_log_prob:2.286929327994585 - timing_s/adv:0.5000464376062155 - timing_s/update_actor:15.382348308339715 - timing_s/step:29.284351360052824 - timing_s/stop_profile:0.0001140497624874115 - timing_per_token_ms/adv:0.0036182548433529096 - timing_per_token_ms/gen:0.15633071206199736 - timing_per_token_ms/update_actor:0.11130417513867277 - perf/total_num_tokens:138201 - perf/time_per_step:29.284351360052824 - perf/throughput:589.9097708397677 (TaskRunner pid=2026458) Training Progress: 48%|████▊ | 48/100 [34:46<28:01, 32.34s/it] (TaskRunner pid=2026458) step:49 - global_seqlen/min:17224 - global_seqlen/max:19716 - global_seqlen/minmax_diff:2492 - global_seqlen/balanced_min:18546 - global_seqlen/balanced_max:18550 - global_seqlen/mean:18548.75 - actor/entropy:0.27977773547172546 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3583575189113617 - training/rollout_probs_diff_mean:0.005447280127555132 - training/rollout_probs_diff_std:0.013272756710648537 - training/rollout_actor_probs_pearson_corr:0.9984596967697144 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998903274536133 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.59693042607978e-05 - rollout_corr/rollout_is_max:1.740144968032837 - rollout_corr/rollout_is_min:0.2820626199245453 - rollout_corr/rollout_is_std:0.04080290347337723 - rollout_corr/rollout_is_eff_sample_size:0.9983375338303951 - rollout_corr/rollout_is_seq_mean:0.9998434782028198 - rollout_corr/rollout_is_seq_std:0.0024030529893934727 - rollout_corr/rollout_is_seq_max:1.0073314905166626 - rollout_corr/rollout_is_seq_min:0.9888448715209961 - rollout_corr/rollout_is_seq_max_deviation:0.011155128479003906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3396195233799517) - rollout_corr/training_log_ppl:np.float64(0.28772373546962626) - rollout_corr/kl:np.float64(0.0007274425354566461) - rollout_corr/k3_kl:np.float64(0.0009657581270516857) - rollout_corr/rollout_ppl:np.float64(1.338642387650907) - rollout_corr/rollout_log_ppl:np.float64(0.2869962909608148) - rollout_corr/log_ppl_diff:np.float64(0.0007274445088114589) - rollout_corr/log_ppl_abs_diff:np.float64(0.002191261184634641) - rollout_corr/log_ppl_diff_max:np.float64(0.011016637086868286) - rollout_corr/log_ppl_diff_min:np.float64(-0.009674102067947388) - rollout_corr/ppl_ratio:np.float64(1.0007315212860703) - rollout_corr/chi2_token:np.float64(0.002396710915490985) - rollout_corr/chi2_seq:np.float64(1.0418477742932737) - actor/pg_loss:np.float64(0.003264832077547908) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005003044952900382) - actor/ppo_kl:np.float64(-0.0002956932246789279) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(133482.32340499433) - self_distillation/token_reweight_w_std:np.float64(1837865.312099801) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993252530694008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07909147051395848) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.44828861951828003) - perf/mfu/actor:np.float64(0.03315053423820138) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(151.27535438537598) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005617484916001558 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005617484916001558 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:300.8359375 - response_length/max:722.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.8359375 - response_length_non_aborted/max:722.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001618172973394394 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2382317576557398) - timing_s/agent_loop/generate_sequences/max:np.float64(5.747685985639691) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2742665582336485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.747685985639691) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:358 - timing_s/agent_loop/slowest/response_length:722 - timing_s/gen:11.949256183579564 - timing_s/reward:0.0566436555236578 - timing_s/old_log_prob:2.331390691921115 - timing_s/adv:0.5100956819951534 - timing_s/update_actor:16.06329415179789 - timing_s/step:30.9992211535573 - timing_s/stop_profile:0.00011705979704856873 - timing_per_token_ms/adv:0.003437534079083182 - timing_per_token_ms/gen:0.15515693488949495 - timing_per_token_ms/update_actor:0.1082505165563575 - perf/total_num_tokens:148390 - perf/time_per_step:30.9992211535573 - perf/throughput:598.3618074827485 (TaskRunner pid=2026458) Training Progress: 49%|████▉ | 49/100 [35:17<27:09, 31.95s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options matches the structure of the reactant that would form this product through a chemical reaction. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a structure with a substituted benzene ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F). The correct reactant should include these key functional groups and match the overall structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a structure with a substituted benzene ring (c2ccccc2) and a nitro group (N), which is consistent with the product's structure. It also includes a chlorine atom and a trifluoromethyl group, making it a strong candidate for the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a chlorine atom and a substituted benzene ring, but it lacks the trifluoromethyl group and does not match the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (an oxygen atom) that is not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (a double bond) that is not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains the necessary functional groups and matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2026458) step:50 - global_seqlen/min:15714 - global_seqlen/max:24076 - global_seqlen/minmax_diff:8362 - global_seqlen/balanced_min:18055 - global_seqlen/balanced_max:18058 - global_seqlen/mean:18056.5 - actor/entropy:0.24377241730690002 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36303967237472534 - training/rollout_probs_diff_mean:0.005148114170879126 - training/rollout_probs_diff_std:0.013693224638700485 - training/rollout_actor_probs_pearson_corr:0.9982551336288452 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:2.7458949261927046e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.864737224532291e-05 - rollout_corr/rollout_is_max:2.6678342819213867 - rollout_corr/rollout_is_min:0.17930372059345245 - rollout_corr/rollout_is_std:0.04035785421729088 - rollout_corr/rollout_is_eff_sample_size:0.9983741296547397 - rollout_corr/rollout_is_seq_mean:1.0000826120376587 - rollout_corr/rollout_is_seq_std:0.002955564297735691 - rollout_corr/rollout_is_seq_max:1.0125819444656372 - rollout_corr/rollout_is_seq_min:0.9909312725067139 - rollout_corr/rollout_is_seq_max_deviation:0.012581944465637207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3021241263486445) - rollout_corr/training_log_ppl:np.float64(0.26025468575244304) - rollout_corr/kl:np.float64(0.0007726539755310569) - rollout_corr/k3_kl:np.float64(0.0010959870580791176) - rollout_corr/rollout_ppl:np.float64(1.3010970023460686) - rollout_corr/rollout_log_ppl:np.float64(0.2594820303056622) - rollout_corr/log_ppl_diff:np.float64(0.0007726554467808455) - rollout_corr/log_ppl_abs_diff:np.float64(0.00248320066020824) - rollout_corr/log_ppl_diff_max:np.float64(0.010783776640892029) - rollout_corr/log_ppl_diff_min:np.float64(-0.014668971300125122) - rollout_corr/ppl_ratio:np.float64(1.0007780094165355) - rollout_corr/chi2_token:np.float64(0.002904584165662527) - rollout_corr/chi2_seq:np.float64(1.7037486750632524) - actor/pg_loss:np.float64(0.00390949088614434) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012861561140198319) - actor/ppo_kl:np.float64(-2.874969411337247e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(303182.6307455059) - self_distillation/token_reweight_w_std:np.float64(3853282.430746559) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001452625496313) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08799264291883446) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.9146713092923164) - perf/mfu/actor:np.float64(0.03308314844340515) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.47300720214844) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08550843930681744) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8024761904761903) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05941417072300563) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6757285714285716) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06458122232163772) - val-aux/sciknoweval/score/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/score/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08550843930681744) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8024761904761903) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05941417072300563) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6757285714285716) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06458122232163772) - val-core/sciknoweval/acc/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/acc/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08550843930681744) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8024761904761903) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05941417072300563) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6757285714285716) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06458122232163772) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00038099291850812733 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00038099291850812733 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:284.515625 - response_length/max:754.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.515625 - response_length_non_aborted/max:754.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.75 - prompt_length/max:900.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001567751169204712 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4153082128614187) - timing_s/agent_loop/generate_sequences/max:np.float64(6.037689724937081) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1098132879196783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.037689724937081) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:216 - timing_s/agent_loop/slowest/response_length:754 - timing_s/gen:12.157338604331017 - timing_s/reward:0.07574324123561382 - timing_s/old_log_prob:2.3888504561036825 - timing_s/adv:0.52120690792799 - timing_s/update_actor:15.507932955399156 - timing_s/step:30.738427244126797 - timing_s/testing:64.3304407633841 - timing_s/save_checkpoint:6.778321286663413 - timing_s/stop_profile:0.00014533661305904388 - timing_per_token_ms/adv:0.0036081667815467417 - timing_per_token_ms/gen:0.16691386957453755 - timing_per_token_ms/update_actor:0.10735699717137288 - perf/total_num_tokens:144452 - perf/time_per_step:30.738427244126797 - perf/throughput:587.4243290521658 (TaskRunner pid=2026458) Training Progress: 50%|█████ | 50/100 [36:59<44:06, 52.94s/it] (TaskRunner pid=2026458) step:51 - global_seqlen/min:17642 - global_seqlen/max:21320 - global_seqlen/minmax_diff:3678 - global_seqlen/balanced_min:19266 - global_seqlen/balanced_max:19274 - global_seqlen/mean:19272.125 - actor/entropy:0.27418410778045654 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3186328411102295 - training/rollout_probs_diff_mean:0.005429690238088369 - training/rollout_probs_diff_std:0.013169948011636734 - training/rollout_actor_probs_pearson_corr:0.9984568953514099 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999269247055054 - rollout_corr/rollout_is_ratio_fraction_high:2.6965442884829827e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6965442884829827e-05 - rollout_corr/rollout_is_max:2.9393632411956787 - rollout_corr/rollout_is_min:0.18107180297374725 - rollout_corr/rollout_is_std:0.04137301445007324 - rollout_corr/rollout_is_eff_sample_size:0.9982909612066878 - rollout_corr/rollout_is_seq_mean:0.9999714493751526 - rollout_corr/rollout_is_seq_std:0.0026941560208797455 - rollout_corr/rollout_is_seq_max:1.0119987726211548 - rollout_corr/rollout_is_seq_min:0.9906357526779175 - rollout_corr/rollout_is_seq_max_deviation:0.011998772621154785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3326939661055803) - rollout_corr/training_log_ppl:np.float64(0.283328754856484) - rollout_corr/kl:np.float64(0.001354399174752885) - rollout_corr/k3_kl:np.float64(0.0011438059663078093) - rollout_corr/rollout_ppl:np.float64(1.3308947938494384) - rollout_corr/rollout_log_ppl:np.float64(0.2819743549771374) - rollout_corr/log_ppl_diff:np.float64(0.0013543998793466017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025673647614894435) - rollout_corr/log_ppl_diff_max:np.float64(0.017022624611854553) - rollout_corr/log_ppl_diff_min:np.float64(-0.007628902792930603) - rollout_corr/ppl_ratio:np.float64(1.001360428519547) - rollout_corr/chi2_token:np.float64(0.0018329326994717121) - rollout_corr/chi2_seq:np.float64(1.2063880120404065) - actor/pg_loss:np.float64(0.006517422734759748) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015835570143281075) - actor/ppo_kl:np.float64(0.0004230728660630234) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(238186.0888677421) - self_distillation/token_reweight_w_std:np.float64(3422829.743828913) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022471062839031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1062716192973312) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.5796995311975479) - perf/mfu/actor:np.float64(0.035632098241036) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.68563079833984) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010413717478513718 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010413717478513718 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:289.72265625 - response_length/max:773.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.72265625 - response_length_non_aborted/max:773.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014895200729370117 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6900891289114952) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7452072985470295) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1825576704068226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7452072985470295) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:321 - timing_s/agent_loop/slowest/response_length:773 - timing_s/gen:11.57777032442391 - timing_s/reward:0.05630043335258961 - timing_s/old_log_prob:2.3727036882191896 - timing_s/adv:0.5643414072692394 - timing_s/update_actor:15.28370295651257 - timing_s/step:29.95019925571978 - timing_s/stop_profile:0.00013649649918079376 - timing_per_token_ms/adv:0.003660347569801199 - timing_per_token_ms/gen:0.15609985741244872 - timing_per_token_ms/update_actor:0.09913088824216693 - perf/total_num_tokens:154177 - perf/time_per_step:29.95019925571978 - perf/throughput:643.472346726357 (TaskRunner pid=2026458) Training Progress: 51%|█████ | 51/100 [37:29<37:36, 46.06s/it] (TaskRunner pid=2026458) step:52 - global_seqlen/min:14188 - global_seqlen/max:21038 - global_seqlen/minmax_diff:6850 - global_seqlen/balanced_min:18334 - global_seqlen/balanced_max:18335 - global_seqlen/mean:18334.5 - actor/entropy:0.2573181688785553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730220556259155 - training/rollout_probs_diff_mean:0.005135765764862299 - training/rollout_probs_diff_std:0.012896839529275894 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9999938011169434 - rollout_corr/rollout_is_ratio_fraction_high:2.519780355214607e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.519780355214607e-05 - rollout_corr/rollout_is_max:2.1215898990631104 - rollout_corr/rollout_is_min:0.3974515199661255 - rollout_corr/rollout_is_std:0.040721017867326736 - rollout_corr/rollout_is_eff_sample_size:0.9983446626766718 - rollout_corr/rollout_is_seq_mean:0.9999977946281433 - rollout_corr/rollout_is_seq_std:0.0026824194937944412 - rollout_corr/rollout_is_seq_max:1.0067678689956665 - rollout_corr/rollout_is_seq_min:0.9906572103500366 - rollout_corr/rollout_is_seq_max_deviation:0.009342789649963379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3137918766587973) - rollout_corr/training_log_ppl:np.float64(0.2694710589275928) - rollout_corr/kl:np.float64(0.0010667287329408914) - rollout_corr/k3_kl:np.float64(0.0010078331736167456) - rollout_corr/rollout_ppl:np.float64(1.3123429850675166) - rollout_corr/rollout_log_ppl:np.float64(0.26840433149482124) - rollout_corr/log_ppl_diff:np.float64(0.0010667274327715859) - rollout_corr/log_ppl_abs_diff:np.float64(0.002380369245656766) - rollout_corr/log_ppl_diff_max:np.float64(0.010203629732131958) - rollout_corr/log_ppl_diff_min:np.float64(-0.006238669157028198) - rollout_corr/ppl_ratio:np.float64(1.0010714137461036) - rollout_corr/chi2_token:np.float64(0.001948290504515171) - rollout_corr/chi2_seq:np.float64(0.5412429641000926) - actor/pg_loss:np.float64(0.003885703510604799) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005619293583549734) - actor/ppo_kl:np.float64(0.00019392038652732424) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(186537.93477050052) - self_distillation/token_reweight_w_std:np.float64(2011255.6988170468) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005899087991565) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06515720163588412) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4466311186552048) - perf/mfu/actor:np.float64(0.03402000847749656) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.35942840576172) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01099569071084261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01099569071084261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:310.046875 - response_length/max:700.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:310.046875 - response_length_non_aborted/max:700.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014230050146579742 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4891746453940868) - timing_s/agent_loop/generate_sequences/max:np.float64(5.560187539085746) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.207101171356044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.560187539085746) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:700 - timing_s/gen:11.716578045859933 - timing_s/reward:0.05753312073647976 - timing_s/old_log_prob:2.249606389552355 - timing_s/adv:0.5425826162099838 - timing_s/update_actor:15.384762125089765 - timing_s/step:30.04255287349224 - timing_s/stop_profile:0.00013405084609985352 - timing_per_token_ms/adv:0.003699191525607351 - timing_per_token_ms/gen:0.14761601126165314 - timing_per_token_ms/update_actor:0.10488943061639099 - perf/total_num_tokens:146676 - perf/time_per_step:30.04255287349224 - perf/throughput:610.2843549017191 (TaskRunner pid=2026458) Training Progress: 52%|█████▏ | 52/100 [37:59<33:00, 41.27s/it] (TaskRunner pid=2026458) step:53 - global_seqlen/min:17379 - global_seqlen/max:20313 - global_seqlen/minmax_diff:2934 - global_seqlen/balanced_min:18481 - global_seqlen/balanced_max:18482 - global_seqlen/mean:18481.75 - actor/entropy:0.278646856546402 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462908267974854 - training/rollout_probs_diff_mean:0.005563129670917988 - training/rollout_probs_diff_std:0.013384878635406494 - training/rollout_actor_probs_pearson_corr:0.9984221458435059 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:0.9999546408653259 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.5409334891010076e-05 - rollout_corr/rollout_is_max:1.8313579559326172 - rollout_corr/rollout_is_min:0.0737028419971466 - rollout_corr/rollout_is_std:0.04174019396305084 - rollout_corr/rollout_is_eff_sample_size:0.9982605487774039 - rollout_corr/rollout_is_seq_mean:0.9999428391456604 - rollout_corr/rollout_is_seq_std:0.0028358809649944305 - rollout_corr/rollout_is_seq_max:1.0075714588165283 - rollout_corr/rollout_is_seq_min:0.9910898804664612 - rollout_corr/rollout_is_seq_max_deviation:0.008910119533538818 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338077249005437) - rollout_corr/training_log_ppl:np.float64(0.2873227519157808) - rollout_corr/kl:np.float64(0.0011784572657376202) - rollout_corr/k3_kl:np.float64(0.0013956419594478575) - rollout_corr/rollout_ppl:np.float64(1.3364087981171906) - rollout_corr/rollout_log_ppl:np.float64(0.2861442923604045) - rollout_corr/log_ppl_diff:np.float64(0.001178459555376321) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027842955314554274) - rollout_corr/log_ppl_diff_max:np.float64(0.0132656991481781) - rollout_corr/log_ppl_diff_min:np.float64(-0.010441526770591736) - rollout_corr/ppl_ratio:np.float64(1.0011851615272462) - rollout_corr/chi2_token:np.float64(0.003180387197062373) - rollout_corr/chi2_seq:np.float64(1.2029992926400155) - actor/pg_loss:np.float64(0.003771678893826902) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009931258632605022) - actor/ppo_kl:np.float64(0.00019185108615538127) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(52868.786943790736) - self_distillation/token_reweight_w_std:np.float64(637934.6799204362) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990235811565071) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07278861297527328) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.44415684044361115) - perf/mfu/actor:np.float64(0.0346355208538403) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.01051712036133) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006060396321117878 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006060396321117878 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:281.9921875 - response_length/max:616.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.9921875 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001376233994960785 - timing_s/agent_loop/generate_sequences/min:np.float64(1.366797188296914) - timing_s/agent_loop/generate_sequences/max:np.float64(5.209049364551902) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1154251298939926) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.209049364551902) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:227 - timing_s/agent_loop/slowest/response_length:616 - timing_s/gen:11.11843347735703 - timing_s/reward:0.05367938429117203 - timing_s/old_log_prob:2.252691153436899 - timing_s/adv:0.4951089806854725 - timing_s/update_actor:15.301375834271312 - timing_s/step:29.318861059844494 - timing_s/stop_profile:0.00011683069169521332 - timing_per_token_ms/adv:0.0033486343330953 - timing_per_token_ms/gen:0.15401625540042985 - timing_per_token_ms/update_actor:0.10348976581134979 - perf/total_num_tokens:147854 - perf/time_per_step:29.318861059844494 - perf/throughput:630.3706669326542 (TaskRunner pid=2026458) Training Progress: 53%|█████▎ | 53/100 [38:29<29:31, 37.70s/it] (TaskRunner pid=2026458) step:54 - global_seqlen/min:15111 - global_seqlen/max:18863 - global_seqlen/minmax_diff:3752 - global_seqlen/balanced_min:17356 - global_seqlen/balanced_max:17358 - global_seqlen/mean:17357.625 - actor/entropy:0.2270389199256897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413564920425415 - training/rollout_probs_diff_mean:0.005126654170453548 - training/rollout_probs_diff_std:0.01343945786356926 - training/rollout_actor_probs_pearson_corr:0.9981903433799744 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001782178878784 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.876001235563308e-05 - rollout_corr/rollout_is_max:1.8594493865966797 - rollout_corr/rollout_is_min:0.35453474521636963 - rollout_corr/rollout_is_std:0.04036819189786911 - rollout_corr/rollout_is_eff_sample_size:0.998373654367098 - rollout_corr/rollout_is_seq_mean:1.000128984451294 - rollout_corr/rollout_is_seq_std:0.002654197160154581 - rollout_corr/rollout_is_seq_max:1.0092252492904663 - rollout_corr/rollout_is_seq_min:0.9911713600158691 - rollout_corr/rollout_is_seq_max_deviation:0.009225249290466309 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2745554209686816) - rollout_corr/training_log_ppl:np.float64(0.23950671087368391) - rollout_corr/kl:np.float64(0.0011859710301269644) - rollout_corr/k3_kl:np.float64(0.0012516252674004136) - rollout_corr/rollout_ppl:np.float64(1.2729601715691388) - rollout_corr/rollout_log_ppl:np.float64(0.23832074020174332) - rollout_corr/log_ppl_diff:np.float64(0.001185970671940595) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027243185904808342) - rollout_corr/log_ppl_diff_max:np.float64(0.01128050684928894) - rollout_corr/log_ppl_diff_min:np.float64(-0.011040359735488892) - rollout_corr/ppl_ratio:np.float64(1.0011925832368433) - rollout_corr/chi2_token:np.float64(0.0025079476181417704) - rollout_corr/chi2_seq:np.float64(1.487264727940783) - actor/pg_loss:np.float64(0.0043368348851799965) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017240041906916304) - actor/ppo_kl:np.float64(0.0004469881809043841) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(275413.10884484975) - self_distillation/token_reweight_w_std:np.float64(3763164.204352789) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020550324115902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08829297052579932) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5274116396903992) - perf/mfu/actor:np.float64(0.03254539451835227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.84357070922852) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0047957319766283035 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0047957319766283035 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.64453125 - response_length/max:650.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.64453125 - response_length_non_aborted/max:650.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.452683687210083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3436731528490782) - timing_s/agent_loop/generate_sequences/max:np.float64(5.206967985257506) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.978192554124689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.206967985257506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:650 - timing_s/gen:10.970427636057138 - timing_s/reward:0.05443461053073406 - timing_s/old_log_prob:2.2816072031855583 - timing_s/adv:0.47508152201771736 - timing_s/update_actor:15.217902828007936 - timing_s/step:29.08791142515838 - timing_s/stop_profile:0.0001310836523771286 - timing_per_token_ms/adv:0.0034212739503367926 - timing_per_token_ms/gen:0.15775481566352423 - timing_per_token_ms/update_actor:0.10959090621562524 - perf/total_num_tokens:138861 - perf/time_per_step:29.08791142515838 - perf/throughput:596.7298492592096 (TaskRunner pid=2026458) Training Progress: 54%|█████▍ | 54/100 [38:58<26:55, 35.13s/it] (TaskRunner pid=2026458) step:55 - global_seqlen/min:15554 - global_seqlen/max:21932 - global_seqlen/minmax_diff:6378 - global_seqlen/balanced_min:18814 - global_seqlen/balanced_max:18817 - global_seqlen/mean:18816.625 - actor/entropy:0.22956356406211853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9232618808746338 - training/rollout_probs_diff_mean:0.005393934901803732 - training/rollout_probs_diff_std:0.014924020506441593 - training/rollout_actor_probs_pearson_corr:0.9978342056274414 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000108242034912 - rollout_corr/rollout_is_ratio_fraction_high:4.133313268539496e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.133313268539496e-05 - rollout_corr/rollout_is_max:2.1670196056365967 - rollout_corr/rollout_is_min:0.04885798692703247 - rollout_corr/rollout_is_std:0.042956262826919556 - rollout_corr/rollout_is_eff_sample_size:0.9981586331310626 - rollout_corr/rollout_is_seq_mean:1.0001986026763916 - rollout_corr/rollout_is_seq_std:0.002816676627844572 - rollout_corr/rollout_is_seq_max:1.0112348794937134 - rollout_corr/rollout_is_seq_min:0.9917329549789429 - rollout_corr/rollout_is_seq_max_deviation:0.011234879493713379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2765053128823638) - rollout_corr/training_log_ppl:np.float64(0.24006979388650507) - rollout_corr/kl:np.float64(0.0010645344639215182) - rollout_corr/k3_kl:np.float64(0.0011318304883047858) - rollout_corr/rollout_ppl:np.float64(1.275088603142649) - rollout_corr/rollout_log_ppl:np.float64(0.23900525833596475) - rollout_corr/log_ppl_diff:np.float64(0.0010645355505403131) - rollout_corr/log_ppl_abs_diff:np.float64(0.002453350374707952) - rollout_corr/log_ppl_diff_max:np.float64(0.011057406663894653) - rollout_corr/log_ppl_diff_min:np.float64(-0.007540911436080933) - rollout_corr/ppl_ratio:np.float64(1.0010695066303015) - rollout_corr/chi2_token:np.float64(0.0023642820306122303) - rollout_corr/chi2_seq:np.float64(0.9924367545172572) - actor/pg_loss:np.float64(0.005413581035099924) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001406622541253455) - actor/ppo_kl:np.float64(0.0002698558086962066) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(295807.0609528662) - self_distillation/token_reweight_w_std:np.float64(3781227.560846627) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014416894409806) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09695331199327484) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5869030505418777) - perf/mfu/actor:np.float64(0.03432748799407685) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77687454223633) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014518261887133121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014518261887133121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:283.51953125 - response_length/max:687.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:283.51953125 - response_length_non_aborted/max:687.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010005757212638855 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9696572478860617) - timing_s/agent_loop/generate_sequences/max:np.float64(5.428103374317288) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.78926639315614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.428103374317288) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:329 - timing_s/agent_loop/slowest/response_length:687 - timing_s/gen:11.274114403873682 - timing_s/reward:0.0568949319422245 - timing_s/old_log_prob:2.307572791352868 - timing_s/adv:0.5059547368437052 - timing_s/update_actor:15.750575561076403 - timing_s/step:29.98292907886207 - timing_s/stop_profile:0.00012236833572387695 - timing_per_token_ms/adv:0.003361088511115205 - timing_per_token_ms/gen:0.1553314834994514 - timing_per_token_ms/update_actor:0.10463204454223594 - perf/total_num_tokens:150533 - perf/time_per_step:29.98292907886207 - perf/throughput:627.5779444532556 (TaskRunner pid=2026458) Training Progress: 55%|█████▌ | 55/100 [39:28<25:11, 33.60s/it] (TaskRunner pid=2026458) step:56 - global_seqlen/min:15183 - global_seqlen/max:19968 - global_seqlen/minmax_diff:4785 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16856 - global_seqlen/mean:16853.75 - actor/entropy:0.23840856552124023 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032567977905273 - training/rollout_probs_diff_mean:0.0051898774690926075 - training/rollout_probs_diff_std:0.013416245579719543 - training/rollout_actor_probs_pearson_corr:0.9982264637947083 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000948905944824 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9873039238736965e-05 - rollout_corr/rollout_is_max:1.868481159210205 - rollout_corr/rollout_is_min:0.42863455414772034 - rollout_corr/rollout_is_std:0.03991232439875603 - rollout_corr/rollout_is_eff_sample_size:0.9984097775176272 - rollout_corr/rollout_is_seq_mean:1.0001156330108643 - rollout_corr/rollout_is_seq_std:0.002804977586492896 - rollout_corr/rollout_is_seq_max:1.008453607559204 - rollout_corr/rollout_is_seq_min:0.9917566776275635 - rollout_corr/rollout_is_seq_max_deviation:0.008453607559204102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.288645532913506) - rollout_corr/training_log_ppl:np.float64(0.25009057341958396) - rollout_corr/kl:np.float64(0.001021610375019577) - rollout_corr/k3_kl:np.float64(0.0012247983307815957) - rollout_corr/rollout_ppl:np.float64(1.2872521225363016) - rollout_corr/rollout_log_ppl:np.float64(0.2490689625556115) - rollout_corr/log_ppl_diff:np.float64(0.0010216108639724553) - rollout_corr/log_ppl_abs_diff:np.float64(0.002677871671039611) - rollout_corr/log_ppl_diff_max:np.float64(0.011799484491348267) - rollout_corr/log_ppl_diff_min:np.float64(-0.013348877429962158) - rollout_corr/ppl_ratio:np.float64(1.001028069993481) - rollout_corr/chi2_token:np.float64(0.0027951118536293507) - rollout_corr/chi2_seq:np.float64(0.911775927990675) - actor/pg_loss:np.float64(0.0038659967249259353) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001200501193125092) - actor/ppo_kl:np.float64(0.0002928259359418206) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(173579.36585247447) - self_distillation/token_reweight_w_std:np.float64(2190372.348648581) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999903631862253) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08299089543288574) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.5205514430999756) - perf/mfu/actor:np.float64(0.03136809590233023) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65859985351562) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015309933223761618 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015309933223761618 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.5234375 - response_length/max:725.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.5234375 - response_length_non_aborted/max:725.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001024492084980011 - timing_s/agent_loop/generate_sequences/min:np.float64(1.564612640067935) - timing_s/agent_loop/generate_sequences/max:np.float64(5.375423099845648) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8240807638067054) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.375423099845648) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:298 - timing_s/agent_loop/slowest/response_length:725 - timing_s/gen:11.348324349150062 - timing_s/reward:0.054198144003748894 - timing_s/old_log_prob:2.395893217995763 - timing_s/adv:0.4817402958869934 - timing_s/update_actor:15.269863871857524 - timing_s/step:29.636497838422656 - timing_s/stop_profile:0.00012066960334777832 - timing_per_token_ms/adv:0.003572945901409133 - timing_per_token_ms/gen:0.1695044712345043 - timing_per_token_ms/update_actor:0.11325271728738058 - perf/total_num_tokens:134830 - perf/time_per_step:29.636497838422656 - perf/throughput:568.6822407926255 (TaskRunner pid=2026458) Training Progress: 56%|█████▌ | 56/100 [39:58<23:46, 32.42s/it] (TaskRunner pid=2026458) step:57 - global_seqlen/min:15145 - global_seqlen/max:21004 - global_seqlen/minmax_diff:5859 - global_seqlen/balanced_min:17327 - global_seqlen/balanced_max:17330 - global_seqlen/mean:17329.125 - actor/entropy:0.2190825492143631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744122624397278 - training/rollout_probs_diff_mean:0.005394987761974335 - training/rollout_probs_diff_std:0.014672977849841118 - training/rollout_actor_probs_pearson_corr:0.9977869987487793 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0002177953720093 - rollout_corr/rollout_is_ratio_fraction_high:9.200055501423776e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013800083252135664 - rollout_corr/rollout_is_max:3.8171095848083496 - rollout_corr/rollout_is_min:0.15978100895881653 - rollout_corr/rollout_is_std:0.04392360523343086 - rollout_corr/rollout_is_eff_sample_size:0.9980752630793054 - rollout_corr/rollout_is_seq_mean:1.0002235174179077 - rollout_corr/rollout_is_seq_std:0.0027234358713030815 - rollout_corr/rollout_is_seq_max:1.0101075172424316 - rollout_corr/rollout_is_seq_min:0.9909296035766602 - rollout_corr/rollout_is_seq_max_deviation:0.01010751724243164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2523039323277771) - rollout_corr/training_log_ppl:np.float64(0.22247063313261606) - rollout_corr/kl:np.float64(0.0011533432969415003) - rollout_corr/k3_kl:np.float64(0.0011055539546305226) - rollout_corr/rollout_ppl:np.float64(1.2508012815378606) - rollout_corr/rollout_log_ppl:np.float64(0.22131728671956807) - rollout_corr/log_ppl_diff:np.float64(0.0011533464130479842) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025634898629505187) - rollout_corr/log_ppl_diff_max:np.float64(0.016222700476646423) - rollout_corr/log_ppl_diff_min:np.float64(-0.007622018456459045) - rollout_corr/ppl_ratio:np.float64(1.0011591671500355) - rollout_corr/chi2_token:np.float64(0.0020452477037906647) - rollout_corr/chi2_seq:np.float64(0.5157537555787712) - actor/pg_loss:np.float64(0.003759522340260446) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008975071282293356) - actor/ppo_kl:np.float64(0.00035579913428307464) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(236077.81382388272) - self_distillation/token_reweight_w_std:np.float64(2308889.815033672) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994992383290082) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0630278438620735) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.43273352086544037) - perf/mfu/actor:np.float64(0.031510354454649704) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.82762956619263) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027676832396537066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027676832396537066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:254.75390625 - response_length/max:747.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:254.75390625 - response_length_non_aborted/max:747.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014020688831806183 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4702318627387285) - timing_s/agent_loop/generate_sequences/max:np.float64(5.407115560024977) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.896707163978135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.407115560024977) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:747 - timing_s/gen:11.385951856151223 - timing_s/reward:0.05382969416677952 - timing_s/old_log_prob:2.539514096453786 - timing_s/adv:0.49977708980441093 - timing_s/update_actor:15.644645780324936 - timing_s/step:30.209736336022615 - timing_s/stop_profile:0.00012044794857501984 - timing_per_token_ms/adv:0.0036050369666992053 - timing_per_token_ms/gen:0.17458564264150794 - timing_per_token_ms/update_actor:0.1128493632852563 - perf/total_num_tokens:138633 - perf/time_per_step:30.209736336022615 - perf/throughput:573.6271514338392 (TaskRunner pid=2026458) Training Progress: 57%|█████▋ | 57/100 [40:28<22:46, 31.77s/it] (TaskRunner pid=2026458) step:58 - global_seqlen/min:14734 - global_seqlen/max:19599 - global_seqlen/minmax_diff:4865 - global_seqlen/balanced_min:17102 - global_seqlen/balanced_max:17103 - global_seqlen/mean:17102.75 - actor/entropy:0.23608146607875824 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5141049027442932 - training/rollout_probs_diff_mean:0.005427653901278973 - training/rollout_probs_diff_std:0.013945525512099266 - training/rollout_actor_probs_pearson_corr:0.9981228709220886 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999080896377563 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5163001990003977e-05 - rollout_corr/rollout_is_max:1.724382996559143 - rollout_corr/rollout_is_min:0.16556662321090698 - rollout_corr/rollout_is_std:0.04180582985281944 - rollout_corr/rollout_is_eff_sample_size:0.9982550842404107 - rollout_corr/rollout_is_seq_mean:0.9999476671218872 - rollout_corr/rollout_is_seq_std:0.002901083789765835 - rollout_corr/rollout_is_seq_max:1.00826895236969 - rollout_corr/rollout_is_seq_min:0.9917892217636108 - rollout_corr/rollout_is_seq_max_deviation:0.008268952369689941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2862667385488749) - rollout_corr/training_log_ppl:np.float64(0.24756408171379007) - rollout_corr/kl:np.float64(0.0008958796237372724) - rollout_corr/k3_kl:np.float64(0.0010313347952433105) - rollout_corr/rollout_ppl:np.float64(1.2851230534724891) - rollout_corr/rollout_log_ppl:np.float64(0.24666820294805802) - rollout_corr/log_ppl_diff:np.float64(0.0008958787657320499) - rollout_corr/log_ppl_abs_diff:np.float64(0.002455878769978881) - rollout_corr/log_ppl_diff_max:np.float64(0.014661163091659546) - rollout_corr/log_ppl_diff_min:np.float64(-0.006324887275695801) - rollout_corr/ppl_ratio:np.float64(1.00090107973665) - rollout_corr/chi2_token:np.float64(0.0023643041495233774) - rollout_corr/chi2_seq:np.float64(0.6059110292699188) - actor/pg_loss:np.float64(0.003710375865921378) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011193532754987245) - actor/ppo_kl:np.float64(-8.923532409355062e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(286496.5454768897) - self_distillation/token_reweight_w_std:np.float64(3628499.957205932) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005135682877153) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07967197702964768) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5009380057454109) - perf/mfu/actor:np.float64(0.03152191694270064) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.63263320922852) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016660349210724235 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016660349210724235 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:257.6171875 - response_length/max:583.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:257.6171875 - response_length_non_aborted/max:583.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001420155167579651 - timing_s/agent_loop/generate_sequences/min:np.float64(1.489874416962266) - timing_s/agent_loop/generate_sequences/max:np.float64(4.795747706666589) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8081408347570687) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.795747706666589) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:583 - timing_s/gen:10.590774795040488 - timing_s/reward:0.05500875972211361 - timing_s/old_log_prob:2.199430391192436 - timing_s/adv:0.4823594633489847 - timing_s/update_actor:15.483584174886346 - timing_s/step:28.898630160838366 - timing_s/stop_profile:0.0001272875815629959 - timing_per_token_ms/adv:0.0035254525101883084 - timing_per_token_ms/gen:0.16058794230539028 - timing_per_token_ms/update_actor:0.11316589565191523 - perf/total_num_tokens:136822 - perf/time_per_step:28.898630160838366 - perf/throughput:591.818709219532 (TaskRunner pid=2026458) Training Progress: 58%|█████▊ | 58/100 [40:57<21:38, 30.92s/it] (TaskRunner pid=2026458) step:59 - global_seqlen/min:15244 - global_seqlen/max:19485 - global_seqlen/minmax_diff:4241 - global_seqlen/balanced_min:16769 - global_seqlen/balanced_max:16771 - global_seqlen/mean:16770.625 - actor/entropy:0.2315026819705963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5046374797821045 - training/rollout_probs_diff_mean:0.0052477153949439526 - training/rollout_probs_diff_std:0.013823322020471096 - training/rollout_actor_probs_pearson_corr:0.9981176853179932 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:1.0000110864639282 - rollout_corr/rollout_is_ratio_fraction_high:1.5287481801351532e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.114992720540613e-05 - rollout_corr/rollout_is_max:2.5501413345336914 - rollout_corr/rollout_is_min:0.3053041398525238 - rollout_corr/rollout_is_std:0.04093707725405693 - rollout_corr/rollout_is_eff_sample_size:0.998326959562596 - rollout_corr/rollout_is_seq_mean:1.000074863433838 - rollout_corr/rollout_is_seq_std:0.0029152550268918276 - rollout_corr/rollout_is_seq_max:1.0087932348251343 - rollout_corr/rollout_is_seq_min:0.9920890927314758 - rollout_corr/rollout_is_seq_max_deviation:0.008793234825134277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2814294979907572) - rollout_corr/training_log_ppl:np.float64(0.24554033184540458) - rollout_corr/kl:np.float64(0.0010128226648049576) - rollout_corr/k3_kl:np.float64(0.0010760613429283694) - rollout_corr/rollout_ppl:np.float64(1.280145457945764) - rollout_corr/rollout_log_ppl:np.float64(0.2445275079808198) - rollout_corr/log_ppl_diff:np.float64(0.0010128238645847887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026435373874846846) - rollout_corr/log_ppl_diff_max:np.float64(0.01640312373638153) - rollout_corr/log_ppl_diff_min:np.float64(-0.01078614592552185) - rollout_corr/ppl_ratio:np.float64(1.001019265735522) - rollout_corr/chi2_token:np.float64(0.0022960123606026173) - rollout_corr/chi2_seq:np.float64(0.9392846249975264) - actor/pg_loss:np.float64(0.002857705345377326) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044364196128299227) - actor/ppo_kl:np.float64(0.0001824798405234418) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(71359.55967221712) - self_distillation/token_reweight_w_std:np.float64(989988.4983619121) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995234010275453) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04141386490664445) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3792598769068718) - perf/mfu/actor:np.float64(0.03147454843432276) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65855026245117) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003848623251542449 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:0.003848623251542449 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:255.51953125 - response_length/max:621.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:255.51953125 - response_length_non_aborted/max:621.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.882924258708954e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.939049456268549) - timing_s/agent_loop/generate_sequences/max:np.float64(5.183893235400319) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.658139636150736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.183893235400319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:217 - timing_s/agent_loop/slowest/response_length:621 - timing_s/gen:10.904466819018126 - timing_s/reward:0.052795736119151115 - timing_s/old_log_prob:2.23296419903636 - timing_s/adv:0.47304221242666245 - timing_s/update_actor:15.1854842081666 - timing_s/step:28.936049316078424 - timing_s/stop_profile:5.451589822769165e-05 - timing_per_token_ms/adv:0.0035258242643510787 - timing_per_token_ms/gen:0.16670183020222473 - timing_per_token_ms/update_actor:0.11318513925514552 - perf/total_num_tokens:134165 - perf/time_per_step:28.936049316078424 - perf/throughput:579.575491346752 (TaskRunner pid=2026458) Training Progress: 59%|█████▉ | 59/100 [41:26<20:43, 30.33s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options contains the correct functional groups and structure that match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a nitrogen-containing group (n1) connected to a cyclohexane ring (Cc2ccc(Cl)cc2), with a Cl group and a trifluoromethyl group (C(F)(F)F). The correct reactant must include these key features. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1) connected to a cyclohexane ring with a Cl group and a trifluoromethyl group. This matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it does not include the Cl group or the trifluoromethyl group in the correct position. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (Oc2ccc(Cl)c(C(F)(F)F)c2) that does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (CC=C(Cc1ccccc1)C(F)(F)F) that does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2026458) step:60 - global_seqlen/min:14552 - global_seqlen/max:17730 - global_seqlen/minmax_diff:3178 - global_seqlen/balanced_min:16889 - global_seqlen/balanced_max:16891 - global_seqlen/mean:16890.25 - actor/entropy:0.258317232131958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5851317644119263 - training/rollout_probs_diff_mean:0.005422248970717192 - training/rollout_probs_diff_std:0.013785075396299362 - training/rollout_actor_probs_pearson_corr:0.9982817769050598 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0001024007797241 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4916022337274626e-05 - rollout_corr/rollout_is_max:1.976409912109375 - rollout_corr/rollout_is_min:0.3921801745891571 - rollout_corr/rollout_is_std:0.041325442492961884 - rollout_corr/rollout_is_eff_sample_size:0.9982955945128663 - rollout_corr/rollout_is_seq_mean:1.000079870223999 - rollout_corr/rollout_is_seq_std:0.002677295822650194 - rollout_corr/rollout_is_seq_max:1.0086641311645508 - rollout_corr/rollout_is_seq_min:0.9929971694946289 - rollout_corr/rollout_is_seq_max_deviation:0.008664131164550781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3128179563209414) - rollout_corr/training_log_ppl:np.float64(0.2678291550837457) - rollout_corr/kl:np.float64(0.001152293861238718) - rollout_corr/k3_kl:np.float64(0.0010784027733734547) - rollout_corr/rollout_ppl:np.float64(1.3112971791997552) - rollout_corr/rollout_log_ppl:np.float64(0.2666768589988351) - rollout_corr/log_ppl_diff:np.float64(0.0011522960849106312) - rollout_corr/log_ppl_abs_diff:np.float64(0.002510662190616131) - rollout_corr/log_ppl_diff_max:np.float64(0.009669288992881775) - rollout_corr/log_ppl_diff_min:np.float64(-0.007052525877952576) - rollout_corr/ppl_ratio:np.float64(1.0011574435047805) - rollout_corr/chi2_token:np.float64(0.002051107119768858) - rollout_corr/chi2_seq:np.float64(0.5519429405685514) - actor/pg_loss:np.float64(0.003786680055782199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008521524032403249) - actor/ppo_kl:np.float64(0.0003417033792416646) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(148988.88128135214) - self_distillation/token_reweight_w_std:np.float64(1866554.9818183868) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014604937750846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08368598294327967) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4603109806776047) - perf/mfu/actor:np.float64(0.03142972200463806) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.4005994796753) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/reward/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07287959815596419) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8152857142857142) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0635348044707675) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6967476190476191) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04617062732616158) - val-aux/sciknoweval/score/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/score/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07287959815596419) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8152857142857142) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0635348044707675) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6967476190476191) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04617062732616158) - val-core/sciknoweval/acc/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/acc/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07287959815596419) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8152857142857142) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0635348044707675) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6967476190476191) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04617062732616158) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000639524485450238 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000639524485450238 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.8828125 - response_length/max:611.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.8828125 - response_length_non_aborted/max:611.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.9375 - prompt_length/max:451.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00021154806017875671 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3727056793868542) - timing_s/agent_loop/generate_sequences/max:np.float64(4.906513184309006) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9096029985012137) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.906513184309006) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:563 - timing_s/gen:11.531464690342546 - timing_s/reward:0.05282927677035332 - timing_s/old_log_prob:2.459550989791751 - timing_s/adv:0.5421040039509535 - timing_s/update_actor:15.288631148636341 - timing_s/step:29.9776940792799 - timing_s/testing:64.2218036558479 - timing_s/save_checkpoint:6.659373983740807 - timing_s/stop_profile:5.859695374965668e-05 - timing_per_token_ms/adv:0.00401195959170937 - timing_per_token_ms/gen:0.17200359014263514 - timing_per_token_ms/update_actor:0.11314686837551502 - perf/total_num_tokens:135122 - perf/time_per_step:29.9776940792799 - perf/throughput:563.4272587922054 (TaskRunner pid=2026458) Training Progress: 60%|██████ | 60/100 [43:07<34:24, 51.61s/it] (TaskRunner pid=2026458) step:61 - global_seqlen/min:14736 - global_seqlen/max:19497 - global_seqlen/minmax_diff:4761 - global_seqlen/balanced_min:17532 - global_seqlen/balanced_max:17536 - global_seqlen/mean:17534.5 - actor/entropy:0.2423284649848938 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49669769406318665 - training/rollout_probs_diff_mean:0.005022233817726374 - training/rollout_probs_diff_std:0.013324090279638767 - training/rollout_actor_probs_pearson_corr:0.9983177781105042 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0002062320709229 - rollout_corr/rollout_is_ratio_fraction_high:5.464779314934276e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0985847590491176e-05 - rollout_corr/rollout_is_max:7.0302934646606445 - rollout_corr/rollout_is_min:0.35777419805526733 - rollout_corr/rollout_is_std:0.03978368639945984 - rollout_corr/rollout_is_eff_sample_size:0.998420353539922 - rollout_corr/rollout_is_seq_mean:1.0001554489135742 - rollout_corr/rollout_is_seq_std:0.002474896376952529 - rollout_corr/rollout_is_seq_max:1.0129971504211426 - rollout_corr/rollout_is_seq_min:0.9919136762619019 - rollout_corr/rollout_is_seq_max_deviation:0.012997150421142578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2963423300534487) - rollout_corr/training_log_ppl:np.float64(0.25556481117382646) - rollout_corr/kl:np.float64(0.0009956922927543133) - rollout_corr/k3_kl:np.float64(0.0010460353189500893) - rollout_corr/rollout_ppl:np.float64(1.2950277719646692) - rollout_corr/rollout_log_ppl:np.float64(0.25456911721266806) - rollout_corr/log_ppl_diff:np.float64(0.0009956939611583948) - rollout_corr/log_ppl_abs_diff:np.float64(0.002202652278356254) - rollout_corr/log_ppl_diff_max:np.float64(0.010044902563095093) - rollout_corr/log_ppl_diff_min:np.float64(-0.009525060653686523) - rollout_corr/ppl_ratio:np.float64(1.0009999251924455) - rollout_corr/chi2_token:np.float64(0.0022548860870301723) - rollout_corr/chi2_seq:np.float64(0.8442793434951454) - actor/pg_loss:np.float64(0.0042004447896033525) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007894714767644473) - actor/ppo_kl:np.float64(0.0002806229531095994) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(132276.08868701942) - self_distillation/token_reweight_w_std:np.float64(1859101.6540320972) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012411118950695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08409801000379957) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4940867945551872) - perf/mfu/actor:np.float64(0.03230972888329671) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0987777709961) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0024950134102255106 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0024950134102255106 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:285.921875 - response_length/max:685.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:285.921875 - response_length_non_aborted/max:685.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.605738937854767e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.467216296121478) - timing_s/agent_loop/generate_sequences/max:np.float64(5.52247816324234) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0488981239686836) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.52247816324234) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:652 - timing_s/gen:11.610483139753342 - timing_s/reward:0.05246041156351566 - timing_s/old_log_prob:2.416056863963604 - timing_s/adv:0.5235678162425756 - timing_s/update_actor:15.423095140606165 - timing_s/step:30.12737883068621 - timing_s/stop_profile:0.00011511333286762238 - timing_per_token_ms/adv:0.003732411932494337 - timing_per_token_ms/gen:0.15862182550622086 - timing_per_token_ms/update_actor:0.10994821024698569 - perf/total_num_tokens:140276 - perf/time_per_step:30.12737883068621 - perf/throughput:582.0121325038822 (TaskRunner pid=2026458) Training Progress: 61%|██████ | 61/100 [43:37<29:22, 45.18s/it] (TaskRunner pid=2026458) step:62 - global_seqlen/min:15907 - global_seqlen/max:20488 - global_seqlen/minmax_diff:4581 - global_seqlen/balanced_min:18196 - global_seqlen/balanced_max:18198 - global_seqlen/mean:18197.0 - actor/entropy:0.26120486855506897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35323476791381836 - training/rollout_probs_diff_mean:0.005538288038223982 - training/rollout_probs_diff_std:0.014037296175956726 - training/rollout_actor_probs_pearson_corr:0.9981968998908997 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.9998036623001099 - rollout_corr/rollout_is_ratio_fraction_high:2.8998956622672267e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014499477401841432 - rollout_corr/rollout_is_max:2.314438819885254 - rollout_corr/rollout_is_min:0.32504311203956604 - rollout_corr/rollout_is_std:0.04294585436582565 - rollout_corr/rollout_is_eff_sample_size:0.9981582768191568 - rollout_corr/rollout_is_seq_mean:0.9998564720153809 - rollout_corr/rollout_is_seq_std:0.0027946694754064083 - rollout_corr/rollout_is_seq_max:1.0084697008132935 - rollout_corr/rollout_is_seq_min:0.9910843372344971 - rollout_corr/rollout_is_seq_max_deviation:0.00891566276550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.316816312726587) - rollout_corr/training_log_ppl:np.float64(0.27135412095231004) - rollout_corr/kl:np.float64(0.0013785573038145316) - rollout_corr/k3_kl:np.float64(0.0011328787384172756) - rollout_corr/rollout_ppl:np.float64(1.3149180496111512) - rollout_corr/rollout_log_ppl:np.float64(0.2699755621724762) - rollout_corr/log_ppl_diff:np.float64(0.0013785587798338383) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027030363271478564) - rollout_corr/log_ppl_diff_max:np.float64(0.010782003402709961) - rollout_corr/log_ppl_diff_min:np.float64(-0.008555859327316284) - rollout_corr/ppl_ratio:np.float64(1.0013843621127307) - rollout_corr/chi2_token:np.float64(0.0016965712420642376) - rollout_corr/chi2_seq:np.float64(0.6767998114228249) - actor/pg_loss:np.float64(0.003519455436617136) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008139853748616588) - actor/ppo_kl:np.float64(0.0002313825499626887) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(140065.4251762007) - self_distillation/token_reweight_w_std:np.float64(1931525.736894751) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999061315320432) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07281633446109481) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.4214628040790558) - perf/mfu/actor:np.float64(0.033835202551722315) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0380401611328) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003894922323524952 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003894922323524952 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:269.40625 - response_length/max:578.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.40625 - response_length_non_aborted/max:578.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001499168574810028 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3069061245769262) - timing_s/agent_loop/generate_sequences/max:np.float64(4.653931180015206) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8548159639394726) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.653931180015206) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:578 - timing_s/gen:10.786067547276616 - timing_s/reward:0.05438114143908024 - timing_s/old_log_prob:2.2739491928368807 - timing_s/adv:0.471931716427207 - timing_s/update_actor:15.285212887451053 - timing_s/step:28.96465684659779 - timing_s/stop_profile:0.00012806802988052368 - timing_per_token_ms/adv:0.003241823627707912 - timing_per_token_ms/gen:0.15639234931093574 - timing_per_token_ms/update_actor:0.10499816513333965 - perf/total_num_tokens:145576 - perf/time_per_step:28.96465684659779 - perf/throughput:628.2484234622456 (TaskRunner pid=2026458) Training Progress: 62%|██████▏ | 62/100 [44:06<25:32, 40.33s/it] (TaskRunner pid=2026458) step:63 - global_seqlen/min:15346 - global_seqlen/max:20074 - global_seqlen/minmax_diff:4728 - global_seqlen/balanced_min:17670 - global_seqlen/balanced_max:17672 - global_seqlen/mean:17671.0 - actor/entropy:0.2494240403175354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44295963644981384 - training/rollout_probs_diff_mean:0.005514265038073063 - training/rollout_probs_diff_std:0.013980090618133545 - training/rollout_actor_probs_pearson_corr:0.9981726408004761 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9999319911003113 - rollout_corr/rollout_is_ratio_fraction_high:1.4685150745208375e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.811090083327144e-05 - rollout_corr/rollout_is_max:2.1109066009521484 - rollout_corr/rollout_is_min:0.21203994750976562 - rollout_corr/rollout_is_std:0.04199644550681114 - rollout_corr/rollout_is_eff_sample_size:0.9982390473577023 - rollout_corr/rollout_is_seq_mean:0.9999494552612305 - rollout_corr/rollout_is_seq_std:0.003092808648943901 - rollout_corr/rollout_is_seq_max:1.0130763053894043 - rollout_corr/rollout_is_seq_min:0.9914883375167847 - rollout_corr/rollout_is_seq_max_deviation:0.013076305389404297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.302847915329039) - rollout_corr/training_log_ppl:np.float64(0.26041782830725424) - rollout_corr/kl:np.float64(0.001262190604840896) - rollout_corr/k3_kl:np.float64(0.0011403532549252304) - rollout_corr/rollout_ppl:np.float64(1.3011768972501159) - rollout_corr/rollout_log_ppl:np.float64(0.2591556366533041) - rollout_corr/log_ppl_diff:np.float64(0.0012621916539501399) - rollout_corr/log_ppl_abs_diff:np.float64(0.002628726389957592) - rollout_corr/log_ppl_diff_max:np.float64(0.010536238551139832) - rollout_corr/log_ppl_diff_min:np.float64(-0.009842857718467712) - rollout_corr/ppl_ratio:np.float64(1.0012680946383625) - rollout_corr/chi2_token:np.float64(0.0019653229974210262) - rollout_corr/chi2_seq:np.float64(0.7706008821260184) - actor/pg_loss:np.float64(0.0034944345243275166) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000709521887529263) - actor/ppo_kl:np.float64(0.00027179806370725146) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(168796.3436012969) - self_distillation/token_reweight_w_std:np.float64(2124951.2078802614) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015958338044584) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0751116150058806) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4171854108572006) - perf/mfu/actor:np.float64(0.03263190662040404) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89733123779297) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005943814758211374 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005943814758211374 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:266.0 - response_length/max:610.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.0 - response_length_non_aborted/max:610.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011231563985347748 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2275551408529282) - timing_s/agent_loop/generate_sequences/max:np.float64(4.99046435020864) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8069054595180205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.99046435020864) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:231 - timing_s/agent_loop/slowest/response_length:607 - timing_s/gen:11.166345795616508 - timing_s/reward:0.05529453977942467 - timing_s/old_log_prob:2.2847698368132114 - timing_s/adv:0.49184093065559864 - timing_s/update_actor:15.462059672921896 - timing_s/step:29.548018461093307 - timing_s/stop_profile:0.00011456944048404694 - timing_per_token_ms/adv:0.0034791532076254785 - timing_per_token_ms/gen:0.1639794671583721 - timing_per_token_ms/update_actor:0.10937453789345464 - perf/total_num_tokens:141368 - perf/time_per_step:29.548018461093307 - perf/throughput:598.0434871890951 (TaskRunner pid=2026458) Training Progress: 63%|██████▎ | 63/100 [44:36<22:53, 37.11s/it] (TaskRunner pid=2026458) step:64 - global_seqlen/min:14067 - global_seqlen/max:24197 - global_seqlen/minmax_diff:10130 - global_seqlen/balanced_min:18752 - global_seqlen/balanced_max:25893 - global_seqlen/mean:19646.75 - actor/entropy:0.21532860398292542 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8697976469993591 - training/rollout_probs_diff_mean:0.004864232614636421 - training/rollout_probs_diff_std:0.013850908726453781 - training/rollout_actor_probs_pearson_corr:0.9980413317680359 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0002894401550293 - rollout_corr/rollout_is_ratio_fraction_high:6.0805057728430256e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.4322023818967864e-05 - rollout_corr/rollout_is_max:8.297144889831543 - rollout_corr/rollout_is_min:0.23644955456256866 - rollout_corr/rollout_is_std:0.04067561775445938 - rollout_corr/rollout_is_eff_sample_size:0.9983489400333069 - rollout_corr/rollout_is_seq_mean:1.000409722328186 - rollout_corr/rollout_is_seq_std:0.0031241707038134336 - rollout_corr/rollout_is_seq_max:1.0219839811325073 - rollout_corr/rollout_is_seq_min:0.9937897324562073 - rollout_corr/rollout_is_seq_max_deviation:0.021983981132507324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2946171956136823) - rollout_corr/training_log_ppl:np.float64(0.25466993783356884) - rollout_corr/kl:np.float64(0.0008532866019912433) - rollout_corr/k3_kl:np.float64(0.0010788948769118178) - rollout_corr/rollout_ppl:np.float64(1.2935234676115215) - rollout_corr/rollout_log_ppl:np.float64(0.25381665135137155) - rollout_corr/log_ppl_diff:np.float64(0.0008532864821972908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022936501272852183) - rollout_corr/log_ppl_diff_max:np.float64(0.00881144404411316) - rollout_corr/log_ppl_diff_min:np.float64(-0.008881300687789917) - rollout_corr/ppl_ratio:np.float64(1.0008577948901802) - rollout_corr/chi2_token:np.float64(0.003156282240524888) - rollout_corr/chi2_seq:np.float64(3.035388365620747) - actor/pg_loss:np.float64(0.0014677918516099453) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004414845861901995) - actor/ppo_kl:np.float64(0.00026685018261352056) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(81162.51946377661) - self_distillation/token_reweight_w_std:np.float64(1170164.887981981) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010531947482377) - self_distillation/token_reweight_w_clip_frac:np.float64(0.049339942750521004) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.263648372143507) - perf/mfu/actor:np.float64(0.03462126720848994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.74031829833984) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.08103185892105103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.08103185892105103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.2109375 - response_length/max:8192.0 - response_length/min:110.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:321.2109375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010729953646659851 - timing_s/agent_loop/generate_sequences/min:np.float64(1.288361655548215) - timing_s/agent_loop/generate_sequences/max:np.float64(50.12881679646671) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2828551806887845) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.12881679646671) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:172 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.03219731338322 - timing_s/reward:0.05905804969370365 - timing_s/old_log_prob:2.5028711035847664 - timing_s/adv:0.508072393015027 - timing_s/update_actor:16.261700792238116 - timing_s/step:75.45720112323761 - timing_s/stop_profile:4.878826439380646e-05 - timing_per_token_ms/adv:0.0032325473234442533 - timing_per_token_ms/gen:0.6814082124940195 - timing_per_token_ms/update_actor:0.10346304600148953 - perf/total_num_tokens:157174 - perf/time_per_step:75.45720112323761 - perf/throughput:260.3694505964075 (TaskRunner pid=2026458) Training Progress: 64%|██████▍ | 64/100 [45:51<29:10, 48.62s/it] (TaskRunner pid=2026458) step:65 - global_seqlen/min:15781 - global_seqlen/max:21398 - global_seqlen/minmax_diff:5617 - global_seqlen/balanced_min:18710 - global_seqlen/balanced_max:18715 - global_seqlen/mean:18712.625 - actor/entropy:0.2451825588941574 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43764662742614746 - training/rollout_probs_diff_mean:0.005050553474575281 - training/rollout_probs_diff_std:0.013090502470731735 - training/rollout_actor_probs_pearson_corr:0.9983762502670288 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999321103096008 - rollout_corr/rollout_is_ratio_fraction_high:1.2970000170753337e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.8910002331249416e-05 - rollout_corr/rollout_is_max:2.924093246459961 - rollout_corr/rollout_is_min:0.4824545979499817 - rollout_corr/rollout_is_std:0.040282461792230606 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:0.9999394416809082 - rollout_corr/rollout_is_seq_std:0.002558288164436817 - rollout_corr/rollout_is_seq_max:1.0071020126342773 - rollout_corr/rollout_is_seq_min:0.9918232560157776 - rollout_corr/rollout_is_seq_max_deviation:0.008176743984222412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2973304083570838) - rollout_corr/training_log_ppl:np.float64(0.2564336106006522) - rollout_corr/kl:np.float64(0.0012040652124341022) - rollout_corr/k3_kl:np.float64(0.000990912636439134) - rollout_corr/rollout_ppl:np.float64(1.2957068742252886) - rollout_corr/rollout_log_ppl:np.float64(0.2552295460482128) - rollout_corr/log_ppl_diff:np.float64(0.0012040645524393767) - rollout_corr/log_ppl_abs_diff:np.float64(0.002526016061892733) - rollout_corr/log_ppl_diff_max:np.float64(0.011661797761917114) - rollout_corr/log_ppl_diff_min:np.float64(-0.006277412176132202) - rollout_corr/ppl_ratio:np.float64(1.001209291163832) - rollout_corr/chi2_token:np.float64(0.0015423318836838007) - rollout_corr/chi2_seq:np.float64(0.9701050808653235) - actor/pg_loss:np.float64(0.0032792859710752964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007688776604481973) - actor/ppo_kl:np.float64(0.0002768948086071532) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(86791.39557419321) - self_distillation/token_reweight_w_std:np.float64(1084418.8798323106) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982076121959835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08052845095517114) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.3758349269628525) - perf/mfu/actor:np.float64(0.03501136292112507) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.70735931396484) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0052852751687169075 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0052852751687169075 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.17578125 - response_length/max:840.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.17578125 - response_length_non_aborted/max:840.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.040077328681946e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2666476387530565) - timing_s/agent_loop/generate_sequences/max:np.float64(6.11080476641655) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.148740480595734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.11080476641655) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:228 - timing_s/agent_loop/slowest/response_length:840 - timing_s/gen:12.000718062743545 - timing_s/reward:0.056409286335110664 - timing_s/old_log_prob:2.2715286798775196 - timing_s/adv:0.546351995319128 - timing_s/update_actor:15.278582841157913 - timing_s/step:30.242278227582574 - timing_s/stop_profile:0.0001621972769498825 - timing_per_token_ms/adv:0.0036496215477460274 - timing_per_token_ms/gen:0.1556493179432633 - timing_per_token_ms/update_actor:0.10206065985636645 - perf/total_num_tokens:149701 - perf/time_per_step:30.242278227582574 - perf/throughput:618.757120716292 (TaskRunner pid=2026458) Training Progress: 65%|██████▌ | 65/100 [46:22<25:09, 43.12s/it] (TaskRunner pid=2026458) step:66 - global_seqlen/min:16986 - global_seqlen/max:21356 - global_seqlen/minmax_diff:4370 - global_seqlen/balanced_min:19034 - global_seqlen/balanced_max:19038 - global_seqlen/mean:19037.375 - actor/entropy:0.2407594919204712 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26579251885414124 - training/rollout_probs_diff_mean:0.005038884002715349 - training/rollout_probs_diff_std:0.013142738491296768 - training/rollout_actor_probs_pearson_corr:0.9983550906181335 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:1.000045657157898 - rollout_corr/rollout_is_ratio_fraction_high:2.5642670152592473e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.974934462457895e-05 - rollout_corr/rollout_is_max:2.355177402496338 - rollout_corr/rollout_is_min:0.19801293313503265 - rollout_corr/rollout_is_std:0.04063016548752785 - rollout_corr/rollout_is_eff_sample_size:0.9983520292581156 - rollout_corr/rollout_is_seq_mean:1.0001461505889893 - rollout_corr/rollout_is_seq_std:0.0026480352971702814 - rollout_corr/rollout_is_seq_max:1.0091423988342285 - rollout_corr/rollout_is_seq_min:0.9913557171821594 - rollout_corr/rollout_is_seq_max_deviation:0.009142398834228516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3078779838979244) - rollout_corr/training_log_ppl:np.float64(0.26354357780655846) - rollout_corr/kl:np.float64(0.0010749662497779866) - rollout_corr/k3_kl:np.float64(0.0014432408553943787) - rollout_corr/rollout_ppl:np.float64(1.3064235104247928) - rollout_corr/rollout_log_ppl:np.float64(0.2624686111230403) - rollout_corr/log_ppl_diff:np.float64(0.0010749666835181415) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029108942835591733) - rollout_corr/log_ppl_diff_max:np.float64(0.013920098543167114) - rollout_corr/log_ppl_diff_min:np.float64(-0.009019121527671814) - rollout_corr/ppl_ratio:np.float64(1.0010822340846062) - rollout_corr/chi2_token:np.float64(0.003545147832483053) - rollout_corr/chi2_seq:np.float64(1.548627549316734) - actor/pg_loss:np.float64(0.003003176534548402) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009145650715254305) - actor/ppo_kl:np.float64(0.0003188287759954278) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(126959.81397781824) - self_distillation/token_reweight_w_std:np.float64(1629415.2238851967) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008254635613412) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06369021107093431) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.4391012266278267) - perf/mfu/actor:np.float64(0.03551272812953314) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.41056060791016) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004540354944765568 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004540354944765568 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:304.66796875 - response_length/max:841.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:304.66796875 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014583580195903778 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5362531188875437) - timing_s/agent_loop/generate_sequences/max:np.float64(6.417147822678089) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.267450157953135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.417147822678089) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:841 - timing_s/gen:12.298093724995852 - timing_s/reward:0.05658415146172047 - timing_s/old_log_prob:2.3751799557358027 - timing_s/adv:0.4913473054766655 - timing_s/update_actor:15.358580730855465 - timing_s/step:30.66549047641456 - timing_s/stop_profile:0.0001288391649723053 - timing_per_token_ms/adv:0.0032262017838374873 - timing_per_token_ms/gen:0.1576779758317309 - timing_per_token_ms/update_actor:0.10084492170569383 - perf/total_num_tokens:152299 - perf/time_per_step:30.66549047641456 - perf/throughput:620.8077778714162 (TaskRunner pid=2026458) Training Progress: 66%|██████▌ | 66/100 [46:52<22:19, 39.40s/it] (TaskRunner pid=2026458) step:67 - global_seqlen/min:15159 - global_seqlen/max:19915 - global_seqlen/minmax_diff:4756 - global_seqlen/balanced_min:18046 - global_seqlen/balanced_max:18050 - global_seqlen/mean:18048.5 - actor/entropy:0.22801370918750763 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9852267503738403 - training/rollout_probs_diff_mean:0.005018836818635464 - training/rollout_probs_diff_std:0.014277998358011246 - training/rollout_actor_probs_pearson_corr:0.9979830384254456 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0002268552780151 - rollout_corr/rollout_is_ratio_fraction_high:2.8704287615255453e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001004650112008676 - rollout_corr/rollout_is_max:2.3428878784179688 - rollout_corr/rollout_is_min:0.0007197936065495014 - rollout_corr/rollout_is_std:0.040458161383867264 - rollout_corr/rollout_is_eff_sample_size:0.9983666439269478 - rollout_corr/rollout_is_seq_mean:1.0003077983856201 - rollout_corr/rollout_is_seq_std:0.0026670496445149183 - rollout_corr/rollout_is_seq_max:1.0164320468902588 - rollout_corr/rollout_is_seq_min:0.9921717047691345 - rollout_corr/rollout_is_seq_max_deviation:0.01643204689025879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2767656035721302) - rollout_corr/training_log_ppl:np.float64(0.24004710461304057) - rollout_corr/kl:np.float64(0.0011609104617287258) - rollout_corr/k3_kl:np.float64(0.001655576513314827) - rollout_corr/rollout_ppl:np.float64(1.2751760962419212) - rollout_corr/rollout_log_ppl:np.float64(0.23888619328499772) - rollout_corr/log_ppl_diff:np.float64(0.0011609113280428573) - rollout_corr/log_ppl_abs_diff:np.float64(0.003236807693610899) - rollout_corr/log_ppl_diff_max:np.float64(0.017100274562835693) - rollout_corr/log_ppl_diff_min:np.float64(-0.02350728213787079) - rollout_corr/ppl_ratio:np.float64(1.0011714671272784) - rollout_corr/chi2_token:np.float64(0.004160782555118203) - rollout_corr/chi2_seq:np.float64(5.24587120115757) - actor/pg_loss:np.float64(0.0036287190159782767) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001126493662923167) - actor/ppo_kl:np.float64(0.0005246336532138685) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(249329.88290784857) - self_distillation/token_reweight_w_std:np.float64(2866482.3437989554) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994784640148282) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06014655635226518) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.471420306712389) - perf/mfu/actor:np.float64(0.033855896624554245) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.65926361083984) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004662652965635061 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004662652965635061 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:272.171875 - response_length/max:628.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.171875 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.801262497901917e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2853057831525803) - timing_s/agent_loop/generate_sequences/max:np.float64(5.174439303576946) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.863578474461974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.174439303576946) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:351 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:10.778497187420726 - timing_s/reward:0.052504874765872955 - timing_s/old_log_prob:2.3092593122273684 - timing_s/adv:0.5003343336284161 - timing_s/update_actor:15.202262729406357 - timing_s/step:28.936630526557565 - timing_s/stop_profile:0.00012007169425487518 - timing_per_token_ms/adv:0.0034652071753083088 - timing_per_token_ms/gen:0.1546945460046605 - timing_per_token_ms/update_actor:0.10528757742614592 - perf/total_num_tokens:144388 - perf/time_per_step:28.936630526557565 - perf/throughput:623.7250043136634 (TaskRunner pid=2026458) Training Progress: 67%|██████▋ | 67/100 [47:21<19:57, 36.28s/it] (TaskRunner pid=2026458) step:68 - global_seqlen/min:14735 - global_seqlen/max:19627 - global_seqlen/minmax_diff:4892 - global_seqlen/balanced_min:17371 - global_seqlen/balanced_max:17375 - global_seqlen/mean:17373.75 - actor/entropy:0.26004546880722046 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23114943504333496 - training/rollout_probs_diff_mean:0.005509829614311457 - training/rollout_probs_diff_std:0.013461560010910034 - training/rollout_actor_probs_pearson_corr:0.9983240365982056 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58984375 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58984375 - rollout_corr/rollout_is_mean:1.0001424551010132 - rollout_corr/rollout_is_ratio_fraction_high:3.430884680710733e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.14632738486398e-05 - rollout_corr/rollout_is_max:2.094249725341797 - rollout_corr/rollout_is_min:0.4616723358631134 - rollout_corr/rollout_is_std:0.04154840111732483 - rollout_corr/rollout_is_eff_sample_size:0.998277299143981 - rollout_corr/rollout_is_seq_mean:1.0000828504562378 - rollout_corr/rollout_is_seq_std:0.0031394551042467356 - rollout_corr/rollout_is_seq_max:1.0125868320465088 - rollout_corr/rollout_is_seq_min:0.9910873770713806 - rollout_corr/rollout_is_seq_max_deviation:0.012586832046508789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3149044853635132) - rollout_corr/training_log_ppl:np.float64(0.26928867847891524) - rollout_corr/kl:np.float64(0.0007745910923624422) - rollout_corr/k3_kl:np.float64(0.0011792316492460486) - rollout_corr/rollout_ppl:np.float64(1.3139065788127482) - rollout_corr/rollout_log_ppl:np.float64(0.26851408526999876) - rollout_corr/log_ppl_diff:np.float64(0.0007745932089164853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025513150612823665) - rollout_corr/log_ppl_diff_max:np.float64(0.009900659322738647) - rollout_corr/log_ppl_diff_min:np.float64(-0.015614360570907593) - rollout_corr/ppl_ratio:np.float64(1.0007805349305272) - rollout_corr/chi2_token:np.float64(0.0031827117782086134) - rollout_corr/chi2_seq:np.float64(0.9155827462673187) - actor/pg_loss:np.float64(0.0037549110129475594) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007232226507767336) - actor/ppo_kl:np.float64(-4.5092836657545377e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(174603.2610843731) - self_distillation/token_reweight_w_std:np.float64(2132750.0757954237) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9986913346219808) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0688194558606483) - self_distillation/empty_target_batch:np.float64(0.41015625) - actor/grad_norm:np.float64(0.4323424845933914) - perf/mfu/actor:np.float64(0.03227397909516918) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.99407958984375) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006370724178850651 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006370724178850651 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:227.7109375 - response_length/max:542.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.7109375 - response_length_non_aborted/max:542.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010382384061813354 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2069221679121256) - timing_s/agent_loop/generate_sequences/max:np.float64(4.399640990421176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.518257544754306) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.399640990421176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:508 - timing_s/agent_loop/slowest/response_length:542 - timing_s/gen:10.388320125639439 - timing_s/reward:0.053288890048861504 - timing_s/old_log_prob:2.246592290699482 - timing_s/adv:0.4770530089735985 - timing_s/update_actor:15.537012735381722 - timing_s/step:28.787579828873277 - timing_s/stop_profile:0.00013300031423568726 - timing_per_token_ms/adv:0.003432282962613127 - timing_per_token_ms/gen:0.1782056493916945 - timing_per_token_ms/update_actor:0.1117851121331155 - perf/total_num_tokens:138990 - perf/time_per_step:28.787579828873277 - perf/throughput:603.5154779692363 (TaskRunner pid=2026458) Training Progress: 68%|██████▊ | 68/100 [47:50<18:09, 34.04s/it] (TaskRunner pid=2026458) step:69 - global_seqlen/min:15072 - global_seqlen/max:24554 - global_seqlen/minmax_diff:9482 - global_seqlen/balanced_min:18606 - global_seqlen/balanced_max:18614 - global_seqlen/mean:18608.375 - actor/entropy:0.24838018417358398 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21835267543792725 - training/rollout_probs_diff_mean:0.004963765386492014 - training/rollout_probs_diff_std:0.012902619317173958 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9996421337127686 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.750844502472319e-05 - rollout_corr/rollout_is_max:1.9659780263900757 - rollout_corr/rollout_is_min:0.26981014013290405 - rollout_corr/rollout_is_std:0.039976488798856735 - rollout_corr/rollout_is_eff_sample_size:0.9984033607144375 - rollout_corr/rollout_is_seq_mean:0.9996398687362671 - rollout_corr/rollout_is_seq_std:0.0026584318839013577 - rollout_corr/rollout_is_seq_max:1.0071197748184204 - rollout_corr/rollout_is_seq_min:0.9907388687133789 - rollout_corr/rollout_is_seq_max_deviation:0.009261131286621094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3142020958475769) - rollout_corr/training_log_ppl:np.float64(0.2681919914903119) - rollout_corr/kl:np.float64(0.0011005508938133346) - rollout_corr/k3_kl:np.float64(0.0010078800825397138) - rollout_corr/rollout_ppl:np.float64(1.3127118106931448) - rollout_corr/rollout_log_ppl:np.float64(0.2670914397749584) - rollout_corr/log_ppl_diff:np.float64(0.0011005517153535038) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023552981147076935) - rollout_corr/log_ppl_diff_max:np.float64(0.013139069080352783) - rollout_corr/log_ppl_diff_min:np.float64(-0.007561713457107544) - rollout_corr/ppl_ratio:np.float64(1.001105590723455) - rollout_corr/chi2_token:np.float64(0.0017845977563410997) - rollout_corr/chi2_seq:np.float64(0.37655540159903467) - actor/pg_loss:np.float64(0.004595633479766548) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005241842873147107) - actor/ppo_kl:np.float64(-0.0001359343073588093) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(242376.49935866683) - self_distillation/token_reweight_w_std:np.float64(3279716.472702818) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019234952051193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08817204824299552) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.47684382647275925) - perf/mfu/actor:np.float64(0.03472723156176391) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89558792114258) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034019839949905872 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034019839949905872 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.69921875 - response_length/max:979.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.69921875 - response_length_non_aborted/max:979.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001641698181629181 - timing_s/agent_loop/generate_sequences/min:np.float64(1.224152859300375) - timing_s/agent_loop/generate_sequences/max:np.float64(6.517763238400221) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8646460080635734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.517763238400221) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:979 - timing_s/gen:12.344549676403403 - timing_s/reward:0.0557593759149313 - timing_s/old_log_prob:2.2437479849904776 - timing_s/adv:0.49268065579235554 - timing_s/update_actor:14.758547224104404 - timing_s/step:29.983076663687825 - timing_s/stop_profile:0.00012305378913879395 - timing_per_token_ms/adv:0.003309535731843562 - timing_per_token_ms/gen:0.17747896882184463 - timing_per_token_ms/update_actor:0.09913914584229147 - perf/total_num_tokens:148867 - perf/time_per_step:29.983076663687825 - perf/throughput:620.6292705957158 (TaskRunner pid=2026458) Training Progress: 69%|██████▉ | 69/100 [48:20<16:58, 32.84s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". To determine the correct answer, we need to analyze the structure of the product and compare it with the options. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a nitrile group (–n) attached to a cyclohexyl ring with a chlorine atom and a trifluoromethyl group (–C(F)(F)F). The structure also includes a benzene ring with a chlorine atom and a trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A: Contains a nitrile group and a cyclohexyl ring with a chlorine atom and a trifluoromethyl group, matching the product's structure. (TaskRunner pid=2026458) - Option B: Contains a nitrile group but does not have the correct trifluoromethyl group or the correct chlorine atom placement. (TaskRunner pid=2026458) - Option C: Contains a nitrile group but does not match the correct structure of the product, especially the trifluoromethyl group. (TaskRunner pid=2026458) - Option D: Contains a nitrile group but does not have the correct structure of the product, especially the trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2026458) step:70 - global_seqlen/min:12837 - global_seqlen/max:19669 - global_seqlen/minmax_diff:6832 - global_seqlen/balanced_min:17821 - global_seqlen/balanced_max:17831 - global_seqlen/mean:17825.125 - actor/entropy:0.2367510199546814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33400535583496094 - training/rollout_probs_diff_mean:0.005260511301457882 - training/rollout_probs_diff_std:0.013495014049112797 - training/rollout_actor_probs_pearson_corr:0.9982122778892517 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.9999286532402039 - rollout_corr/rollout_is_ratio_fraction_high:4.527823330136016e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5092745343281422e-05 - rollout_corr/rollout_is_max:2.0651345252990723 - rollout_corr/rollout_is_min:0.28940898180007935 - rollout_corr/rollout_is_std:0.04225678741931915 - rollout_corr/rollout_is_eff_sample_size:0.9982174280795298 - rollout_corr/rollout_is_seq_mean:0.9999866485595703 - rollout_corr/rollout_is_seq_std:0.00303856679238379 - rollout_corr/rollout_is_seq_max:1.0101226568222046 - rollout_corr/rollout_is_seq_min:0.9901509284973145 - rollout_corr/rollout_is_seq_max_deviation:0.01012265682220459 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2921742992475629) - rollout_corr/training_log_ppl:np.float64(0.2528939112526132) - rollout_corr/kl:np.float64(0.0014884298633148774) - rollout_corr/k3_kl:np.float64(0.0010952503159273874) - rollout_corr/rollout_ppl:np.float64(1.2902372791431844) - rollout_corr/rollout_log_ppl:np.float64(0.2514054805797059) - rollout_corr/log_ppl_diff:np.float64(0.0014884306729072705) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028520045889308676) - rollout_corr/log_ppl_diff_max:np.float64(0.0136488676071167) - rollout_corr/log_ppl_diff_min:np.float64(-0.008753955364227295) - rollout_corr/ppl_ratio:np.float64(1.0014950232580304) - rollout_corr/chi2_token:np.float64(0.0014098831452429295) - rollout_corr/chi2_seq:np.float64(0.360071137547493) - actor/pg_loss:np.float64(0.0027717185439541936) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434370364630013) - actor/ppo_kl:np.float64(0.0005119445677905787) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(69071.25273821992) - self_distillation/token_reweight_w_std:np.float64(1045658.8316577639) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004556903149933) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06493389664683491) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.3887019380927086) - perf/mfu/actor:np.float64(0.0325070599234879) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.87537002563477) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07272307414099477) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8114142857142858) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06774926114873739) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.712047619047619) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0541181009825183) - val-aux/sciknoweval/score/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/score/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/score/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07272307414099477) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8114142857142858) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06774926114873739) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.712047619047619) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0541181009825183) - val-core/sciknoweval/acc/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07272307414099477) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8114142857142858) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06774926114873739) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.712047619047619) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0541181009825183) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:1 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036373515613377094 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036373515613377094 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:258.81640625 - response_length/max:938.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:258.81640625 - response_length_non_aborted/max:938.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.21875 - prompt_length/max:701.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016323477029800415 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2699729204177856) - timing_s/agent_loop/generate_sequences/max:np.float64(6.416786439716816) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.761028841654479) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.416786439716816) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:938 - timing_s/gen:12.333267584443092 - timing_s/reward:0.05591795593500137 - timing_s/old_log_prob:2.2401132248342037 - timing_s/adv:0.506872920319438 - timing_s/update_actor:15.221702938899398 - timing_s/step:30.44651348516345 - timing_s/testing:81.57863584533334 - timing_s/save_checkpoint:6.6424451023340225 - timing_s/stop_profile:0.00010693445801734924 - timing_per_token_ms/adv:0.003554483631387143 - timing_per_token_ms/gen:0.18614286165149482 - timing_per_token_ms/update_actor:0.1067433113295096 - perf/total_num_tokens:142601 - perf/time_per_step:30.44651348516345 - perf/throughput:585.4570182128132 (TaskRunner pid=2026458) Training Progress: 70%|███████ | 70/100 [50:19<29:18, 58.61s/it] (TaskRunner pid=2026458) step:71 - global_seqlen/min:15483 - global_seqlen/max:28966 - global_seqlen/minmax_diff:13483 - global_seqlen/balanced_min:18380 - global_seqlen/balanced_max:25891 - global_seqlen/mean:19333.5 - actor/entropy:0.21173985302448273 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4046512842178345 - training/rollout_probs_diff_mean:0.0045140874572098255 - training/rollout_probs_diff_std:0.012615382671356201 - training/rollout_actor_probs_pearson_corr:0.9983184337615967 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998964667320251 - rollout_corr/rollout_is_ratio_fraction_high:1.2943308320245706e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.471654342021793e-05 - rollout_corr/rollout_is_max:2.0498809814453125 - rollout_corr/rollout_is_min:0.38866347074508667 - rollout_corr/rollout_is_std:0.03843024745583534 - rollout_corr/rollout_is_eff_sample_size:0.9985250563653204 - rollout_corr/rollout_is_seq_mean:0.9998904466629028 - rollout_corr/rollout_is_seq_std:0.0028965510427951813 - rollout_corr/rollout_is_seq_max:1.0107746124267578 - rollout_corr/rollout_is_seq_min:0.9922298192977905 - rollout_corr/rollout_is_seq_max_deviation:0.010774612426757812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2945193266496062) - rollout_corr/training_log_ppl:np.float64(0.2542406297643538) - rollout_corr/kl:np.float64(0.0010824492369110317) - rollout_corr/k3_kl:np.float64(0.0011051825894128342) - rollout_corr/rollout_ppl:np.float64(1.293056899216026) - rollout_corr/rollout_log_ppl:np.float64(0.2531581802359142) - rollout_corr/log_ppl_diff:np.float64(0.0010824495284396107) - rollout_corr/log_ppl_abs_diff:np.float64(0.00271785432141769) - rollout_corr/log_ppl_diff_max:np.float64(0.014131128787994385) - rollout_corr/log_ppl_diff_min:np.float64(-0.009531468152999878) - rollout_corr/ppl_ratio:np.float64(1.0010888520628214) - rollout_corr/chi2_token:np.float64(0.002212358172982931) - rollout_corr/chi2_seq:np.float64(0.6793141439557076) - actor/pg_loss:np.float64(0.002975177369080484) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000972424295014207) - actor/ppo_kl:np.float64(7.461081707615946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(134136.12361396686) - self_distillation/token_reweight_w_std:np.float64(1624645.7259825736) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988281566184014) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07092063533491455) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.4537533223628998) - perf/mfu/actor:np.float64(0.03374089352858298) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.65079879760742) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06832286715507507 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06832286715507507 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.796875 - response_length/max:8192.0 - response_length/min:93.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:301.796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.677132427692413e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1272850260138512) - timing_s/agent_loop/generate_sequences/max:np.float64(47.67028555460274) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.04078216401831) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(47.67028555460274) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:666 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:53.34413158334792 - timing_s/reward:0.05393144115805626 - timing_s/old_log_prob:2.626548172906041 - timing_s/adv:0.5594039708375931 - timing_s/update_actor:16.17577089741826 - timing_s/step:72.8568678945303 - timing_s/stop_profile:2.8194859623908997e-05 - timing_per_token_ms/adv:0.0036168048389944468 - timing_per_token_ms/gen:0.6904495415913527 - timing_per_token_ms/update_actor:0.1045838240451694 - perf/total_num_tokens:154668 - perf/time_per_step:72.8568678945303 - perf/throughput:265.3627661840711 (TaskRunner pid=2026458) Training Progress: 71%|███████ | 71/100 [51:32<30:23, 62.89s/it] (TaskRunner pid=2026458) step:72 - global_seqlen/min:14180 - global_seqlen/max:19404 - global_seqlen/minmax_diff:5224 - global_seqlen/balanced_min:16784 - global_seqlen/balanced_max:16786 - global_seqlen/mean:16785.25 - actor/entropy:0.2532234191894531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37658625841140747 - training/rollout_probs_diff_mean:0.005358405876904726 - training/rollout_probs_diff_std:0.01324445940554142 - training/rollout_actor_probs_pearson_corr:0.9983224272727966 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.000357747077942 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6467605829238892 - rollout_corr/rollout_is_min:0.5746954679489136 - rollout_corr/rollout_is_std:0.04072979837656021 - rollout_corr/rollout_is_eff_sample_size:0.9983450191216581 - rollout_corr/rollout_is_seq_mean:1.0003376007080078 - rollout_corr/rollout_is_seq_std:0.0029273421969264746 - rollout_corr/rollout_is_seq_max:1.0102252960205078 - rollout_corr/rollout_is_seq_min:0.9919271469116211 - rollout_corr/rollout_is_seq_max_deviation:0.010225296020507812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3001699862070382) - rollout_corr/training_log_ppl:np.float64(0.25781238308991306) - rollout_corr/kl:np.float64(0.0006664992815785808) - rollout_corr/k3_kl:np.float64(0.0009609586412011595) - rollout_corr/rollout_ppl:np.float64(1.2992715733125806) - rollout_corr/rollout_log_ppl:np.float64(0.25714588271512184) - rollout_corr/log_ppl_diff:np.float64(0.0006665003747912124) - rollout_corr/log_ppl_abs_diff:np.float64(0.002382488935836591) - rollout_corr/log_ppl_diff_max:np.float64(0.009035348892211914) - rollout_corr/log_ppl_diff_min:np.float64(-0.009852290153503418) - rollout_corr/ppl_ratio:np.float64(1.0006713322363794) - rollout_corr/chi2_token:np.float64(0.0025634136982262135) - rollout_corr/chi2_seq:np.float64(1.15624994575046) - actor/pg_loss:np.float64(0.003654895001091063) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009709688329166966) - actor/ppo_kl:np.float64(0.00014433233151223135) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(213152.86371882842) - self_distillation/token_reweight_w_std:np.float64(2646677.052918058) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9974161058198661) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09049051211331971) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.43892450630664825) - perf/mfu/actor:np.float64(0.03149296183920366) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40920639038086) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009050944820046425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009050944820046425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:237.6953125 - response_length/max:467.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:237.6953125 - response_length_non_aborted/max:467.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.147497773170471e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3375506270676851) - timing_s/agent_loop/generate_sequences/max:np.float64(4.0367792677134275) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.622172743664123) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.0367792677134275) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:467 - timing_s/gen:10.345969874411821 - timing_s/reward:0.05066956579685211 - timing_s/old_log_prob:2.2474361546337605 - timing_s/adv:0.4828864671289921 - timing_s/update_actor:14.702434957027435 - timing_s/step:27.925694411620498 - timing_s/stop_profile:3.0016526579856873e-05 - timing_per_token_ms/adv:0.0035960625186472655 - timing_per_token_ms/gen:0.17002415570109813 - timing_per_token_ms/update_actor:0.1094892461910564 - perf/total_num_tokens:134282 - perf/time_per_step:27.925694411620498 - perf/throughput:601.0683119491305 (TaskRunner pid=2026458) Training Progress: 72%|███████▏ | 72/100 [52:00<24:27, 52.41s/it] (TaskRunner pid=2026458) step:73 - global_seqlen/min:15559 - global_seqlen/max:20190 - global_seqlen/minmax_diff:4631 - global_seqlen/balanced_min:17173 - global_seqlen/balanced_max:17174 - global_seqlen/mean:17173.5 - actor/entropy:0.24391600489616394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135395884513855 - training/rollout_probs_diff_mean:0.0049339784309268 - training/rollout_probs_diff_std:0.012776896357536316 - training/rollout_actor_probs_pearson_corr:0.998493492603302 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.000167965888977 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.0463656003121287e-05 - rollout_corr/rollout_is_max:1.767087697982788 - rollout_corr/rollout_is_min:0.47380420565605164 - rollout_corr/rollout_is_std:0.03922998905181885 - rollout_corr/rollout_is_eff_sample_size:0.9984639671449654 - rollout_corr/rollout_is_seq_mean:1.0002386569976807 - rollout_corr/rollout_is_seq_std:0.0029904802795499563 - rollout_corr/rollout_is_seq_max:1.0164623260498047 - rollout_corr/rollout_is_seq_min:0.9890998005867004 - rollout_corr/rollout_is_seq_max_deviation:0.016462326049804688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2968735643662512) - rollout_corr/training_log_ppl:np.float64(0.25511286556138657) - rollout_corr/kl:np.float64(0.0009459122673263209) - rollout_corr/k3_kl:np.float64(0.0009600993125502555) - rollout_corr/rollout_ppl:np.float64(1.2955679851584136) - rollout_corr/rollout_log_ppl:np.float64(0.2541669542551972) - rollout_corr/log_ppl_diff:np.float64(0.0009459113061893731) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024720369547139853) - rollout_corr/log_ppl_diff_max:np.float64(0.012520670890808105) - rollout_corr/log_ppl_diff_min:np.float64(-0.0084361732006073) - rollout_corr/ppl_ratio:np.float64(1.0009514002595097) - rollout_corr/chi2_token:np.float64(0.001954789971932769) - rollout_corr/chi2_seq:np.float64(0.6583140406291932) - actor/pg_loss:np.float64(0.0032611776841804385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000914203752472531) - actor/ppo_kl:np.float64(0.00038008014968227144) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(311191.56539016124) - self_distillation/token_reweight_w_std:np.float64(2860058.778589467) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015116163995117) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06564340484328568) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.42411575466394424) - perf/mfu/actor:np.float64(0.031904336867600726) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.16930770874023) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001247106003575027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001247106003575027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:256.453125 - response_length/max:556.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.453125 - response_length_non_aborted/max:556.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.804398536682129e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1476398799568415) - timing_s/agent_loop/generate_sequences/max:np.float64(4.690207960084081) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7287142683780985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.690207960084081) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:556 - timing_s/gen:10.320297587662935 - timing_s/reward:0.0441241730004549 - timing_s/old_log_prob:2.164985027164221 - timing_s/adv:0.4551666658371687 - timing_s/update_actor:14.973258456215262 - timing_s/step:27.992081409320235 - timing_s/stop_profile:2.880394458770752e-05 - timing_per_token_ms/adv:0.003313001614676454 - timing_per_token_ms/gen:0.15719700218824917 - timing_per_token_ms/update_actor:0.10898519853418975 - perf/total_num_tokens:137388 - perf/time_per_step:27.992081409320235 - perf/throughput:613.5127913096851 (TaskRunner pid=2026458) Training Progress: 73%|███████▎ | 73/100 [52:28<20:17, 45.09s/it] (TaskRunner pid=2026458) step:74 - global_seqlen/min:13447 - global_seqlen/max:21573 - global_seqlen/minmax_diff:8126 - global_seqlen/balanced_min:16831 - global_seqlen/balanced_max:16917 - global_seqlen/mean:16843.875 - actor/entropy:0.24996380507946014 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424726128578186 - training/rollout_probs_diff_mean:0.004949183203279972 - training/rollout_probs_diff_std:0.012561521492898464 - training/rollout_actor_probs_pearson_corr:0.9985408782958984 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998319149017334 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.6325607399921864e-05 - rollout_corr/rollout_is_max:1.7145717144012451 - rollout_corr/rollout_is_min:0.17540410161018372 - rollout_corr/rollout_is_std:0.03882066160440445 - rollout_corr/rollout_is_eff_sample_size:0.9984948673727597 - rollout_corr/rollout_is_seq_mean:0.9999223947525024 - rollout_corr/rollout_is_seq_std:0.0029323359485715628 - rollout_corr/rollout_is_seq_max:1.009851336479187 - rollout_corr/rollout_is_seq_min:0.9908139109611511 - rollout_corr/rollout_is_seq_max_deviation:0.009851336479187012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3159835655242205) - rollout_corr/training_log_ppl:np.float64(0.26891483918007) - rollout_corr/kl:np.float64(0.0010735450956096404) - rollout_corr/k3_kl:np.float64(0.0011378318484958072) - rollout_corr/rollout_ppl:np.float64(1.3145075496286154) - rollout_corr/rollout_log_ppl:np.float64(0.26784129165753257) - rollout_corr/log_ppl_diff:np.float64(0.0010735475225374103) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030432986677624285) - rollout_corr/log_ppl_diff_max:np.float64(0.012433648109436035) - rollout_corr/log_ppl_diff_min:np.float64(-0.009890466928482056) - rollout_corr/ppl_ratio:np.float64(1.0010810112580657) - rollout_corr/chi2_token:np.float64(0.0024086034391075373) - rollout_corr/chi2_seq:np.float64(1.130532833514735) - actor/pg_loss:np.float64(0.0028035463765263557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007331344913836801) - actor/ppo_kl:np.float64(0.00016281594710676472) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(183593.82668150146) - self_distillation/token_reweight_w_std:np.float64(2093931.331328835) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001840919721872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060318807751173154) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.3842148035764694) - perf/mfu/actor:np.float64(0.031244242735268143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2487654685974) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016619311645627022 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016619311645627022 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:252.96484375 - response_length/max:993.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:252.96484375 - response_length_non_aborted/max:993.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7175992727279663e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.240340730175376) - timing_s/agent_loop/generate_sequences/max:np.float64(6.539421858265996) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.619629321692628) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.539421858265996) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:993 - timing_s/gen:12.270534712821245 - timing_s/reward:0.04380444623529911 - timing_s/old_log_prob:2.2239825148135424 - timing_s/adv:0.5528004374355078 - timing_s/update_actor:15.180137295275927 - timing_s/step:30.30796923674643 - timing_s/stop_profile:5.2906572818756104e-05 - timing_per_token_ms/adv:0.004102384675701908 - timing_per_token_ms/gen:0.18947999062402515 - timing_per_token_ms/update_actor:0.11265324409671117 - perf/total_num_tokens:134751 - perf/time_per_step:30.30796923674643 - perf/throughput:555.7572950014052 (TaskRunner pid=2026458) Training Progress: 74%|███████▍ | 74/100 [52:58<17:37, 40.66s/it] (TaskRunner pid=2026458) step:75 - global_seqlen/min:14038 - global_seqlen/max:20039 - global_seqlen/minmax_diff:6001 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18457.625 - actor/entropy:0.24194937944412231 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4665491580963135 - training/rollout_probs_diff_mean:0.005109612364321947 - training/rollout_probs_diff_std:0.013375498354434967 - training/rollout_actor_probs_pearson_corr:0.9982732534408569 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9997668862342834 - rollout_corr/rollout_is_ratio_fraction_high:2.9664348403457552e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.449652260518633e-05 - rollout_corr/rollout_is_max:2.278925657272339 - rollout_corr/rollout_is_min:0.3029129207134247 - rollout_corr/rollout_is_std:0.040616225451231 - rollout_corr/rollout_is_eff_sample_size:0.9983523857083623 - rollout_corr/rollout_is_seq_mean:0.9997424483299255 - rollout_corr/rollout_is_seq_std:0.002785229589790106 - rollout_corr/rollout_is_seq_max:1.00969660282135 - rollout_corr/rollout_is_seq_min:0.9920895099639893 - rollout_corr/rollout_is_seq_max_deviation:0.009696602821350098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2928350591100752) - rollout_corr/training_log_ppl:np.float64(0.2525325442838948) - rollout_corr/kl:np.float64(0.0011320576819571215) - rollout_corr/k3_kl:np.float64(0.0010110275231340893) - rollout_corr/rollout_ppl:np.float64(1.2913245973177254) - rollout_corr/rollout_log_ppl:np.float64(0.2514004855765961) - rollout_corr/log_ppl_diff:np.float64(0.001132058707298711) - rollout_corr/log_ppl_abs_diff:np.float64(0.002472092892276123) - rollout_corr/log_ppl_diff_max:np.float64(0.01102399080991745) - rollout_corr/log_ppl_diff_min:np.float64(-0.008037805557250977) - rollout_corr/ppl_ratio:np.float64(1.0011374710593373) - rollout_corr/chi2_token:np.float64(0.0017729024402797222) - rollout_corr/chi2_seq:np.float64(0.6597908593248576) - actor/pg_loss:np.float64(0.0018322557443752885) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036231086232874077) - actor/ppo_kl:np.float64(-3.7132614664869834e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(138490.71220888244) - self_distillation/token_reweight_w_std:np.float64(1204680.9713628318) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000044918153435) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041466833936283365) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.29594044387340546) - perf/mfu/actor:np.float64(0.03412545800281451) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2256622314453) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:6.489075894933194e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:6.489075894933194e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:263.36328125 - response_length/max:680.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.36328125 - response_length_non_aborted/max:680.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.105005741119385e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.53872056491673) - timing_s/agent_loop/generate_sequences/max:np.float64(5.3331572357565165) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.911040708415385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.3331572357565165) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:367 - timing_s/agent_loop/slowest/response_length:680 - timing_s/gen:11.2033476177603 - timing_s/reward:0.055844465270638466 - timing_s/old_log_prob:2.219141924753785 - timing_s/adv:0.4940295107662678 - timing_s/update_actor:15.452317954972386 - timing_s/step:29.51427774503827 - timing_s/stop_profile:0.0001329611986875534 - timing_per_token_ms/adv:0.003345700697992481 - timing_per_token_ms/gen:0.16617000070838908 - timing_per_token_ms/update_actor:0.10464725252417623 - perf/total_num_tokens:147661 - perf/time_per_step:29.51427774503827 - perf/throughput:625.3795251046915 (TaskRunner pid=2026458) Training Progress: 75%|███████▌ | 75/100 [53:28<15:33, 37.33s/it] (TaskRunner pid=2026458) step:76 - global_seqlen/min:15400 - global_seqlen/max:20933 - global_seqlen/minmax_diff:5533 - global_seqlen/balanced_min:18318 - global_seqlen/balanced_max:18443 - global_seqlen/mean:18349.25 - actor/entropy:0.23565568029880524 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6839720010757446 - training/rollout_probs_diff_mean:0.00476441252976656 - training/rollout_probs_diff_std:0.01295106578618288 - training/rollout_actor_probs_pearson_corr:0.9984387159347534 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:0.999829113483429 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.072877345606685e-05 - rollout_corr/rollout_is_max:1.8591279983520508 - rollout_corr/rollout_is_min:0.14236801862716675 - rollout_corr/rollout_is_std:0.038981545716524124 - rollout_corr/rollout_is_eff_sample_size:0.9984822693566601 - rollout_corr/rollout_is_seq_mean:0.9998883008956909 - rollout_corr/rollout_is_seq_std:0.0026900693774223328 - rollout_corr/rollout_is_seq_max:1.0127232074737549 - rollout_corr/rollout_is_seq_min:0.9910237193107605 - rollout_corr/rollout_is_seq_max_deviation:0.012723207473754883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3050027494318783) - rollout_corr/training_log_ppl:np.float64(0.26096004332066514) - rollout_corr/kl:np.float64(0.001120272666678801) - rollout_corr/k3_kl:np.float64(0.000998899616035942) - rollout_corr/rollout_ppl:np.float64(1.3034690767526627) - rollout_corr/rollout_log_ppl:np.float64(0.25983976959832944) - rollout_corr/log_ppl_diff:np.float64(0.0011202737223356962) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024523677420802414) - rollout_corr/log_ppl_diff_max:np.float64(0.01079338788986206) - rollout_corr/log_ppl_diff_min:np.float64(-0.011696979403495789) - rollout_corr/ppl_ratio:np.float64(1.001125503797084) - rollout_corr/chi2_token:np.float64(0.0016837881412357092) - rollout_corr/chi2_seq:np.float64(0.4817654227372259) - actor/pg_loss:np.float64(0.0022594364127144217) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005095393812553084) - actor/ppo_kl:np.float64(0.000131376795138749) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(86914.96509032976) - self_distillation/token_reweight_w_std:np.float64(1369863.7220702441) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999393313191831) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041579058073693886) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3407774642109871) - perf/mfu/actor:np.float64(0.034170829781458016) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.09953689575195) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002358874771744013 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002358874771744013 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:287.7265625 - response_length/max:956.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.7265625 - response_length_non_aborted/max:956.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015220046043395996 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3281499948352575) - timing_s/agent_loop/generate_sequences/max:np.float64(6.804745152592659) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0793025807652157) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.804745152592659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:306 - timing_s/agent_loop/slowest/response_length:956 - timing_s/gen:12.672567792236805 - timing_s/reward:0.05679433047771454 - timing_s/old_log_prob:2.200064066797495 - timing_s/adv:0.5124469976872206 - timing_s/update_actor:15.322908757254481 - timing_s/step:30.852899638935924 - timing_s/stop_profile:0.00013393722474575043 - timing_per_token_ms/adv:0.0034909260438929423 - timing_per_token_ms/gen:0.17204604784594754 - timing_per_token_ms/update_actor:0.10438375381319728 - perf/total_num_tokens:146794 - perf/time_per_step:30.852899638935924 - perf/throughput:594.7334031723717 (TaskRunner pid=2026458) Training Progress: 76%|███████▌ | 76/100 [53:59<14:09, 35.40s/it] (TaskRunner pid=2026458) step:77 - global_seqlen/min:16068 - global_seqlen/max:21136 - global_seqlen/minmax_diff:5068 - global_seqlen/balanced_min:18347 - global_seqlen/balanced_max:18355 - global_seqlen/mean:18353.125 - actor/entropy:0.24358664453029633 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3763711750507355 - training/rollout_probs_diff_mean:0.005085653159767389 - training/rollout_probs_diff_std:0.012961719185113907 - training/rollout_actor_probs_pearson_corr:0.9984139204025269 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999815821647644 - rollout_corr/rollout_is_ratio_fraction_high:1.5232524674502201e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.61626215535216e-05 - rollout_corr/rollout_is_max:2.443542003631592 - rollout_corr/rollout_is_min:0.0030977046117186546 - rollout_corr/rollout_is_std:0.04062869772315025 - rollout_corr/rollout_is_eff_sample_size:0.9983521480748362 - rollout_corr/rollout_is_seq_mean:1.0000320672988892 - rollout_corr/rollout_is_seq_std:0.0030800357926636934 - rollout_corr/rollout_is_seq_max:1.008837342262268 - rollout_corr/rollout_is_seq_min:0.9924508333206177 - rollout_corr/rollout_is_seq_max_deviation:0.008837342262268066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.30547040887177) - rollout_corr/training_log_ppl:np.float64(0.262247261358425) - rollout_corr/kl:np.float64(0.001232846368517393) - rollout_corr/k3_kl:np.float64(0.0012621798805412254) - rollout_corr/rollout_ppl:np.float64(1.303758448921144) - rollout_corr/rollout_log_ppl:np.float64(0.2610144141508499) - rollout_corr/log_ppl_diff:np.float64(0.0012328472075751051) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028479194588726386) - rollout_corr/log_ppl_diff_max:np.float64(0.01525259017944336) - rollout_corr/log_ppl_diff_min:np.float64(-0.007772982120513916) - rollout_corr/ppl_ratio:np.float64(1.0012400422710925) - rollout_corr/chi2_token:np.float64(0.0023893536999821663) - rollout_corr/chi2_seq:np.float64(0.8159184374380857) - actor/pg_loss:np.float64(0.002790838829241693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009325358823843999) - actor/ppo_kl:np.float64(0.0003359223030457992) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(180959.5167653456) - self_distillation/token_reweight_w_std:np.float64(2185127.907001757) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988750158809125) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06124208416440524) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.4322587177157402) - perf/mfu/actor:np.float64(0.034241717008689325) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1767692565918) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0021534981206059456 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0021534981206059456 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:256.44140625 - response_length/max:782.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.44140625 - response_length_non_aborted/max:782.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013564899563789368 - timing_s/agent_loop/generate_sequences/min:np.float64(1.442766698077321) - timing_s/agent_loop/generate_sequences/max:np.float64(5.666013702750206) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7960805580951273) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.666013702750206) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:368 - timing_s/agent_loop/slowest/response_length:782 - timing_s/gen:11.443953750655055 - timing_s/reward:0.054556334391236305 - timing_s/old_log_prob:2.218324951827526 - timing_s/adv:0.4792825896292925 - timing_s/update_actor:15.45035495236516 - timing_s/step:29.73319257237017 - timing_s/stop_profile:2.9521062970161438e-05 - timing_per_token_ms/adv:0.0032643118653450875 - timing_per_token_ms/gen:0.1743203057267446 - timing_per_token_ms/update_actor:0.10522972894510581 - perf/total_num_tokens:146825 - perf/time_per_step:29.73319257237017 - perf/throughput:617.2604894455499 (TaskRunner pid=2026458) Training Progress: 77%|███████▋ | 77/100 [54:28<12:55, 33.71s/it] (TaskRunner pid=2026458) step:78 - global_seqlen/min:15356 - global_seqlen/max:19696 - global_seqlen/minmax_diff:4340 - global_seqlen/balanced_min:17649 - global_seqlen/balanced_max:17655 - global_seqlen/mean:17652.75 - actor/entropy:0.23699037730693817 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975830912590027 - training/rollout_probs_diff_mean:0.005081570707261562 - training/rollout_probs_diff_std:0.013243689201772213 - training/rollout_actor_probs_pearson_corr:0.9983371496200562 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999217987060547 - rollout_corr/rollout_is_ratio_fraction_high:4.307003109715879e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.307003109715879e-05 - rollout_corr/rollout_is_max:2.9357402324676514 - rollout_corr/rollout_is_min:0.4458056390285492 - rollout_corr/rollout_is_std:0.04097927734255791 - rollout_corr/rollout_is_eff_sample_size:0.9983230388235971 - rollout_corr/rollout_is_seq_mean:1.0000085830688477 - rollout_corr/rollout_is_seq_std:0.002981689525768161 - rollout_corr/rollout_is_seq_max:1.0086852312088013 - rollout_corr/rollout_is_seq_min:0.9888802170753479 - rollout_corr/rollout_is_seq_max_deviation:0.0111197829246521 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3004110623151064) - rollout_corr/training_log_ppl:np.float64(0.257169630160206) - rollout_corr/kl:np.float64(0.0012380461494814199) - rollout_corr/k3_kl:np.float64(0.001083872083654569) - rollout_corr/rollout_ppl:np.float64(1.2987679266370833) - rollout_corr/rollout_log_ppl:np.float64(0.25593158273841254) - rollout_corr/log_ppl_diff:np.float64(0.0012380474217934534) - rollout_corr/log_ppl_abs_diff:np.float64(0.002574279482359998) - rollout_corr/log_ppl_diff_max:np.float64(0.015157654881477356) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076891034841537476) - rollout_corr/ppl_ratio:np.float64(1.0012440220452845) - rollout_corr/chi2_token:np.float64(0.001788182882592082) - rollout_corr/chi2_seq:np.float64(1.9581220541149378) - actor/pg_loss:np.float64(0.0033810948953032494) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009038109546963824) - actor/ppo_kl:np.float64(0.00033419562218028886) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(189436.75045557343) - self_distillation/token_reweight_w_std:np.float64(2503367.794994272) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009928282815963) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07160566429956816) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3780764937400818) - perf/mfu/actor:np.float64(0.03291894268049457) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0838165283203) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019830160308629274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019830160308629274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.0859375 - response_length/max:624.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.0859375 - response_length_non_aborted/max:624.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.438022971153259e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3432003632187843) - timing_s/agent_loop/generate_sequences/max:np.float64(5.082683224231005) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9419547133074957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.082683224231005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:624 - timing_s/gen:10.891948658972979 - timing_s/reward:0.054630380123853683 - timing_s/old_log_prob:2.192325107753277 - timing_s/adv:0.4803696293383837 - timing_s/update_actor:15.322288809344172 - timing_s/step:29.02985128760338 - timing_s/stop_profile:0.00014253705739974976 - timing_per_token_ms/adv:0.003401521217220997 - timing_per_token_ms/gen:0.15637219196274413 - timing_per_token_ms/update_actor:0.10849788849714756 - perf/total_num_tokens:141222 - perf/time_per_step:29.02985128760338 - perf/throughput:608.0895773495835 (TaskRunner pid=2026458) Training Progress: 78%|███████▊ | 78/100 [54:57<11:51, 32.32s/it] (TaskRunner pid=2026458) step:79 - global_seqlen/min:13318 - global_seqlen/max:19723 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16855 - global_seqlen/mean:16852.75 - actor/entropy:0.2380763292312622 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24131831526756287 - training/rollout_probs_diff_mean:0.00509747164323926 - training/rollout_probs_diff_std:0.01309633906930685 - training/rollout_actor_probs_pearson_corr:0.998324453830719 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.999850869178772 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4865467164781876e-05 - rollout_corr/rollout_is_max:1.6486144065856934 - rollout_corr/rollout_is_min:0.48596420884132385 - rollout_corr/rollout_is_std:0.04019506648182869 - rollout_corr/rollout_is_eff_sample_size:0.998386606117186 - rollout_corr/rollout_is_seq_mean:0.9999116659164429 - rollout_corr/rollout_is_seq_std:0.0028780419379472733 - rollout_corr/rollout_is_seq_max:1.0085704326629639 - rollout_corr/rollout_is_seq_min:0.9908472299575806 - rollout_corr/rollout_is_seq_max_deviation:0.009152770042419434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2909451411105692) - rollout_corr/training_log_ppl:np.float64(0.250693125941325) - rollout_corr/kl:np.float64(0.0009612457183649781) - rollout_corr/k3_kl:np.float64(0.0010165603632685816) - rollout_corr/rollout_ppl:np.float64(1.2896752119995654) - rollout_corr/rollout_log_ppl:np.float64(0.24973187799332663) - rollout_corr/log_ppl_diff:np.float64(0.0009612479479983449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024955651024356484) - rollout_corr/log_ppl_diff_max:np.float64(0.008482903242111206) - rollout_corr/log_ppl_diff_min:np.float64(-0.007829368114471436) - rollout_corr/ppl_ratio:np.float64(1.0009662464726716) - rollout_corr/chi2_token:np.float64(0.002142100827768445) - rollout_corr/chi2_seq:np.float64(0.36312326532788575) - actor/pg_loss:np.float64(0.0036202146438881755) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008493167902088317) - actor/ppo_kl:np.float64(1.813888876256442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(207000.82824485935) - self_distillation/token_reweight_w_std:np.float64(2774328.0958690653) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004315504338592) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07667552542989142) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5015818998217583) - perf/mfu/actor:np.float64(0.031159245026246143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.08302688598633) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000724691548384726 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000724691548384726 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:262.7734375 - response_length/max:630.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.7734375 - response_length_non_aborted/max:630.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013626180589199066 - timing_s/agent_loop/generate_sequences/min:np.float64(1.074292877689004) - timing_s/agent_loop/generate_sequences/max:np.float64(4.940646631643176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7851551758358255) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.940646631643176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:630 - timing_s/gen:11.244631016626954 - timing_s/reward:0.05328981205821037 - timing_s/old_log_prob:2.2739136423915625 - timing_s/adv:0.5032551754266024 - timing_s/update_actor:15.453622449189425 - timing_s/step:29.61371649429202 - timing_s/stop_profile:0.00012503564357757568 - timing_per_token_ms/adv:0.0037327377981828066 - timing_per_token_ms/gen:0.16715669714028472 - timing_per_token_ms/update_actor:0.1146224091705317 - perf/total_num_tokens:134822 - perf/time_per_step:29.61371649429202 - perf/throughput:569.0859505340484 (TaskRunner pid=2026458) Training Progress: 79%|███████▉ | 79/100 [55:27<11:02, 31.52s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group, matching the product structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but does not have the correct nitrile group and trifluoromethyl group structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" has a different structure with an oxygen atom and a different substitution pattern, not matching the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a different structure with a double bond and a different substitution pattern, not matching the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2026458) step:80 - global_seqlen/min:16122 - global_seqlen/max:23121 - global_seqlen/minmax_diff:6999 - global_seqlen/balanced_min:18851 - global_seqlen/balanced_max:18860 - global_seqlen/mean:18857.375 - actor/entropy:0.22489960491657257 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3807961344718933 - training/rollout_probs_diff_mean:0.005050069652497768 - training/rollout_probs_diff_std:0.013410150073468685 - training/rollout_actor_probs_pearson_corr:0.998170018196106 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0001280307769775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.252001963322982e-05 - rollout_corr/rollout_is_max:1.815222144126892 - rollout_corr/rollout_is_min:0.23840588331222534 - rollout_corr/rollout_is_std:0.040402136743068695 - rollout_corr/rollout_is_eff_sample_size:0.9983706838295904 - rollout_corr/rollout_is_seq_mean:1.0001842975616455 - rollout_corr/rollout_is_seq_std:0.002584717469289899 - rollout_corr/rollout_is_seq_max:1.0079708099365234 - rollout_corr/rollout_is_seq_min:0.9935450553894043 - rollout_corr/rollout_is_seq_max_deviation:0.007970809936523438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.284651028458029) - rollout_corr/training_log_ppl:np.float64(0.2446684459282551) - rollout_corr/kl:np.float64(0.0011069629296169836) - rollout_corr/k3_kl:np.float64(0.0011388705046897485) - rollout_corr/rollout_ppl:np.float64(1.2831353461369872) - rollout_corr/rollout_log_ppl:np.float64(0.24356148090737406) - rollout_corr/log_ppl_diff:np.float64(0.0011069650208810344) - rollout_corr/log_ppl_abs_diff:np.float64(0.002547982890973799) - rollout_corr/log_ppl_diff_max:np.float64(0.011913180351257324) - rollout_corr/log_ppl_diff_min:np.float64(-0.00954301655292511) - rollout_corr/ppl_ratio:np.float64(1.0011127449106425) - rollout_corr/chi2_token:np.float64(0.0022547231055796146) - rollout_corr/chi2_seq:np.float64(0.705732943257317) - actor/pg_loss:np.float64(0.003671735990792513) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012357970999801182) - actor/ppo_kl:np.float64(0.00041334695177397407) - actor/pg_clipfrac_lower:np.float64(2.100134406646248e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(147860.0034218824) - self_distillation/token_reweight_w_std:np.float64(2243142.9639211292) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017643314786255) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08743216533912346) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.46084368228912354) - perf/mfu/actor:np.float64(0.0348035546181388) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0935287475586) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07152449490492849) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8101809523809524) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05480896953178682) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6956380952380953) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04910459662444814) - val-aux/sciknoweval/score/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07152449490492849) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8101809523809524) - val-aux/sciknoweval/score/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05480896953178682) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6956380952380953) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04910459662444814) - val-core/sciknoweval/acc/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07152449490492849) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8101809523809524) - val-core/sciknoweval/acc/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05480896953178682) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6956380952380953) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04910459662444814) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0028665580321103334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0028665580321103334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.60546875 - response_length/max:780.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.60546875 - response_length_non_aborted/max:780.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.6875 - prompt_length/max:950.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012242421507835388 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5708349235355854) - timing_s/agent_loop/generate_sequences/max:np.float64(5.78690424002707) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9946807451196946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.78690424002707) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:635 - timing_s/agent_loop/slowest/response_length:780 - timing_s/gen:11.676643127575517 - timing_s/reward:0.060109492391347885 - timing_s/old_log_prob:2.263966953381896 - timing_s/adv:0.5860696248710155 - timing_s/update_actor:15.427983490750194 - timing_s/step:30.10305192694068 - timing_s/testing:71.47364217042923 - timing_s/save_checkpoint:6.949206165969372 - timing_s/stop_profile:0.00013513118028640747 - timing_per_token_ms/adv:0.0038848834002016158 - timing_per_token_ms/gen:0.16549703249345216 - timing_per_token_ms/update_actor:0.10226757098184526 - perf/total_num_tokens:150859 - perf/time_per_step:30.10305192694068 - perf/throughput:626.4273484883312 (TaskRunner pid=2026458) Training Progress: 80%|████████ | 80/100 [57:16<18:12, 54.65s/it] (TaskRunner pid=2026458) step:81 - global_seqlen/min:15749 - global_seqlen/max:23070 - global_seqlen/minmax_diff:7321 - global_seqlen/balanced_min:19345 - global_seqlen/balanced_max:19350 - global_seqlen/mean:19349.125 - actor/entropy:0.24268725514411926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.0049947695806622505 - training/rollout_probs_diff_std:0.013053881004452705 - training/rollout_actor_probs_pearson_corr:0.9983764290809631 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997670650482178 - rollout_corr/rollout_is_ratio_fraction_high:1.3156336535757873e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.262534614303149e-05 - rollout_corr/rollout_is_max:2.0260822772979736 - rollout_corr/rollout_is_min:0.2609400451183319 - rollout_corr/rollout_is_std:0.03953344747424126 - rollout_corr/rollout_is_eff_sample_size:0.9984388918274393 - rollout_corr/rollout_is_seq_mean:0.999757707118988 - rollout_corr/rollout_is_seq_std:0.002598333638161421 - rollout_corr/rollout_is_seq_max:1.0065761804580688 - rollout_corr/rollout_is_seq_min:0.9917704463005066 - rollout_corr/rollout_is_seq_max_deviation:0.008229553699493408 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2969164904206991) - rollout_corr/training_log_ppl:np.float64(0.2555096454452723) - rollout_corr/kl:np.float64(0.0012044042446035519) - rollout_corr/k3_kl:np.float64(0.001068380127094315) - rollout_corr/rollout_ppl:np.float64(1.295284100342542) - rollout_corr/rollout_log_ppl:np.float64(0.25430524366674945) - rollout_corr/log_ppl_diff:np.float64(0.0012044017785228789) - rollout_corr/log_ppl_abs_diff:np.float64(0.002569538773968816) - rollout_corr/log_ppl_diff_max:np.float64(0.00988227128982544) - rollout_corr/log_ppl_diff_min:np.float64(-0.007966101169586182) - rollout_corr/ppl_ratio:np.float64(1.0012098925653845) - rollout_corr/chi2_token:np.float64(0.0018374514766037464) - rollout_corr/chi2_seq:np.float64(0.6602487270720303) - actor/pg_loss:np.float64(0.0036679430631920695) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007375691257038852) - actor/ppo_kl:np.float64(0.00012186931077984298) - actor/pg_clipfrac_lower:np.float64(8.037551197048742e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(152346.46296780813) - self_distillation/token_reweight_w_std:np.float64(2223816.070475471) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006990339607) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07411914720432833) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.43313899636268616) - perf/mfu/actor:np.float64(0.035338759221170066) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.7124252319336) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0049204700626432896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0049204700626432896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:296.91015625 - response_length/max:761.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.91015625 - response_length_non_aborted/max:761.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.6648626923561096e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7210864126682281) - timing_s/agent_loop/generate_sequences/max:np.float64(5.891790520399809) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.159814590260794) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.891790520399809) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:761 - timing_s/gen:11.6525589954108 - timing_s/reward:0.05602879822254181 - timing_s/old_log_prob:2.4974898006767035 - timing_s/adv:0.5450900681316853 - timing_s/update_actor:15.660592023283243 - timing_s/step:30.51396298967302 - timing_s/stop_profile:0.00014164485037326813 - timing_per_token_ms/adv:0.0035214129071190897 - timing_per_token_ms/gen:0.15330499013815205 - timing_per_token_ms/update_actor:0.10117119006210386 - perf/total_num_tokens:154793 - perf/time_per_step:30.51396298967302 - perf/throughput:634.1072448225888 (TaskRunner pid=2026458) Training Progress: 81%|████████ | 81/100 [57:46<15:01, 47.43s/it] (TaskRunner pid=2026458) step:82 - global_seqlen/min:15004 - global_seqlen/max:21121 - global_seqlen/minmax_diff:6117 - global_seqlen/balanced_min:17893 - global_seqlen/balanced_max:17896 - global_seqlen/mean:17895.25 - actor/entropy:0.23795437812805176 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31782522797584534 - training/rollout_probs_diff_mean:0.004893430043011904 - training/rollout_probs_diff_std:0.012663507834076881 - training/rollout_actor_probs_pearson_corr:0.9984432458877563 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.000130534172058 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6648900529835373e-05 - rollout_corr/rollout_is_max:1.7479747533798218 - rollout_corr/rollout_is_min:0.3629961311817169 - rollout_corr/rollout_is_std:0.038970839232206345 - rollout_corr/rollout_is_eff_sample_size:0.998484170923625 - rollout_corr/rollout_is_seq_mean:1.0002964735031128 - rollout_corr/rollout_is_seq_std:0.002696148119866848 - rollout_corr/rollout_is_seq_max:1.0123590230941772 - rollout_corr/rollout_is_seq_min:0.9923802018165588 - rollout_corr/rollout_is_seq_max_deviation:0.012359023094177246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.296073100529611) - rollout_corr/training_log_ppl:np.float64(0.2552825820748694) - rollout_corr/kl:np.float64(0.0009574908228102696) - rollout_corr/k3_kl:np.float64(0.0011017269548574404) - rollout_corr/rollout_ppl:np.float64(1.2947798878885806) - rollout_corr/rollout_log_ppl:np.float64(0.254325088521) - rollout_corr/log_ppl_diff:np.float64(0.0009574935538694263) - rollout_corr/log_ppl_abs_diff:np.float64(0.002439796517137438) - rollout_corr/log_ppl_diff_max:np.float64(0.014934465289115906) - rollout_corr/log_ppl_diff_min:np.float64(-0.010826826095581055) - rollout_corr/ppl_ratio:np.float64(1.0009630019776523) - rollout_corr/chi2_token:np.float64(0.002452905522659421) - rollout_corr/chi2_seq:np.float64(0.423899469897151) - actor/pg_loss:np.float64(0.0041424884693697095) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013430351091301418) - actor/ppo_kl:np.float64(0.000410171516927349) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(233069.21662305342) - self_distillation/token_reweight_w_std:np.float64(2612623.9466522) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009098192676902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0835862404492218) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.45514093339443207) - perf/mfu/actor:np.float64(0.03317730103506178) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.53085327148438) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003221185877919197 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003221185877919197 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.1640625 - response_length/max:790.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.1640625 - response_length_non_aborted/max:790.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010550394654273987 - timing_s/agent_loop/generate_sequences/min:np.float64(1.162438377737999) - timing_s/agent_loop/generate_sequences/max:np.float64(6.011627459898591) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.059660606115358) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.011627459898591) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:298 - timing_s/agent_loop/slowest/response_length:790 - timing_s/gen:12.346360467374325 - timing_s/reward:0.05675779469311237 - timing_s/old_log_prob:2.2316900305449963 - timing_s/adv:0.5058559346944094 - timing_s/update_actor:15.28704277612269 - timing_s/step:30.49560439772904 - timing_s/stop_profile:0.00011858716607093811 - timing_per_token_ms/adv:0.0035334511580895024 - timing_per_token_ms/gen:0.16450846725348867 - timing_per_token_ms/update_actor:0.10678142786579323 - perf/total_num_tokens:143162 - perf/time_per_step:30.49560439772904 - perf/throughput:586.8140787310525 (TaskRunner pid=2026458) Training Progress: 82%|████████▏ | 82/100 [58:17<12:42, 42.36s/it] (TaskRunner pid=2026458) step:83 - global_seqlen/min:15510 - global_seqlen/max:20545 - global_seqlen/minmax_diff:5035 - global_seqlen/balanced_min:18663 - global_seqlen/balanced_max:18665 - global_seqlen/mean:18664.0 - actor/entropy:0.253889262676239 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553972005844116 - training/rollout_probs_diff_mean:0.005415121093392372 - training/rollout_probs_diff_std:0.013402428478002548 - training/rollout_actor_probs_pearson_corr:0.9983406662940979 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:1.0002059936523438 - rollout_corr/rollout_is_ratio_fraction_high:2.7475547540234402e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3737773770117201e-05 - rollout_corr/rollout_is_max:2.8205859661102295 - rollout_corr/rollout_is_min:0.3914983570575714 - rollout_corr/rollout_is_std:0.04187990352511406 - rollout_corr/rollout_is_eff_sample_size:0.9982497385555893 - rollout_corr/rollout_is_seq_mean:1.000227689743042 - rollout_corr/rollout_is_seq_std:0.002809274010360241 - rollout_corr/rollout_is_seq_max:1.009268045425415 - rollout_corr/rollout_is_seq_min:0.9934084415435791 - rollout_corr/rollout_is_seq_max_deviation:0.009268045425415039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3143565426580608) - rollout_corr/training_log_ppl:np.float64(0.2695897765806876) - rollout_corr/kl:np.float64(0.000838180147326284) - rollout_corr/k3_kl:np.float64(0.001138548173230447) - rollout_corr/rollout_ppl:np.float64(1.313231692649424) - rollout_corr/rollout_log_ppl:np.float64(0.26875159548944794) - rollout_corr/log_ppl_diff:np.float64(0.0008381810912396759) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024882252619136125) - rollout_corr/log_ppl_diff_max:np.float64(0.013383328914642334) - rollout_corr/log_ppl_diff_min:np.float64(-0.00783604383468628) - rollout_corr/ppl_ratio:np.float64(1.0008434464689344) - rollout_corr/chi2_token:np.float64(0.002809858415275812) - rollout_corr/chi2_seq:np.float64(0.9911151258274913) - actor/pg_loss:np.float64(0.004593900288455188) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000947352435105131) - actor/ppo_kl:np.float64(0.00014004049228333315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(142169.04989654478) - self_distillation/token_reweight_w_std:np.float64(1783576.419247636) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992821563500911) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07421927264658734) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.4911562651395798) - perf/mfu/actor:np.float64(0.03439597980816678) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.20166397094727) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015317617217078805 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015317617217078805 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.34375 - response_length/max:727.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.34375 - response_length_non_aborted/max:727.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.794160723686218e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2851199880242348) - timing_s/agent_loop/generate_sequences/max:np.float64(5.475394682958722) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0284413718109136) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.475394682958722) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:727 - timing_s/gen:11.69941053725779 - timing_s/reward:0.055808091536164284 - timing_s/old_log_prob:2.32892463542521 - timing_s/adv:0.5197819676250219 - timing_s/update_actor:15.535809490829706 - timing_s/step:30.227494234219193 - timing_s/stop_profile:0.00012695416808128357 - timing_per_token_ms/adv:0.0034811801303647524 - timing_per_token_ms/gen:0.16072385066020703 - timing_per_token_ms/update_actor:0.10404930274076904 - perf/total_num_tokens:149312 - perf/time_per_step:30.227494234219193 - perf/throughput:617.4511143855027 (TaskRunner pid=2026458) Training Progress: 83%|████████▎ | 83/100 [58:47<10:58, 38.73s/it] (TaskRunner pid=2026458) step:84 - global_seqlen/min:14717 - global_seqlen/max:21368 - global_seqlen/minmax_diff:6651 - global_seqlen/balanced_min:17413 - global_seqlen/balanced_max:17422 - global_seqlen/mean:17419.375 - actor/entropy:0.22988727688789368 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35856932401657104 - training/rollout_probs_diff_mean:0.005215250421315432 - training/rollout_probs_diff_std:0.013718393631279469 - training/rollout_actor_probs_pearson_corr:0.9981431365013123 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9996470212936401 - rollout_corr/rollout_is_ratio_fraction_high:1.3454785403155256e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.072871423792094e-05 - rollout_corr/rollout_is_max:2.1147913932800293 - rollout_corr/rollout_is_min:0.390633225440979 - rollout_corr/rollout_is_std:0.041462238878011703 - rollout_corr/rollout_is_eff_sample_size:0.9982825263238067 - rollout_corr/rollout_is_seq_mean:0.9997439980506897 - rollout_corr/rollout_is_seq_std:0.0025515363086014986 - rollout_corr/rollout_is_seq_max:1.0085729360580444 - rollout_corr/rollout_is_seq_min:0.9917205572128296 - rollout_corr/rollout_is_seq_max_deviation:0.008572936058044434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2788517312146723) - rollout_corr/training_log_ppl:np.float64(0.24251381697831675) - rollout_corr/kl:np.float64(0.0013615037188472456) - rollout_corr/k3_kl:np.float64(0.0013387326044949077) - rollout_corr/rollout_ppl:np.float64(1.2770492630079389) - rollout_corr/rollout_log_ppl:np.float64(0.2411523106857203) - rollout_corr/log_ppl_diff:np.float64(0.0013615062925964594) - rollout_corr/log_ppl_abs_diff:np.float64(0.002768128179013729) - rollout_corr/log_ppl_diff_max:np.float64(0.011708617210388184) - rollout_corr/log_ppl_diff_min:np.float64(-0.010206863284111023) - rollout_corr/ppl_ratio:np.float64(1.0013677747920156) - rollout_corr/chi2_token:np.float64(0.0024920080322772264) - rollout_corr/chi2_seq:np.float64(0.9803334202151746) - actor/pg_loss:np.float64(0.0052698974031955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015525902631452482) - actor/ppo_kl:np.float64(0.00019933821041862032) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(172811.0304318224) - self_distillation/token_reweight_w_std:np.float64(2611723.788451826) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009630725253373) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09219890262465924) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.5317341759800911) - perf/mfu/actor:np.float64(0.03233182941898891) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27649307250977) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011396203190088272 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011396203190088272 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:290.32421875 - response_length/max:900.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:290.32421875 - response_length_non_aborted/max:900.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003882218152284622 - timing_s/agent_loop/generate_sequences/min:np.float64(1.449787463992834) - timing_s/agent_loop/generate_sequences/max:np.float64(6.467833820730448) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1213262150922674) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.467833820730448) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:245 - timing_s/agent_loop/slowest/response_length:900 - timing_s/gen:12.418748388066888 - timing_s/reward:0.06932966038584709 - timing_s/old_log_prob:2.232461268082261 - timing_s/adv:0.5236291699111462 - timing_s/update_actor:15.485476253554225 - timing_s/step:30.817566089332104 - timing_s/stop_profile:0.000115908682346344 - timing_per_token_ms/adv:0.0037575197869552305 - timing_per_token_ms/gen:0.16709159194417458 - timing_per_token_ms/update_actor:0.1111225019091832 - perf/total_num_tokens:139355 - perf/time_per_step:30.817566089332104 - perf/throughput:565.2417504194123 (TaskRunner pid=2026458) Training Progress: 84%|████████▍ | 84/100 [59:18<09:42, 36.38s/it] (TaskRunner pid=2026458) step:85 - global_seqlen/min:16721 - global_seqlen/max:24217 - global_seqlen/minmax_diff:7496 - global_seqlen/balanced_min:19463 - global_seqlen/balanced_max:19465 - global_seqlen/mean:19464.25 - actor/entropy:0.23170782625675201 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5486323833465576 - training/rollout_probs_diff_mean:0.005258036311715841 - training/rollout_probs_diff_std:0.013953270390629768 - training/rollout_actor_probs_pearson_corr:0.998067319393158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0003063678741455 - rollout_corr/rollout_is_ratio_fraction_high:2.5328639821964316e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.065727964392863e-05 - rollout_corr/rollout_is_max:2.629549741744995 - rollout_corr/rollout_is_min:0.32894784212112427 - rollout_corr/rollout_is_std:0.04162293300032616 - rollout_corr/rollout_is_eff_sample_size:0.9982714780308726 - rollout_corr/rollout_is_seq_mean:1.0004109144210815 - rollout_corr/rollout_is_seq_std:0.002749430714175105 - rollout_corr/rollout_is_seq_max:1.0109564065933228 - rollout_corr/rollout_is_seq_min:0.9916076064109802 - rollout_corr/rollout_is_seq_max_deviation:0.010956406593322754 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2771666692569852) - rollout_corr/training_log_ppl:np.float64(0.24135592477978207) - rollout_corr/kl:np.float64(0.0007578132510701607) - rollout_corr/k3_kl:np.float64(0.0010257670675173358) - rollout_corr/rollout_ppl:np.float64(1.2761707119643688) - rollout_corr/rollout_log_ppl:np.float64(0.24059811038023327) - rollout_corr/log_ppl_diff:np.float64(0.0007578143995488063) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024299477954627946) - rollout_corr/log_ppl_diff_max:np.float64(0.011833712458610535) - rollout_corr/log_ppl_diff_min:np.float64(-0.010344773530960083) - rollout_corr/ppl_ratio:np.float64(1.0007629161700606) - rollout_corr/chi2_token:np.float64(0.0026269492227584124) - rollout_corr/chi2_seq:np.float64(1.3403879939578474) - actor/pg_loss:np.float64(0.002229055855423212) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019035061723116087) - actor/ppo_kl:np.float64(0.00024953764273760726) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(17317.446490210947) - self_distillation/token_reweight_w_std:np.float64(276465.0479241874) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991332795470953) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03143878703122027) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2347664088010788) - perf/mfu/actor:np.float64(0.03632172645099713) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27149963378906) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004859932523686439 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004859932523686439 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:308.4453125 - response_length/max:777.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.4453125 - response_length_non_aborted/max:777.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014561228454113007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4749425910413265) - timing_s/agent_loop/generate_sequences/max:np.float64(6.026074631139636) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2942245794984046) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.026074631139636) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:235 - timing_s/agent_loop/slowest/response_length:777 - timing_s/gen:12.095296228304505 - timing_s/reward:0.05645551718771458 - timing_s/old_log_prob:2.26230669952929 - timing_s/adv:0.4968310799449682 - timing_s/update_actor:15.393540350720286 - timing_s/step:30.395571315661073 - timing_s/stop_profile:0.00012073665857315063 - timing_per_token_ms/adv:0.0031906641660028527 - timing_per_token_ms/gen:0.15317869644011683 - timing_per_token_ms/update_actor:0.09885777997302932 - perf/total_num_tokens:155714 - perf/time_per_step:30.395571315661073 - perf/throughput:640.3646701640117 (TaskRunner pid=2026458) Training Progress: 85%|████████▌ | 85/100 [59:48<08:38, 34.60s/it] (TaskRunner pid=2026458) step:86 - global_seqlen/min:16265 - global_seqlen/max:20200 - global_seqlen/minmax_diff:3935 - global_seqlen/balanced_min:18647 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18649.75 - actor/entropy:0.23041389882564545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.898743212223053 - training/rollout_probs_diff_mean:0.005259583238512278 - training/rollout_probs_diff_std:0.014470859430730343 - training/rollout_actor_probs_pearson_corr:0.9979584813117981 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9998511075973511 - rollout_corr/rollout_is_ratio_fraction_high:3.836807809420861e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.394679803634062e-05 - rollout_corr/rollout_is_max:2.1972851753234863 - rollout_corr/rollout_is_min:0.0777815580368042 - rollout_corr/rollout_is_std:0.04242711886763573 - rollout_corr/rollout_is_eff_sample_size:0.9982026989977762 - rollout_corr/rollout_is_seq_mean:0.9998696446418762 - rollout_corr/rollout_is_seq_std:0.0025317175313830376 - rollout_corr/rollout_is_seq_max:1.009598731994629 - rollout_corr/rollout_is_seq_min:0.9924889802932739 - rollout_corr/rollout_is_seq_max_deviation:0.009598731994628906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2799476818181574) - rollout_corr/training_log_ppl:np.float64(0.24274212954333052) - rollout_corr/kl:np.float64(0.0012324780372026112) - rollout_corr/k3_kl:np.float64(0.0011905511769896293) - rollout_corr/rollout_ppl:np.float64(1.2783302562311292) - rollout_corr/rollout_log_ppl:np.float64(0.24150965179433115) - rollout_corr/log_ppl_diff:np.float64(0.0012324777489993721) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024851207563187927) - rollout_corr/log_ppl_diff_max:np.float64(0.009878866374492645) - rollout_corr/log_ppl_diff_min:np.float64(-0.00684240460395813) - rollout_corr/ppl_ratio:np.float64(1.0012374119833112) - rollout_corr/chi2_token:np.float64(0.0022098186891525984) - rollout_corr/chi2_seq:np.float64(0.6068784659728408) - actor/pg_loss:np.float64(0.0033142786705866456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009858607454589219) - actor/ppo_kl:np.float64(0.0001475137922213321) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(165894.66037644562) - self_distillation/token_reweight_w_std:np.float64(2346342.531521928) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011869312729686) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07381428830558434) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.42134569585323334) - perf/mfu/actor:np.float64(0.0348608857845462) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1626739501953) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0060413735918700695 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0060413735918700695 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.4296875 - response_length/max:825.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.4296875 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016151554882526398 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6213552672415972) - timing_s/agent_loop/generate_sequences/max:np.float64(6.171470927074552) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.300390396914736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.171470927074552) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:12.024075420573354 - timing_s/reward:0.05586143955588341 - timing_s/old_log_prob:2.3416189160197973 - timing_s/adv:0.5560324247926474 - timing_s/update_actor:15.314254153519869 - timing_s/step:30.382415380328894 - timing_s/stop_profile:0.0001287907361984253 - timing_per_token_ms/adv:0.0037268088365302976 - timing_per_token_ms/gen:0.15378022024009916 - timing_per_token_ms/update_actor:0.10264383003471808 - perf/total_num_tokens:149198 - perf/time_per_step:30.382415380328894 - perf/throughput:613.8336852597567 (TaskRunner pid=2026458) Training Progress: 86%|████████▌ | 86/100 [1:00:19<07:46, 33.35s/it] (TaskRunner pid=2026458) (TaskRunner pid=2026458) step:87 - global_seqlen/min:16648 - global_seqlen/max:20097 - global_seqlen/minmax_diff:3449 - global_seqlen/balanced_min:18341 - global_seqlen/balanced_max:18344 - global_seqlen/mean:18342.375 - actor/entropy:0.24005606770515442 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44461295008659363 - training/rollout_probs_diff_mean:0.005454737227410078 - training/rollout_probs_diff_std:0.014133700169622898 - training/rollout_actor_probs_pearson_corr:0.998093843460083 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.53125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.9998968243598938 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.625958146993071e-05 - rollout_corr/rollout_is_max:1.91322922706604 - rollout_corr/rollout_is_min:0.3312951922416687 - rollout_corr/rollout_is_std:0.0426471121609211 - rollout_corr/rollout_is_eff_sample_size:0.998184050703313 - rollout_corr/rollout_is_seq_mean:0.999923050403595 - rollout_corr/rollout_is_seq_std:0.0029033836908638477 - rollout_corr/rollout_is_seq_max:1.0088640451431274 - rollout_corr/rollout_is_seq_min:0.990547776222229 - rollout_corr/rollout_is_seq_max_deviation:0.009452223777770996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2908267816528678) - rollout_corr/training_log_ppl:np.float64(0.25219142163405195) - rollout_corr/kl:np.float64(0.0013515868227500505) - rollout_corr/k3_kl:np.float64(0.001383622248795291) - rollout_corr/rollout_ppl:np.float64(1.2890665112063289) - rollout_corr/rollout_log_ppl:np.float64(0.25083983279182576) - rollout_corr/log_ppl_diff:np.float64(0.0013515888422261924) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028635623457375914) - rollout_corr/log_ppl_diff_max:np.float64(0.012585937976837158) - rollout_corr/log_ppl_diff_min:np.float64(-0.009711101651191711) - rollout_corr/ppl_ratio:np.float64(1.0013586606364697) - rollout_corr/chi2_token:np.float64(0.0029732428956776857) - rollout_corr/chi2_seq:np.float64(0.6213931296952069) - actor/pg_loss:np.float64(0.005519538884982467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011970178129558917) - actor/ppo_kl:np.float64(0.00032003364054844496) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53125) - self_distillation/token_reweight_w_mean:np.float64(104019.92419685167) - self_distillation/token_reweight_w_std:np.float64(1491533.3107083726) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000025857705623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07910635319421999) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.44405730813741684) - perf/mfu/actor:np.float64(0.03427082030528879) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2020378112793) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009922783821821213 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009922783821821213 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:277.73046875 - response_length/max:642.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:277.73046875 - response_length_non_aborted/max:642.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011511892080307007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3540627993643284) - timing_s/agent_loop/generate_sequences/max:np.float64(5.077815180644393) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9762204922371893) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.077815180644393) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:172 - timing_s/agent_loop/slowest/response_length:642 - timing_s/gen:11.154959298670292 - timing_s/reward:0.057516856119036674 - timing_s/old_log_prob:2.265875956043601 - timing_s/adv:0.5384714491665363 - timing_s/update_actor:15.248491061851382 - timing_s/step:29.354633087292314 - timing_s/stop_profile:0.0001263357698917389 - timing_per_token_ms/adv:0.003669586470989555 - timing_per_token_ms/gen:0.15689333603384426 - timing_per_token_ms/update_actor:0.10391573516141846 - perf/total_num_tokens:146739 - perf/time_per_step:29.354633087292314 - perf/throughput:624.8545142926844 (TaskRunner pid=2026458) Training Progress: 87%|████████▋ | 87/100 [1:00:48<06:58, 32.18s/it] (TaskRunner pid=2026458) step:88 - global_seqlen/min:14981 - global_seqlen/max:20248 - global_seqlen/minmax_diff:5267 - global_seqlen/balanced_min:17171 - global_seqlen/balanced_max:17176 - global_seqlen/mean:17174.125 - actor/entropy:0.21586236357688904 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5279986262321472 - training/rollout_probs_diff_mean:0.005149980541318655 - training/rollout_probs_diff_std:0.014039847999811172 - training/rollout_actor_probs_pearson_corr:0.9979734420776367 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9998298287391663 - rollout_corr/rollout_is_ratio_fraction_high:2.9164297302486375e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4582148651243187e-05 - rollout_corr/rollout_is_max:2.4087512493133545 - rollout_corr/rollout_is_min:0.3157358169555664 - rollout_corr/rollout_is_std:0.04135211929678917 - rollout_corr/rollout_is_eff_sample_size:0.998292149229813 - rollout_corr/rollout_is_seq_mean:0.9997440576553345 - rollout_corr/rollout_is_seq_std:0.0026299559976905584 - rollout_corr/rollout_is_seq_max:1.0093685388565063 - rollout_corr/rollout_is_seq_min:0.9928037524223328 - rollout_corr/rollout_is_seq_max_deviation:0.009368538856506348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.263124991208315) - rollout_corr/training_log_ppl:np.float64(0.23035154843819328) - rollout_corr/kl:np.float64(0.0012507079140435806) - rollout_corr/k3_kl:np.float64(0.0012822421521718752) - rollout_corr/rollout_ppl:np.float64(1.2614940297789872) - rollout_corr/rollout_log_ppl:np.float64(0.22910083959868643) - rollout_corr/log_ppl_diff:np.float64(0.0012507088395068422) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027966901106992736) - rollout_corr/log_ppl_diff_max:np.float64(0.014727145433425903) - rollout_corr/log_ppl_diff_min:np.float64(-0.014272093772888184) - rollout_corr/ppl_ratio:np.float64(1.0012579679023474) - rollout_corr/chi2_token:np.float64(0.002700346987694502) - rollout_corr/chi2_seq:np.float64(0.8957935450598598) - actor/pg_loss:np.float64(0.0045979529386386275) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008685116490596556) - actor/ppo_kl:np.float64(0.00010619230280539682) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(126031.5825594815) - self_distillation/token_reweight_w_std:np.float64(1914441.9060592402) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010992304887623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07210871722782031) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4531179219484329) - perf/mfu/actor:np.float64(0.032191482757026275) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.15955352783203) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014600376598536968 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014600376598536968 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:267.87890625 - response_length/max:807.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:267.87890625 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012998655438423157 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3904380090534687) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8301611468195915) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8218457526090788) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8301611468195915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:807 - timing_s/gen:11.686277892440557 - timing_s/reward:0.054150139912962914 - timing_s/old_log_prob:2.385304968804121 - timing_s/adv:0.4924223218113184 - timing_s/update_actor:15.133386766538024 - timing_s/step:29.843838276341558 - timing_s/stop_profile:5.164928734302521e-05 - timing_per_token_ms/adv:0.003584042286079483 - timing_per_token_ms/gen:0.170411040034422 - timing_per_token_ms/update_actor:0.11014670883187662 - perf/total_num_tokens:137393 - perf/time_per_step:29.843838276341558 - perf/throughput:575.466360626094 (TaskRunner pid=2026458) Training Progress: 88%|████████▊ | 88/100 [1:01:18<06:17, 31.49s/it] (TaskRunner pid=2026458) step:89 - global_seqlen/min:16152 - global_seqlen/max:22412 - global_seqlen/minmax_diff:6260 - global_seqlen/balanced_min:18887 - global_seqlen/balanced_max:18895 - global_seqlen/mean:18893.375 - actor/entropy:0.22386007010936737 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2151239514350891 - training/rollout_probs_diff_mean:0.005173899233341217 - training/rollout_probs_diff_std:0.013607512228190899 - training/rollout_actor_probs_pearson_corr:0.9981110692024231 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000429153442383 - rollout_corr/rollout_is_ratio_fraction_high:2.6122277631657198e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.83668365329504e-05 - rollout_corr/rollout_is_max:2.51365327835083 - rollout_corr/rollout_is_min:0.1054598018527031 - rollout_corr/rollout_is_std:0.04179442301392555 - rollout_corr/rollout_is_eff_sample_size:0.9982565097660324 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0027215098962187767 - rollout_corr/rollout_is_seq_max:1.0083972215652466 - rollout_corr/rollout_is_seq_min:0.9915516376495361 - rollout_corr/rollout_is_seq_max_deviation:0.008448362350463867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2823212435469031) - rollout_corr/training_log_ppl:np.float64(0.24520724279864226) - rollout_corr/kl:np.float64(0.0008697826999508607) - rollout_corr/k3_kl:np.float64(0.0010809666277964425) - rollout_corr/rollout_ppl:np.float64(1.2811899655498564) - rollout_corr/rollout_log_ppl:np.float64(0.24433745855640154) - rollout_corr/log_ppl_diff:np.float64(0.000869784242240712) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022608327853959054) - rollout_corr/log_ppl_diff_max:np.float64(0.010239839553833008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071149468421936035) - rollout_corr/ppl_ratio:np.float64(1.0008741298224777) - rollout_corr/chi2_token:np.float64(0.0025997732300311327) - rollout_corr/chi2_seq:np.float64(0.5740627928171307) - actor/pg_loss:np.float64(0.0036603367188945413) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008643267137813382) - actor/ppo_kl:np.float64(-9.36468737013918e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(95742.81214469112) - self_distillation/token_reweight_w_std:np.float64(1397763.4353546314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014505761209875) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07612263548071496) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.43359871953725815) - perf/mfu/actor:np.float64(0.035258064824575705) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1541633605957) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00761464424431324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00761464424431324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.07421875 - response_length/max:1025.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.07421875 - response_length_non_aborted/max:1025.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2774325013160706e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3900669310241938) - timing_s/agent_loop/generate_sequences/max:np.float64(7.259778779000044) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.168158708969713) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.259778779000044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:1025 - timing_s/gen:12.976460268720984 - timing_s/reward:0.055463384836912155 - timing_s/old_log_prob:2.4733975417912006 - timing_s/adv:0.5145350992679596 - timing_s/update_actor:15.292766243219376 - timing_s/step:31.39907766878605 - timing_s/stop_profile:4.6625733375549316e-05 - timing_per_token_ms/adv:0.003404203188074918 - timing_per_token_ms/gen:0.16948735379649418 - timing_per_token_ms/update_actor:0.10117809975202535 - perf/total_num_tokens:151147 - perf/time_per_step:31.39907766878605 - perf/throughput:601.7175153772744 (TaskRunner pid=2026458) Training Progress: 89%|████████▉ | 89/100 [1:01:50<05:46, 31.47s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a structure with a chlorine atom, a trifluoromethyl group, and the correct aromatic ring structure. The formula is "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", which matches the required components of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B contains a chlorine atom but does not have the correct trifluoromethyl group or the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains all the necessary components of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2026458) step:90 - global_seqlen/min:14894 - global_seqlen/max:19933 - global_seqlen/minmax_diff:5039 - global_seqlen/balanced_min:18191 - global_seqlen/balanced_max:18194 - global_seqlen/mean:18192.75 - actor/entropy:0.21710193157196045 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3145221471786499 - training/rollout_probs_diff_mean:0.004972291644662619 - training/rollout_probs_diff_std:0.013361528515815735 - training/rollout_actor_probs_pearson_corr:0.9982044100761414 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0000026226043701 - rollout_corr/rollout_is_ratio_fraction_high:2.6879552024183795e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0319329855265096e-05 - rollout_corr/rollout_is_max:3.3088538646698 - rollout_corr/rollout_is_min:0.46167826652526855 - rollout_corr/rollout_is_std:0.041136063635349274 - rollout_corr/rollout_is_eff_sample_size:0.9983105639498434 - rollout_corr/rollout_is_seq_mean:1.0000332593917847 - rollout_corr/rollout_is_seq_std:0.0027625819202512503 - rollout_corr/rollout_is_seq_max:1.0086408853530884 - rollout_corr/rollout_is_seq_min:0.9904849529266357 - rollout_corr/rollout_is_seq_max_deviation:0.009515047073364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2710709455423057) - rollout_corr/training_log_ppl:np.float64(0.23564479569904506) - rollout_corr/kl:np.float64(0.0014168473381213076) - rollout_corr/k3_kl:np.float64(0.0014046167735841664) - rollout_corr/rollout_ppl:np.float64(1.2692075530067086) - rollout_corr/rollout_log_ppl:np.float64(0.2342279483564198) - rollout_corr/log_ppl_diff:np.float64(0.0014168473426252604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028690413455478847) - rollout_corr/log_ppl_diff_max:np.float64(0.03278002142906189) - rollout_corr/log_ppl_diff_min:np.float64(-0.008600443601608276) - rollout_corr/ppl_ratio:np.float64(1.0014270655810833) - rollout_corr/chi2_token:np.float64(0.002751560416072607) - rollout_corr/chi2_seq:np.float64(0.7767476551234722) - actor/pg_loss:np.float64(0.0019601467065513134) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009453206475882325) - actor/ppo_kl:np.float64(0.0005355499455532708) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(99385.55734024942) - self_distillation/token_reweight_w_std:np.float64(1449312.0253919773) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012841576244682) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045599023724207655) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.30510950461030006) - perf/mfu/actor:np.float64(0.03390671346645702) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.11214065551758) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/reward/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07254164422675807) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8122666666666666) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05053109084853488) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7015285714285714) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05338104590757411) - val-aux/sciknoweval/score/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/score/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07254164422675807) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8122666666666666) - val-aux/sciknoweval/score/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05053109084853488) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7015285714285714) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05338104590757411) - val-core/sciknoweval/acc/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/acc/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07254164422675807) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8122666666666666) - val-core/sciknoweval/acc/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05053109084853488) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7015285714285714) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05338104590757411) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0003677473252630062) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9995190476190476) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.001434898686445427) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997809523809524) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0009975480142912052) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9990666666666667) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0018903262505010432) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999142857142858) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0006331005658902953) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9981523809523809) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023204503433425093) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0003677473252630062) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9970095238095239) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0023016113846578) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:1 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002840832807123661 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002840832807123661 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:290.6484375 - response_length/max:721.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:290.6484375 - response_length_non_aborted/max:721.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:621.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.825903594493866e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7425885982811451) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7153176963329315) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0369765825307695) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7153176963329315) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:721 - timing_s/gen:11.199985072016716 - timing_s/reward:0.05536814592778683 - timing_s/old_log_prob:2.218005755916238 - timing_s/adv:0.4808800220489502 - timing_s/update_actor:15.297142412513494 - timing_s/step:29.337590511888266 - timing_s/testing:111.78063779324293 - timing_s/save_checkpoint:6.6935704834759235 - timing_s/stop_profile:0.00011844001710414886 - timing_per_token_ms/adv:0.003304063583357039 - timing_per_token_ms/gen:0.15052529462700207 - timing_per_token_ms/update_actor:0.10510465990925984 - perf/total_num_tokens:145542 - perf/time_per_step:29.337590511888266 - perf/throughput:620.1173880529786 (TaskRunner pid=2026458) Training Progress: 90%|█████████ | 90/100 [1:04:17<11:03, 66.39s/it] (TaskRunner pid=2026458) step:91 - global_seqlen/min:15865 - global_seqlen/max:25244 - global_seqlen/minmax_diff:9379 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18818 - global_seqlen/mean:18815.125 - actor/entropy:0.2314702272415161 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21055352687835693 - training/rollout_probs_diff_mean:0.005051769316196442 - training/rollout_probs_diff_std:0.013311025686562061 - training/rollout_actor_probs_pearson_corr:0.9982390999794006 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999205470085144 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.47056513116695e-05 - rollout_corr/rollout_is_max:1.7587093114852905 - rollout_corr/rollout_is_min:0.23516227304935455 - rollout_corr/rollout_is_std:0.039755210280418396 - rollout_corr/rollout_is_eff_sample_size:0.9984218983709217 - rollout_corr/rollout_is_seq_mean:0.9999246001243591 - rollout_corr/rollout_is_seq_std:0.002499330323189497 - rollout_corr/rollout_is_seq_max:1.0098474025726318 - rollout_corr/rollout_is_seq_min:0.9917200207710266 - rollout_corr/rollout_is_seq_max_deviation:0.009847402572631836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2789378436282277) - rollout_corr/training_log_ppl:np.float64(0.24201013351557776) - rollout_corr/kl:np.float64(0.0009981181076650358) - rollout_corr/k3_kl:np.float64(0.0010161043154539584) - rollout_corr/rollout_ppl:np.float64(1.2776579768396914) - rollout_corr/rollout_log_ppl:np.float64(0.2410120142158121) - rollout_corr/log_ppl_diff:np.float64(0.0009981192997656763) - rollout_corr/log_ppl_abs_diff:np.float64(0.002134583191946149) - rollout_corr/log_ppl_diff_max:np.float64(0.00839272141456604) - rollout_corr/log_ppl_diff_min:np.float64(-0.010266929864883423) - rollout_corr/ppl_ratio:np.float64(1.0010020262561738) - rollout_corr/chi2_token:np.float64(0.0021203176584094763) - rollout_corr/chi2_seq:np.float64(0.651163290720433) - actor/pg_loss:np.float64(0.002993874717503786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000568103197565506) - actor/ppo_kl:np.float64(6.333214638765128e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(82354.48971123807) - self_distillation/token_reweight_w_std:np.float64(1252927.9291606762) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009026953484863) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06239277429995127) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.33179403841495514) - perf/mfu/actor:np.float64(0.034982824171721626) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.52531814575195) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002352050505578518 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002352050505578518 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.84765625 - response_length/max:674.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.84765625 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.271526217460632e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2008809298276901) - timing_s/agent_loop/generate_sequences/max:np.float64(5.367214092984796) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2526293304326828) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.367214092984796) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:11.187385501340032 - timing_s/reward:0.05164775624871254 - timing_s/old_log_prob:2.4035099502652884 - timing_s/adv:0.5490099098533392 - timing_s/update_actor:15.261925529688597 - timing_s/step:29.56569225527346 - timing_s/stop_profile:0.00012163445353507996 - timing_per_token_ms/adv:0.0036473974385855742 - timing_per_token_ms/gen:0.14477741903821556 - timing_per_token_ms/update_actor:0.10139399505509927 - perf/total_num_tokens:150521 - perf/time_per_step:29.56569225527346 - perf/throughput:636.3837124985313 (TaskRunner pid=2026458) Training Progress: 91%|█████████ | 91/100 [1:04:47<08:18, 55.36s/it] (TaskRunner pid=2026458) step:92 - global_seqlen/min:16281 - global_seqlen/max:21475 - global_seqlen/minmax_diff:5194 - global_seqlen/balanced_min:18650 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18650.625 - actor/entropy:0.22619383037090302 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.005189774092286825 - training/rollout_probs_diff_std:0.013963738456368446 - training/rollout_actor_probs_pearson_corr:0.9980257153511047 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999138116836548 - rollout_corr/rollout_is_ratio_fraction_high:1.3226638657215517e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011903974518645555 - rollout_corr/rollout_is_max:2.0864930152893066 - rollout_corr/rollout_is_min:0.0921701192855835 - rollout_corr/rollout_is_std:0.04230894520878792 - rollout_corr/rollout_is_eff_sample_size:0.9982129142824776 - rollout_corr/rollout_is_seq_mean:0.9999446272850037 - rollout_corr/rollout_is_seq_std:0.0027264179661870003 - rollout_corr/rollout_is_seq_max:1.0081113576889038 - rollout_corr/rollout_is_seq_min:0.9917219281196594 - rollout_corr/rollout_is_seq_max_deviation:0.008278071880340576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2724861456081271) - rollout_corr/training_log_ppl:np.float64(0.2372018521418795) - rollout_corr/kl:np.float64(0.0009739822945320498) - rollout_corr/k3_kl:np.float64(0.0010123744499423992) - rollout_corr/rollout_ppl:np.float64(1.2711997576989233) - rollout_corr/rollout_log_ppl:np.float64(0.23622786854684819) - rollout_corr/log_ppl_diff:np.float64(0.0009739835950313136) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024230604904005304) - rollout_corr/log_ppl_diff_max:np.float64(0.012229472398757935) - rollout_corr/log_ppl_diff_min:np.float64(-0.010651081800460815) - rollout_corr/ppl_ratio:np.float64(1.0009791306219995) - rollout_corr/chi2_token:np.float64(0.0021006297320127487) - rollout_corr/chi2_seq:np.float64(0.7930810896214098) - actor/pg_loss:np.float64(0.002591841504909098) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004880208070972003) - actor/ppo_kl:np.float64(-2.9975729340492308e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(72610.33176031034) - self_distillation/token_reweight_w_std:np.float64(1092734.639293266) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988934537395835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06326231267303228) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.3366621397435665) - perf/mfu/actor:np.float64(0.03433711138530011) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.60525131225586) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006365319713950157 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006365319713950157 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:295.33203125 - response_length/max:674.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:295.33203125 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.356330752372742e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4242578689008951) - timing_s/agent_loop/generate_sequences/max:np.float64(5.455314517021179) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2662257757328916) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.455314517021179) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:674 - timing_s/gen:11.507964111864567 - timing_s/reward:0.056888729333877563 - timing_s/old_log_prob:2.3034613393247128 - timing_s/adv:0.4925750717520714 - timing_s/update_actor:15.52449726127088 - timing_s/step:29.975734889507294 - timing_s/stop_profile:0.000132659450173378 - timing_per_token_ms/adv:0.0033013308652663875 - timing_per_token_ms/gen:0.15221168060134338 - timing_per_token_ms/update_actor:0.10404810335626072 - perf/total_num_tokens:149205 - perf/time_per_step:29.975734889507294 - perf/throughput:622.1907509106129 (TaskRunner pid=2026458) Training Progress: 92%|█████████▏| 92/100 [1:05:17<06:22, 47.76s/it] (TaskRunner pid=2026458) step:93 - global_seqlen/min:15058 - global_seqlen/max:22608 - global_seqlen/minmax_diff:7550 - global_seqlen/balanced_min:18025 - global_seqlen/balanced_max:18028 - global_seqlen/mean:18026.625 - actor/entropy:0.20888566970825195 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24053943157196045 - training/rollout_probs_diff_mean:0.004918403923511505 - training/rollout_probs_diff_std:0.013119952753186226 - training/rollout_actor_probs_pearson_corr:0.9981909394264221 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.98828125 - self_distillation/correct_sample_fraction:0.87890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:0.9998151659965515 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263365528662689e-05 - rollout_corr/rollout_is_max:1.6370371580123901 - rollout_corr/rollout_is_min:0.2820625901222229 - rollout_corr/rollout_is_std:0.03976345434784889 - rollout_corr/rollout_is_eff_sample_size:0.9984207100389592 - rollout_corr/rollout_is_seq_mean:0.9998314380645752 - rollout_corr/rollout_is_seq_std:0.0023552256170660257 - rollout_corr/rollout_is_seq_max:1.0079485177993774 - rollout_corr/rollout_is_seq_min:0.9893273115158081 - rollout_corr/rollout_is_seq_max_deviation:0.010672688484191895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2420122399926186) - rollout_corr/training_log_ppl:np.float64(0.2130839267920237) - rollout_corr/kl:np.float64(0.0009293865148691793) - rollout_corr/k3_kl:np.float64(0.000889873810649533) - rollout_corr/rollout_ppl:np.float64(1.2408293401822448) - rollout_corr/rollout_log_ppl:np.float64(0.2121545401896583) - rollout_corr/log_ppl_diff:np.float64(0.0009293866023654118) - rollout_corr/log_ppl_abs_diff:np.float64(0.00208455762185622) - rollout_corr/log_ppl_diff_max:np.float64(0.008339852094650269) - rollout_corr/log_ppl_diff_min:np.float64(-0.005493566393852234) - rollout_corr/ppl_ratio:np.float64(1.0009328972082585) - rollout_corr/chi2_token:np.float64(0.00168294133618474) - rollout_corr/chi2_seq:np.float64(0.3678914944175631) - actor/pg_loss:np.float64(0.00139771425165236) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002747084317888948) - actor/ppo_kl:np.float64(-7.94472919167788e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.87890625) - self_distillation/token_reweight_w_mean:np.float64(72035.54537807149) - self_distillation/token_reweight_w_std:np.float64(1075218.813784783) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001510763540864) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03214944276260212) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.2579457201063633) - perf/mfu/actor:np.float64(0.03364477703539987) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.39699172973633) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.87890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.87890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036366567946970463 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0036366567946970463 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:296.86328125 - response_length/max:640.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.86328125 - response_length_non_aborted/max:640.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017446838319301605 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6789815910160542) - timing_s/agent_loop/generate_sequences/max:np.float64(5.267441829666495) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.250441880845756) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.267441829666495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:640 - timing_s/gen:11.287975236773491 - timing_s/reward:0.0554285254329443 - timing_s/old_log_prob:2.2384933289140463 - timing_s/adv:0.5122805014252663 - timing_s/update_actor:15.286979779601097 - timing_s/step:29.47672649472952 - timing_s/stop_profile:0.00014898553490638733 - timing_per_token_ms/adv:0.003552249113639313 - timing_per_token_ms/gen:0.14853185305700872 - timing_per_token_ms/update_actor:0.106002786015138 - perf/total_num_tokens:144213 - perf/time_per_step:29.47672649472952 - perf/throughput:611.5545090538865 (TaskRunner pid=2026458) Training Progress: 93%|█████████▎| 93/100 [1:05:47<04:56, 42.29s/it] (TaskRunner pid=2026458) step:94 - global_seqlen/min:15786 - global_seqlen/max:20010 - global_seqlen/minmax_diff:4224 - global_seqlen/balanced_min:17735 - global_seqlen/balanced_max:17737 - global_seqlen/mean:17736.0 - actor/entropy:0.22735926508903503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311311483383179 - training/rollout_probs_diff_mean:0.005040409974753857 - training/rollout_probs_diff_std:0.013486634008586407 - training/rollout_actor_probs_pearson_corr:0.9981728792190552 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.9998102784156799 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0001036699177348055 - rollout_corr/rollout_is_max:1.7523070573806763 - rollout_corr/rollout_is_min:0.05793754756450653 - rollout_corr/rollout_is_std:0.03992203250527382 - rollout_corr/rollout_is_eff_sample_size:0.9984081138940653 - rollout_corr/rollout_is_seq_mean:0.9999222159385681 - rollout_corr/rollout_is_seq_std:0.002507504541426897 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9931005835533142 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2725239442661405) - rollout_corr/training_log_ppl:np.float64(0.23770636528206524) - rollout_corr/kl:np.float64(0.001132996877289294) - rollout_corr/k3_kl:np.float64(0.0012387144401486694) - rollout_corr/rollout_ppl:np.float64(1.2710481900721788) - rollout_corr/rollout_log_ppl:np.float64(0.23657336641917937) - rollout_corr/log_ppl_diff:np.float64(0.00113299886288587) - rollout_corr/log_ppl_abs_diff:np.float64(0.002540186673286371) - rollout_corr/log_ppl_diff_max:np.float64(0.02026122808456421) - rollout_corr/log_ppl_diff_min:np.float64(-0.015430659055709839) - rollout_corr/ppl_ratio:np.float64(1.0011393770109862) - rollout_corr/chi2_token:np.float64(0.002516603097319603) - rollout_corr/chi2_seq:np.float64(0.9786048375535756) - actor/pg_loss:np.float64(0.002404728322289884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001062956328951259) - actor/ppo_kl:np.float64(0.0001939034154609942) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(96479.46470297547) - self_distillation/token_reweight_w_std:np.float64(1265708.629157153) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990413470659405) - self_distillation/token_reweight_w_clip_frac:np.float64(0.054301650350680575) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.35001390427351) - perf/mfu/actor:np.float64(0.032954674991088666) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4532012939453) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00508630508556962 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00508630508556962 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:301.4375 - response_length/max:631.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.4375 - response_length_non_aborted/max:631.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017067231237888336 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4371750131249428) - timing_s/agent_loop/generate_sequences/max:np.float64(5.477192688733339) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2628613221095293) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.477192688733339) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:631 - timing_s/gen:11.104732798412442 - timing_s/reward:0.05309521034359932 - timing_s/old_log_prob:2.3046134896576405 - timing_s/adv:0.4992757458239794 - timing_s/update_actor:15.339771771803498 - timing_s/step:29.391625313088298 - timing_s/stop_profile:0.000131336972117424 - timing_per_token_ms/adv:0.0035188017719890293 - timing_per_token_ms/gen:0.14390333815068995 - timing_per_token_ms/update_actor:0.10811183307822718 - perf/total_num_tokens:141888 - perf/time_per_step:29.391625313088298 - perf/throughput:603.4371971971906 (TaskRunner pid=2026458) Training Progress: 94%|█████████▍| 94/100 [1:06:16<03:50, 38.44s/it] (TaskRunner pid=2026458) step:95 - global_seqlen/min:14931 - global_seqlen/max:21336 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:17803 - global_seqlen/balanced_max:17812 - global_seqlen/mean:17810.75 - actor/entropy:0.22719933092594147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3546398878097534 - training/rollout_probs_diff_mean:0.004889350384473801 - training/rollout_probs_diff_std:0.012968857772648335 - training/rollout_actor_probs_pearson_corr:0.9983217716217041 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999182224273682 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3313806448422838e-05 - rollout_corr/rollout_is_max:1.646693468093872 - rollout_corr/rollout_is_min:0.012675890699028969 - rollout_corr/rollout_is_std:0.03914937749505043 - rollout_corr/rollout_is_eff_sample_size:0.9984694339690914 - rollout_corr/rollout_is_seq_mean:0.9998860955238342 - rollout_corr/rollout_is_seq_std:0.002519140485674143 - rollout_corr/rollout_is_seq_max:1.0078048706054688 - rollout_corr/rollout_is_seq_min:0.993129551410675 - rollout_corr/rollout_is_seq_max_deviation:0.00780487060546875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.269720992539078) - rollout_corr/training_log_ppl:np.float64(0.23537427955307066) - rollout_corr/kl:np.float64(0.000860856562256096) - rollout_corr/k3_kl:np.float64(0.0010757347620256041) - rollout_corr/rollout_ppl:np.float64(1.2686140895821154) - rollout_corr/rollout_log_ppl:np.float64(0.2345134206698276) - rollout_corr/log_ppl_diff:np.float64(0.0008608588832430542) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024572820402681828) - rollout_corr/log_ppl_diff_max:np.float64(0.022655338048934937) - rollout_corr/log_ppl_diff_min:np.float64(-0.008849263191223145) - rollout_corr/ppl_ratio:np.float64(1.0008665048517287) - rollout_corr/chi2_token:np.float64(0.002582071116194129) - rollout_corr/chi2_seq:np.float64(1.1929384064860642) - actor/pg_loss:np.float64(0.0030269279377534986) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001186676310680923) - actor/ppo_kl:np.float64(-0.00011882417348374474) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(142327.77909193072) - self_distillation/token_reweight_w_std:np.float64(1811469.8086368854) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001867031678557) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07137267180951312) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4086311012506485) - perf/mfu/actor:np.float64(0.03299815181672398) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.32865142822266) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007821861072443426 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007821861072443426 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.3984375 - response_length/max:742.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.3984375 - response_length_non_aborted/max:742.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011094845831394196 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2119843941181898) - timing_s/agent_loop/generate_sequences/max:np.float64(5.851273892447352) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2298050526515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.851273892447352) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:411 - timing_s/agent_loop/slowest/response_length:742 - timing_s/gen:11.692811168730259 - timing_s/reward:0.051967186853289604 - timing_s/old_log_prob:2.230885935947299 - timing_s/adv:0.5060996375977993 - timing_s/update_actor:15.389210676774383 - timing_s/step:29.97092897631228 - timing_s/stop_profile:0.0001416075974702835 - timing_per_token_ms/adv:0.003551925365283602 - timing_per_token_ms/gen:0.15567582437398828 - timing_per_token_ms/update_actor:0.10800507191425392 - perf/total_num_tokens:142486 - perf/time_per_step:29.97092897631228 - perf/throughput:594.2675321834984 (TaskRunner pid=2026458) Training Progress: 95%|█████████▌| 95/100 [1:06:46<02:59, 35.91s/it] (TaskRunner pid=2026458) step:96 - global_seqlen/min:15504 - global_seqlen/max:21947 - global_seqlen/minmax_diff:6443 - global_seqlen/balanced_min:17847 - global_seqlen/balanced_max:17850 - global_seqlen/mean:17848.25 - actor/entropy:0.24302901327610016 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7704145312309265 - training/rollout_probs_diff_mean:0.0050596678629517555 - training/rollout_probs_diff_std:0.013549817726016045 - training/rollout_actor_probs_pearson_corr:0.9982749223709106 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000941753387451 - rollout_corr/rollout_is_ratio_fraction_high:4.025656744488515e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.709428271278739e-05 - rollout_corr/rollout_is_max:2.5671136379241943 - rollout_corr/rollout_is_min:0.13419120013713837 - rollout_corr/rollout_is_std:0.039693690836429596 - rollout_corr/rollout_is_eff_sample_size:0.9984272458997584 - rollout_corr/rollout_is_seq_mean:1.000090479850769 - rollout_corr/rollout_is_seq_std:0.0026221859734505415 - rollout_corr/rollout_is_seq_max:1.009491205215454 - rollout_corr/rollout_is_seq_min:0.9908090233802795 - rollout_corr/rollout_is_seq_max_deviation:0.009491205215454102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2904391190968454) - rollout_corr/training_log_ppl:np.float64(0.2496467249293346) - rollout_corr/kl:np.float64(0.0008864650635374005) - rollout_corr/k3_kl:np.float64(0.0009545528546368587) - rollout_corr/rollout_ppl:np.float64(1.2892436645925045) - rollout_corr/rollout_log_ppl:np.float64(0.24876026091806125) - rollout_corr/log_ppl_diff:np.float64(0.0008864640112733468) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021889270428800955) - rollout_corr/log_ppl_diff_max:np.float64(0.013020426034927368) - rollout_corr/log_ppl_diff_min:np.float64(-0.006367728114128113) - rollout_corr/ppl_ratio:np.float64(1.0008906812872738) - rollout_corr/chi2_token:np.float64(0.0020458281505852938) - rollout_corr/chi2_seq:np.float64(0.6757269599474967) - actor/pg_loss:np.float64(0.0028436718275770545) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006703252311126562) - actor/ppo_kl:np.float64(0.0001225946314420412) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(73408.10212582699) - self_distillation/token_reweight_w_std:np.float64(1221313.2646529041) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004472548607737) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07176223069836851) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.3494473174214363) - perf/mfu/actor:np.float64(0.03336222886368918) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.36438751220703) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0025512599386274815 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0025512599386274815 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.1015625 - response_length/max:636.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.1015625 - response_length_non_aborted/max:636.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012776069343090057 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3372939508408308) - timing_s/agent_loop/generate_sequences/max:np.float64(5.064935522153974) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.084805842998321) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.064935522153974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:572 - timing_s/gen:11.018641920760274 - timing_s/reward:0.05196058563888073 - timing_s/old_log_prob:2.2254901994019747 - timing_s/adv:0.4944445975124836 - timing_s/update_actor:15.122869728133082 - timing_s/step:29.00846175290644 - timing_s/stop_profile:2.9573217034339905e-05 - timing_per_token_ms/adv:0.0034628366752516605 - timing_per_token_ms/gen:0.14785757119723403 - timing_per_token_ms/update_actor:0.10591283268761001 - perf/total_num_tokens:142786 - perf/time_per_step:29.00846175290644 - perf/throughput:615.2773681014552 (TaskRunner pid=2026458) Training Progress: 96%|█████████▌| 96/100 [1:07:15<02:15, 33.85s/it] (TaskRunner pid=2026458) step:97 - global_seqlen/min:16071 - global_seqlen/max:23461 - global_seqlen/minmax_diff:7390 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18815 - global_seqlen/mean:18813.875 - actor/entropy:0.2445484697818756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4984980821609497 - training/rollout_probs_diff_mean:0.005207015201449394 - training/rollout_probs_diff_std:0.013489038683474064 - training/rollout_actor_probs_pearson_corr:0.9982696175575256 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:0.9999282360076904 - rollout_corr/rollout_is_ratio_fraction_high:1.330795930698514e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.65397965349257e-05 - rollout_corr/rollout_is_max:3.3646888732910156 - rollout_corr/rollout_is_min:0.3885670304298401 - rollout_corr/rollout_is_std:0.04048614203929901 - rollout_corr/rollout_is_eff_sample_size:0.9983633169728505 - rollout_corr/rollout_is_seq_mean:0.9999846816062927 - rollout_corr/rollout_is_seq_std:0.002680700272321701 - rollout_corr/rollout_is_seq_max:1.0078043937683105 - rollout_corr/rollout_is_seq_min:0.9888970255851746 - rollout_corr/rollout_is_seq_max_deviation:0.01110297441482544 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3025756301358342) - rollout_corr/training_log_ppl:np.float64(0.2600360629003262) - rollout_corr/kl:np.float64(0.001345034299074399) - rollout_corr/k3_kl:np.float64(0.0010471967440253138) - rollout_corr/rollout_ppl:np.float64(1.3007347490638494) - rollout_corr/rollout_log_ppl:np.float64(0.2586910269019427) - rollout_corr/log_ppl_diff:np.float64(0.001345035998383537) - rollout_corr/log_ppl_abs_diff:np.float64(0.002600655221613124) - rollout_corr/log_ppl_diff_max:np.float64(0.013873368501663208) - rollout_corr/log_ppl_diff_min:np.float64(-0.008637338876724243) - rollout_corr/ppl_ratio:np.float64(1.0013508247211576) - rollout_corr/chi2_token:np.float64(0.001424239482730627) - rollout_corr/chi2_seq:np.float64(0.4958528857678175) - actor/pg_loss:np.float64(0.0023965955479070544) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000528412345374818) - actor/ppo_kl:np.float64(0.00046769405014401855) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(76633.93915633648) - self_distillation/token_reweight_w_std:np.float64(1211350.6270655608) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995253349188715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0652933964447584) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.31651656329631805) - perf/mfu/actor:np.float64(0.03500956607334557) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4477882385254) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004978840239346027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004978840239346027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.52734375 - response_length/max:730.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.52734375 - response_length_non_aborted/max:730.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.985447645187378e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1220364272594452) - timing_s/agent_loop/generate_sequences/max:np.float64(5.692030295729637) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1152001094887964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.692030295729637) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:730 - timing_s/gen:11.270145887508988 - timing_s/reward:0.05703514441847801 - timing_s/old_log_prob:2.3115832544863224 - timing_s/adv:0.5203931517899036 - timing_s/update_actor:15.463589219376445 - timing_s/step:29.711128624156117 - timing_s/stop_profile:0.00012691877782344818 - timing_per_token_ms/adv:0.0034575090976068437 - timing_per_token_ms/gen:0.14998264492379848 - timing_per_token_ms/update_actor:0.10274059184628662 - perf/total_num_tokens:150511 - perf/time_per_step:29.711128624156117 - perf/throughput:633.2265340032793 (TaskRunner pid=2026458) Training Progress: 97%|█████████▋| 97/100 [1:07:45<01:37, 32.62s/it] (TaskRunner pid=2026458) step:98 - global_seqlen/min:14853 - global_seqlen/max:23186 - global_seqlen/minmax_diff:8333 - global_seqlen/balanced_min:19016 - global_seqlen/balanced_max:19021 - global_seqlen/mean:19019.875 - actor/entropy:0.23993536829948425 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2247922420501709 - training/rollout_probs_diff_mean:0.005032053682953119 - training/rollout_probs_diff_std:0.013124685734510422 - training/rollout_actor_probs_pearson_corr:0.99836266040802 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9997885227203369 - rollout_corr/rollout_is_ratio_fraction_high:2.6424617317388766e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.9636925066588446e-05 - rollout_corr/rollout_is_max:2.206674814224243 - rollout_corr/rollout_is_min:0.19827817380428314 - rollout_corr/rollout_is_std:0.04013793542981148 - rollout_corr/rollout_is_eff_sample_size:0.9983908838332385 - rollout_corr/rollout_is_seq_mean:0.9997617602348328 - rollout_corr/rollout_is_seq_std:0.0025118952617049217 - rollout_corr/rollout_is_seq_max:1.0087740421295166 - rollout_corr/rollout_is_seq_min:0.9924179315567017 - rollout_corr/rollout_is_seq_max_deviation:0.008774042129516602 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2941957069560885) - rollout_corr/training_log_ppl:np.float64(0.254570292920107) - rollout_corr/kl:np.float64(0.0012180306670934726) - rollout_corr/k3_kl:np.float64(0.000970762781207668) - rollout_corr/rollout_ppl:np.float64(1.2925921510905027) - rollout_corr/rollout_log_ppl:np.float64(0.25335226202150807) - rollout_corr/log_ppl_diff:np.float64(0.001218030898598954) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023932511976454407) - rollout_corr/log_ppl_diff_max:np.float64(0.008434802293777466) - rollout_corr/log_ppl_diff_min:np.float64(-0.005523428320884705) - rollout_corr/ppl_ratio:np.float64(1.0012223366647959) - rollout_corr/chi2_token:np.float64(0.0014376682229340076) - rollout_corr/chi2_seq:np.float64(0.6267413701862097) - actor/pg_loss:np.float64(0.003052193787880242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006503687072836328) - actor/ppo_kl:np.float64(0.000135291372203028) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(107416.14729232923) - self_distillation/token_reweight_w_std:np.float64(1626133.0662139303) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999873915920034) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0583975906483829) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.36374419182538986) - perf/mfu/actor:np.float64(0.03551806757350935) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40508651733398) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001578870927914977 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001578870927914977 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:295.65234375 - response_length/max:718.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:295.65234375 - response_length_non_aborted/max:718.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011442229151725769 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4412503708153963) - timing_s/agent_loop/generate_sequences/max:np.float64(5.591642700135708) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1361403591217822) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.591642700135708) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:718 - timing_s/gen:11.411643352359533 - timing_s/reward:0.05464274063706398 - timing_s/old_log_prob:2.2537942565977573 - timing_s/adv:0.5091272983700037 - timing_s/update_actor:15.217187775298953 - timing_s/step:29.54008462652564 - timing_s/stop_profile:0.00011652149260044098 - timing_per_token_ms/adv:0.003346021585118223 - timing_per_token_ms/gen:0.1507741534525022 - timing_per_token_ms/update_actor:0.1000084633528017 - perf/total_num_tokens:152159 - perf/time_per_step:29.54008462652564 - perf/throughput:643.8666388559031 (TaskRunner pid=2026458) Training Progress: 98%|█████████▊| 98/100 [1:08:14<01:03, 31.71s/it] (TaskRunner pid=2026458) step:99 - global_seqlen/min:16270 - global_seqlen/max:23777 - global_seqlen/minmax_diff:7507 - global_seqlen/balanced_min:18894 - global_seqlen/balanced_max:18898 - global_seqlen/mean:18896.125 - actor/entropy:0.23954525589942932 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5315115451812744 - training/rollout_probs_diff_mean:0.005148096010088921 - training/rollout_probs_diff_std:0.013385861180722713 - training/rollout_actor_probs_pearson_corr:0.9983036518096924 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.00002920627594 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.40447464218596e-05 - rollout_corr/rollout_is_max:1.9394363164901733 - rollout_corr/rollout_is_min:0.2951362729072571 - rollout_corr/rollout_is_std:0.04090065881609917 - rollout_corr/rollout_is_eff_sample_size:0.9983299298399158 - rollout_corr/rollout_is_seq_mean:1.0000901222229004 - rollout_corr/rollout_is_seq_std:0.0026464853435754776 - rollout_corr/rollout_is_seq_max:1.00861656665802 - rollout_corr/rollout_is_seq_min:0.9930309057235718 - rollout_corr/rollout_is_seq_max_deviation:0.00861656665802002 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2940249065868556) - rollout_corr/training_log_ppl:np.float64(0.25375668989727274) - rollout_corr/kl:np.float64(0.0008266884788987738) - rollout_corr/k3_kl:np.float64(0.0009614558624662095) - rollout_corr/rollout_ppl:np.float64(1.2929239594377577) - rollout_corr/rollout_log_ppl:np.float64(0.2529299992165761) - rollout_corr/log_ppl_diff:np.float64(0.000826690680696629) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025123875675490126) - rollout_corr/log_ppl_diff_max:np.float64(0.009278476238250732) - rollout_corr/log_ppl_diff_min:np.float64(-0.007872521877288818) - rollout_corr/ppl_ratio:np.float64(1.000831762328744) - rollout_corr/chi2_token:np.float64(0.002216932363808155) - rollout_corr/chi2_seq:np.float64(0.6580137938726693) - actor/pg_loss:np.float64(0.004063432686962187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007904909580247477) - actor/ppo_kl:np.float64(4.468652047506794e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(131557.28193879593) - self_distillation/token_reweight_w_std:np.float64(2127779.07550747) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999837173614651) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07958601297286805) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.4794868528842926) - perf/mfu/actor:np.float64(0.035181320287522105) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.34246444702148) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003239069599658251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003239069599658251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:278.12890625 - response_length/max:681.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:278.12890625 - response_length_non_aborted/max:681.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017566047608852386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2292504087090492) - timing_s/agent_loop/generate_sequences/max:np.float64(5.23788345977664) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0256088213573094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.23788345977664) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:681 - timing_s/gen:11.295774277299643 - timing_s/reward:0.05612022429704666 - timing_s/old_log_prob:2.242926351726055 - timing_s/adv:0.5197359751909971 - timing_s/update_actor:15.403783183544874 - timing_s/step:29.607021318748593 - timing_s/stop_profile:0.00011566095054149628 - timing_per_token_ms/adv:0.0034381121472722392 - timing_per_token_ms/gen:0.1586462869524254 - timing_per_token_ms/update_actor:0.101897764644503 - perf/total_num_tokens:151169 - perf/time_per_step:29.607021318748593 - perf/throughput:638.2312086232755 (TaskRunner pid=2026458) Training Progress: 99%|█████████▉| 99/100 [1:08:44<00:31, 31.09s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains the correct structure with a chlorine atom, a trifluoromethyl group, and the necessary aromatic ring, matching the product structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but lacks the correct trifluoromethyl group and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" contains a trifluoromethyl group and a chlorine atom but has an incorrect aromatic ring structure and additional functional groups not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the only option that matches the product structure is Option A. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2026458) step:100 - global_seqlen/min:17070 - global_seqlen/max:22010 - global_seqlen/minmax_diff:4940 - global_seqlen/balanced_min:19621 - global_seqlen/balanced_max:19622 - global_seqlen/mean:19621.75 - actor/entropy:0.21838146448135376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163663625717163 - training/rollout_probs_diff_mean:0.004920980893075466 - training/rollout_probs_diff_std:0.013346283696591854 - training/rollout_actor_probs_pearson_corr:0.9981935024261475 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000070333480835 - rollout_corr/rollout_is_ratio_fraction_high:1.2321644135226961e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.16082179476507e-05 - rollout_corr/rollout_is_max:2.1037662029266357 - rollout_corr/rollout_is_min:0.39213550090789795 - rollout_corr/rollout_is_std:0.04028838127851486 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:1.0000463724136353 - rollout_corr/rollout_is_seq_std:0.002407139865681529 - rollout_corr/rollout_is_seq_max:1.0090001821517944 - rollout_corr/rollout_is_seq_min:0.9934689998626709 - rollout_corr/rollout_is_seq_max_deviation:0.009000182151794434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2685748748481274) - rollout_corr/training_log_ppl:np.float64(0.23310650100756902) - rollout_corr/kl:np.float64(0.0009287315268036167) - rollout_corr/k3_kl:np.float64(0.001084987000297133) - rollout_corr/rollout_ppl:np.float64(1.2673596912063658) - rollout_corr/rollout_log_ppl:np.float64(0.2321777678298531) - rollout_corr/log_ppl_diff:np.float64(0.0009287331777159125) - rollout_corr/log_ppl_abs_diff:np.float64(0.002280793822137639) - rollout_corr/log_ppl_diff_max:np.float64(0.015468984842300415) - rollout_corr/log_ppl_diff_min:np.float64(-0.010868668556213379) - rollout_corr/ppl_ratio:np.float64(1.0009334608912468) - rollout_corr/chi2_token:np.float64(0.002497109118849039) - rollout_corr/chi2_seq:np.float64(1.0585334873758256) - actor/pg_loss:np.float64(0.0023272527614608407) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004233177678543143) - actor/ppo_kl:np.float64(0.00010989303795483352) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(73094.00544022233) - self_distillation/token_reweight_w_std:np.float64(1188161.9467263469) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000056762015447) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03758277394808829) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.25844827108085155) - perf/mfu/actor:np.float64(0.03632957727570209) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.29161834716797) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/reward/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.05779624954713381) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8013714285714286) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03927314998986781) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7074904761904762) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.03834834240388913) - val-aux/sciknoweval/score/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/score/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/score/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.05779624954713381) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8013714285714286) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03927314998986781) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7074904761904762) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.03834834240388913) - val-core/sciknoweval/acc/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/acc/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.05779624954713381) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8013714285714286) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03927314998986781) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7074904761904762) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.03834834240388913) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994476190476191) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015248805759714617) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011569664768572288) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988761904761905) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0020220104047945714) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999476190476191) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0004966783087710074) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980238095238095) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.002346295386522406) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9967095238095238) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0022003864842959084) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011474530911073089 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011474530911073089 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:317.0234375 - response_length/max:616.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:317.0234375 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.15625 - prompt_length/max:625.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018987245857715607 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5517499037086964) - timing_s/agent_loop/generate_sequences/max:np.float64(5.5460103284567595) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.556732240387646) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.5460103284567595) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:602 - timing_s/gen:11.436559945344925 - timing_s/reward:0.05615238472819328 - timing_s/old_log_prob:2.225490480661392 - timing_s/adv:0.4910304043442011 - timing_s/update_actor:15.488128297030926 - timing_s/step:29.756166722625494 - timing_s/testing:108.75369282439351 - timing_s/save_checkpoint:7.179236024618149 - timing_s/stop_profile:0.00012516416609287262 - timing_per_token_ms/adv:0.003128100222611395 - timing_per_token_ms/gen:0.14091722252082264 - timing_per_token_ms/update_actor:0.0986668384384097 - perf/total_num_tokens:156974 - perf/time_per_step:29.756166722625494 - perf/throughput:659.4179345379304 (TaskRunner pid=2026458) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2026458) 'np.float64(0.03834834240388913), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/mean@16': " (TaskRunner pid=2026458) 'np.float64(0.999702380952381), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2026458) 'np.float64(0.0011526736149426837), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2026458) 'np.float64(0.9999904761904761), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0002127457895589398), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2026458) 'np.float64(0.9994476190476191), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0015248805759714617), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9997), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0011569664768572288), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2026458) 'np.float64(0.9988761904761905), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2026458) 'np.float64(0.0020220104047945714), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2026458) 'np.float64(0.9999476190476191), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2026458) 'np.float64(0.0004966783087710074), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2026458) 'np.float64(0.9980238095238095), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2026458) 'np.float64(0.002346295386522406), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2026458) 'np.float64(0.9967095238095238), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2026458) 'np.float64(0.0022003864842959084), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2026458) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2026458) Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 65.49s/it] Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 42.70s/it] (TaskRunner pid=2026458) wandb: updating run metadata (TaskRunner pid=2026458) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2026458) wandb: uploading output.log (TaskRunner pid=2026458) wandb: uploading history steps 98-99, summary, console lines 651-774 (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: Run history: (TaskRunner pid=2026458) wandb: actor/entropy ▂▁▂▅▄█▅▅▃▆▆▆▇█▆▅▆▆▄▆▃▂▄▅▄▃▃▄▃▄▃▃▄▂▂▂▂▂▃▂ (TaskRunner pid=2026458) wandb: actor/grad_norm ▆▄▆▆▆▄▆▅▄▅▆▇▅▄▄▄▃▄▃▃▄▃█▃▄▄▃▁▂▃▃▂▂▃▃▁▃▃▂▂ (TaskRunner pid=2026458) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2026458) wandb: actor/lr ▁▃▄▅▅▇██████████████████████████████████ (TaskRunner pid=2026458) wandb: actor/pg_clipfrac ▁▅▆▆▅▅▄▅▃█▃▆▄▄▆▃▃▆▃▃▃▅▃▇▆▄▃▄▄▄▅▃▃▄▅▂▄▂▅▂ (TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▅▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁ (TaskRunner pid=2026458) wandb: actor/pg_loss █▅▅▆▆▄▇▅▄▄▃▃▃▄▁▂▃▂▄▃▃▃▃▂▃▂▃▂▂▂▃▃▃▄▄▁▂▂▂▂ (TaskRunner pid=2026458) wandb: actor/ppo_kl ▃▄▁▃▆▂▅▃▄▂▁▃▃█▆▆▄▄▅▅▃▅▆▂▆▆▆▂█▄▃▄▆▄▅▆█▄▇▄ (TaskRunner pid=2026458) wandb: critic/advantages/max █████████▃██▆████████████████▆██▆▆▆▁████ (TaskRunner pid=2026458) wandb: critic/advantages/mean ▃█▅▄▃▂▃▄▁▂▃▄▄▁▄▃▄▂▃▅▄▄▃▄▃▂▄▃▃▄▄▂▄▁▄▃▃▄▂▃ (TaskRunner pid=2026458) wandb: +220 ... (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: Run summary: (TaskRunner pid=2026458) wandb: actor/entropy 0.21838 (TaskRunner pid=2026458) wandb: actor/grad_norm 0.25845 (TaskRunner pid=2026458) wandb: actor/kl_loss 0 (TaskRunner pid=2026458) wandb: actor/lr 0.0 (TaskRunner pid=2026458) wandb: actor/pg_clipfrac 0.00042 (TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2026458) wandb: actor/pg_loss 0.00233 (TaskRunner pid=2026458) wandb: actor/ppo_kl 0.00011 (TaskRunner pid=2026458) wandb: critic/advantages/max 0.75 (TaskRunner pid=2026458) wandb: critic/advantages/mean -0.00115 (TaskRunner pid=2026458) wandb: +220 ... (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: 🚀 View run qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/bz6p2yxy (TaskRunner pid=2026458) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2026458) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2026458) wandb: Find logs at: ./wandb/run-20260702_062054-bz6p2yxy/logs (TaskRunner pid=2026458) Exception ignored in atexit callback: .teardown_atexit at 0x7a92e933fb00> (TaskRunner pid=2026458) Traceback (most recent call last): (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2026458) conn.teardown(hooks.exit_code) (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2026458) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2026458) return future.result() (TaskRunner pid=2026458) ^^^^^^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2026458) return self.__get_result() (TaskRunner pid=2026458) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2026458) raise self._exception (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2026458) return await fn() (TaskRunner pid=2026458) ^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2026458) await self._client.close() (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2026458) await self._writer.wait_closed() (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2026458) await self._protocol._get_close_waiter(self) (TaskRunner pid=2026458) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 07:32:12] Finished chemistry rlsd_tr [2026-07-02 07:32:12] Starting physics grpo Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_grpo_Qwen3_4B 2026-07-02 07:32:14,409 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 07:32:24,172 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2049544) TaskRunner hostname: mole-workspace-rw, PID: 2049544 (TaskRunner pid=2049544) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2049544) 'calculate_entropy': False, (TaskRunner pid=2049544) 'calculate_sum_pi_squared': False, (TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2049544) 'async_save': False, (TaskRunner pid=2049544) 'load_contents': ['model', (TaskRunner pid=2049544) 'optimizer', (TaskRunner pid=2049544) 'extra'], (TaskRunner pid=2049544) 'save_contents': ['model', (TaskRunner pid=2049544) 'optimizer', (TaskRunner pid=2049544) 'extra']}, (TaskRunner pid=2049544) 'clip_ratio': 0.2, (TaskRunner pid=2049544) 'clip_ratio_c': 3.0, (TaskRunner pid=2049544) 'clip_ratio_high': 0.28, (TaskRunner pid=2049544) 'clip_ratio_low': 0.2, (TaskRunner pid=2049544) 'data_loader_seed': 42, (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_coeff': 0, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'freeze_vision_tower': False, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': False, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'grad_clip': 1.0, (TaskRunner pid=2049544) 'kl_loss_coef': 0.0, (TaskRunner pid=2049544) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2049544) 'loss_scale_factor': None, (TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2049544) 'betas': [0.9, 0.999], (TaskRunner pid=2049544) 'clip_grad': 1.0, (TaskRunner pid=2049544) 'lr': 1e-06, (TaskRunner pid=2049544) 'lr_scheduler_type': 'constant', (TaskRunner pid=2049544) 'lr_warmup_steps': 10, (TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2049544) 'min_lr_ratio': 0.0, (TaskRunner pid=2049544) 'num_cycles': 0.5, (TaskRunner pid=2049544) 'optimizer': 'AdamW', (TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2049544) 'override_optimizer_config': None, (TaskRunner pid=2049544) 'total_training_steps': -1, (TaskRunner pid=2049544) 'warmup_style': None, (TaskRunner pid=2049544) 'weight_decay': 0.01}, (TaskRunner pid=2049544) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2049544) 'clip_cov_lb': 1.0, (TaskRunner pid=2049544) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2049544) 'clip_cov_ub': 5.0, (TaskRunner pid=2049544) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2049544) 'loss_mode': 'vanilla', (TaskRunner pid=2049544) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2049544) 'ppo_epochs': 1, (TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'ppo_micro_batch_size': None, (TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'ppo_mini_batch_size': 8, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2049544) 'mode': 'disabled', (TaskRunner pid=2049544) 'record_file': None, (TaskRunner pid=2049544) 'replay_file': None}, (TaskRunner pid=2049544) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2049544) 'alpha': 0.5, (TaskRunner pid=2049544) 'distillation_add_tail': True, (TaskRunner pid=2049544) 'distillation_topk': 100, (TaskRunner pid=2049544) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2049544) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2049544) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'loss_weight': 0.0, (TaskRunner pid=2049544) 'mask': 'all'}, (TaskRunner pid=2049544) 'feedback_template': '\n' (TaskRunner pid=2049544) 'The ' (TaskRunner pid=2049544) 'following ' (TaskRunner pid=2049544) 'is ' (TaskRunner pid=2049544) 'feedback ' (TaskRunner pid=2049544) 'from ' (TaskRunner pid=2049544) 'your ' (TaskRunner pid=2049544) 'unsuccessful ' (TaskRunner pid=2049544) 'earlier ' (TaskRunner pid=2049544) 'attempt:\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) '{feedback_raw}', (TaskRunner pid=2049544) 'full_logit_distillation': True, (TaskRunner pid=2049544) 'include_environment_feedback': True, (TaskRunner pid=2049544) 'is_clip': 2, (TaskRunner pid=2049544) 'max_reprompt_len': 22528, (TaskRunner pid=2049544) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2049544) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) 'Correctly ' (TaskRunner pid=2049544) 'solve ' (TaskRunner pid=2049544) 'the ' (TaskRunner pid=2049544) 'original ' (TaskRunner pid=2049544) 'question.', (TaskRunner pid=2049544) 'reprompt_truncation': 'right', (TaskRunner pid=2049544) 'solution_template': '\n' (TaskRunner pid=2049544) 'Correct ' (TaskRunner pid=2049544) 'solution:\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) '{successful_previous_attempt}', (TaskRunner pid=2049544) 'srpo_dynamic_weighting': False, (TaskRunner pid=2049544) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2049544) 'success_reward_threshold': 0.5, (TaskRunner pid=2049544) 'teacher_regularization': 'ema', (TaskRunner pid=2049544) 'teacher_update_rate': 0.0, (TaskRunner pid=2049544) 'token_reweight_decay_steps': None, (TaskRunner pid=2049544) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2049544) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2049544) 'shuffle': False, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2049544) 'tau_neg': 1.05, (TaskRunner pid=2049544) 'tau_pos': 1.0, (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_kl_loss': False, (TaskRunner pid=2049544) 'use_prefix_grouper': False, (TaskRunner pid=2049544) 'use_remove_padding': True, (TaskRunner pid=2049544) 'use_torch_compile': True}, (TaskRunner pid=2049544) 'hybrid_engine': True, (TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2049544) 'custom_chat_template': None, (TaskRunner pid=2049544) 'enable_activation_offload': False, (TaskRunner pid=2049544) 'enable_gradient_checkpointing': True, (TaskRunner pid=2049544) 'exclude_modules': None, (TaskRunner pid=2049544) 'external_lib': None, (TaskRunner pid=2049544) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2049544) 'hf_config_path': None, (TaskRunner pid=2049544) 'lora_adapter_path': None, (TaskRunner pid=2049544) 'lora_alpha': 16, (TaskRunner pid=2049544) 'lora_rank': 0, (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'target_modules': 'all-linear', (TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2049544) 'num_shards': 4}, (TaskRunner pid=2049544) 'tokenizer_path': None, (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_liger': False, (TaskRunner pid=2049544) 'use_remove_padding': True, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'nccl_timeout': 600, (TaskRunner pid=2049544) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': True, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'log_prob_micro_batch_size': None, (TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2049544) 'mode': 'disabled', (TaskRunner pid=2049544) 'record_file': None, (TaskRunner pid=2049544) 'replay_file': None}, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_torch_compile': True}, (TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2049544) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2049544) 'agent_loop_config_path': None, (TaskRunner pid=2049544) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2049544) 'name': None, (TaskRunner pid=2049544) 'path': None}, (TaskRunner pid=2049544) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2049544) 'num_workers': 8}, (TaskRunner pid=2049544) 'calculate_log_probs': True, (TaskRunner pid=2049544) 'cudagraph_capture_sizes': None, (TaskRunner pid=2049544) 'data_parallel_size': 1, (TaskRunner pid=2049544) 'disable_log_stats': True, (TaskRunner pid=2049544) 'do_sample': True, (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'enable_chunked_prefill': True, (TaskRunner pid=2049544) 'enable_prefix_caching': True, (TaskRunner pid=2049544) 'enable_rollout_routing_replay': False, (TaskRunner pid=2049544) 'enforce_eager': False, (TaskRunner pid=2049544) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2049544) 'expert_parallel_size': 1, (TaskRunner pid=2049544) 'free_cache_engine': True, (TaskRunner pid=2049544) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2049544) 'ignore_eos': False, (TaskRunner pid=2049544) 'layered_summon': False, (TaskRunner pid=2049544) 'load_format': 'dummy', (TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'log_prob_micro_batch_size': None, (TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2049544) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2049544) 'max_model_len': 10240, (TaskRunner pid=2049544) 'max_num_batched_tokens': 10240, (TaskRunner pid=2049544) 'max_num_seqs': 1024, (TaskRunner pid=2049544) 'mode': 'async', (TaskRunner pid=2049544) 'multi_stage_wake_up': False, (TaskRunner pid=2049544) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'format': 'hermes', (TaskRunner pid=2049544) 'interaction_config_path': None, (TaskRunner pid=2049544) 'max_assistant_turns': None, (TaskRunner pid=2049544) 'max_parallel_calls': 1, (TaskRunner pid=2049544) 'max_tool_response_length': 256, (TaskRunner pid=2049544) 'max_user_turns': None, (TaskRunner pid=2049544) 'num_repeat_rollouts': None, (TaskRunner pid=2049544) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2049544) 'tool_config_path': None, (TaskRunner pid=2049544) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2049544) 'use_inference_chat_template': False}, (TaskRunner pid=2049544) 'n': 8, (TaskRunner pid=2049544) 'name': 'vllm', (TaskRunner pid=2049544) 'over_sample_rate': 0, (TaskRunner pid=2049544) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2049544) 'port': 9090, (TaskRunner pid=2049544) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2049544) 'prompt_length': 2048, (TaskRunner pid=2049544) 'quantization': None, (TaskRunner pid=2049544) 'quantization_config_file': None, (TaskRunner pid=2049544) 'response_length': 8192, (TaskRunner pid=2049544) 'scheduling_policy': 'fcfs', (TaskRunner pid=2049544) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2049544) 'skip_rollout': False, (TaskRunner pid=2049544) 'skip_tokenizer_init': True, (TaskRunner pid=2049544) 'temperature': 1.0, (TaskRunner pid=2049544) 'tensor_model_parallel_size': 2, (TaskRunner pid=2049544) 'top_k': -1, (TaskRunner pid=2049544) 'top_p': 1.0, (TaskRunner pid=2049544) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2049544) 'backend': None, (TaskRunner pid=2049544) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2049544) 'token2text': False}, (TaskRunner pid=2049544) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2049544) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2049544) 'do_sample': True, (TaskRunner pid=2049544) 'n': 16, (TaskRunner pid=2049544) 'temperature': 0.6, (TaskRunner pid=2049544) 'top_k': -1, (TaskRunner pid=2049544) 'top_p': 0.95}}}, (TaskRunner pid=2049544) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2049544) 'adv_estimator': 'grpo', (TaskRunner pid=2049544) 'gamma': 1.0, (TaskRunner pid=2049544) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2049544) 'horizon': 10000, (TaskRunner pid=2049544) 'kl_coef': 0.001, (TaskRunner pid=2049544) 'target_kl': 0.1, (TaskRunner pid=2049544) 'type': 'fixed'}, (TaskRunner pid=2049544) 'kl_penalty': 'kl', (TaskRunner pid=2049544) 'lam': 1.0, (TaskRunner pid=2049544) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2049544) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2049544) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2049544) 'loss_type': 'ppo_clip', (TaskRunner pid=2049544) 'rollout_is': 'token', (TaskRunner pid=2049544) 'rollout_is_batch_normalize': False, (TaskRunner pid=2049544) 'rollout_is_threshold': 2.0, (TaskRunner pid=2049544) 'rollout_rs': None, (TaskRunner pid=2049544) 'rollout_rs_threshold': None}, (TaskRunner pid=2049544) 'use_kl_in_reward': False, (TaskRunner pid=2049544) 'use_pf_ppo': False}, (TaskRunner pid=2049544) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2049544) 'async_save': False, (TaskRunner pid=2049544) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2049544) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2049544) 'cliprange_value': 0.5, (TaskRunner pid=2049544) 'data_loader_seed': 42, (TaskRunner pid=2049544) 'enable': None, (TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'forward_micro_batch_size': None, (TaskRunner pid=2049544) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'grad_clip': 1.0, (TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2049544) 'enable_activation_offload': False, (TaskRunner pid=2049544) 'enable_gradient_checkpointing': True, (TaskRunner pid=2049544) 'external_lib': None, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': False, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'lora_alpha': 16, (TaskRunner pid=2049544) 'lora_rank': 0, (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2049544) 'target_modules': 'all-linear', (TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2049544) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_remove_padding': False, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2049544) 'betas': [0.9, 0.999], (TaskRunner pid=2049544) 'clip_grad': 1.0, (TaskRunner pid=2049544) 'lr': 1e-05, (TaskRunner pid=2049544) 'lr_scheduler_type': 'constant', (TaskRunner pid=2049544) 'lr_warmup_steps': -1, (TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2049544) 'min_lr_ratio': 0.0, (TaskRunner pid=2049544) 'num_cycles': 0.5, (TaskRunner pid=2049544) 'optimizer': 'AdamW', (TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2049544) 'override_optimizer_config': None, (TaskRunner pid=2049544) 'total_training_steps': -1, (TaskRunner pid=2049544) 'warmup_style': None, (TaskRunner pid=2049544) 'weight_decay': 0.01}, (TaskRunner pid=2049544) 'ppo_epochs': 1, (TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'ppo_micro_batch_size': None, (TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'ppo_mini_batch_size': 8, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'shuffle': False, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False}, (TaskRunner pid=2049544) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2049544) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2049544) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2049544) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2049544) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2049544) 'dataloader_num_workers': 8, (TaskRunner pid=2049544) 'filter_overlong_prompts': True, (TaskRunner pid=2049544) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2049544) 'image_key': 'images', (TaskRunner pid=2049544) 'image_patch_size': 14, (TaskRunner pid=2049544) 'max_prompt_length': 2048, (TaskRunner pid=2049544) 'max_response_length': 8192, (TaskRunner pid=2049544) 'prompt_key': 'prompt', (TaskRunner pid=2049544) 'return_full_prompt': False, (TaskRunner pid=2049544) 'return_multi_modal_inputs': True, (TaskRunner pid=2049544) 'return_raw_chat': True, (TaskRunner pid=2049544) 'return_raw_input_ids': False, (TaskRunner pid=2049544) 'reward_fn_key': 'data_source', (TaskRunner pid=2049544) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2049544) 'seed': None, (TaskRunner pid=2049544) 'shuffle': True, (TaskRunner pid=2049544) 'tokenizer': None, (TaskRunner pid=2049544) 'tool_config_path': None, (TaskRunner pid=2049544) 'train_batch_size': 32, (TaskRunner pid=2049544) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2049544) 'train_max_samples': 3200, (TaskRunner pid=2049544) 'truncation': 'error', (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_shm': False, (TaskRunner pid=2049544) 'val_batch_size': None, (TaskRunner pid=2049544) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2049544) 'val_max_samples': -1, (TaskRunner pid=2049544) 'validation_shuffle': False, (TaskRunner pid=2049544) 'video_key': 'videos'}, (TaskRunner pid=2049544) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2049544) 'cuda-memory-usage': 'true', (TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2049544) 'capture-range-end': None, (TaskRunner pid=2049544) 'cuda-graph-trace': 'graph', (TaskRunner pid=2049544) 'cuda-memory-usage': 'true', (TaskRunner pid=2049544) 'kill': 'none', (TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2049544) 'torch_memory': {'context': 'all', (TaskRunner pid=2049544) 'kw_args': {}, (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'stacks': 'all', (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2049544) 'profile_continuous_steps': False, (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'steps': None, (TaskRunner pid=2049544) 'tool': None}, (TaskRunner pid=2049544) 'max_model_len': 10240, (TaskRunner pid=2049544) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', (TaskRunner pid=2049544) 'include_dashboard': False, (TaskRunner pid=2049544) 'num_cpus': None, (TaskRunner pid=2049544) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2049544) 'USER': 'root'}}}, (TaskRunner pid=2049544) 'timeline_json_file': None}, (TaskRunner pid=2049544) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2049544) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2049544) 'name': 'custom_reward_manager', (TaskRunner pid=2049544) 'path': None}, (TaskRunner pid=2049544) 'name': 'naive', (TaskRunner pid=2049544) 'source': 'register'}, (TaskRunner pid=2049544) 'reward_model': {'enable': False, (TaskRunner pid=2049544) 'enable_resource_pool': False, (TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'launch_reward_fn_async': False, (TaskRunner pid=2049544) 'max_length': None, (TaskRunner pid=2049544) 'micro_batch_size': None, (TaskRunner pid=2049544) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'model': {'external_lib': None, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2049544) 'trust_remote_code': False, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_remove_padding': False, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'n_gpus_per_node': 8, (TaskRunner pid=2049544) 'nnodes': 0, (TaskRunner pid=2049544) 'num_workers': 1, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'reward_loop_class_name': None, (TaskRunner pid=2049544) 'reward_loop_module_path': None, (TaskRunner pid=2049544) 'reward_loop_source': 'register', (TaskRunner pid=2049544) 'reward_manager': 'naive', (TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2049544) 'cudagraph_capture_sizes': None, (TaskRunner pid=2049544) 'data_parallel_size': 1, (TaskRunner pid=2049544) 'disable_log_stats': True, (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'enable_chunked_prefill': True, (TaskRunner pid=2049544) 'enable_prefix_caching': True, (TaskRunner pid=2049544) 'enforce_eager': True, (TaskRunner pid=2049544) 'engine_kwargs': {}, (TaskRunner pid=2049544) 'expert_parallel_size': 1, (TaskRunner pid=2049544) 'free_cache_engine': True, (TaskRunner pid=2049544) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2049544) 'limit_images': None, (TaskRunner pid=2049544) 'load_format': 'auto', (TaskRunner pid=2049544) 'max_model_len': None, (TaskRunner pid=2049544) 'max_num_batched_tokens': 8192, (TaskRunner pid=2049544) 'max_num_seqs': 1024, (TaskRunner pid=2049544) 'name': '???', (TaskRunner pid=2049544) 'prompt_length': 2048, (TaskRunner pid=2049544) 'response_length': 2048, (TaskRunner pid=2049544) 'skip_tokenizer_init': False, (TaskRunner pid=2049544) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2049544) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2049544) 'memory_limit_mb': 1024, (TaskRunner pid=2049544) 'url': None}, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False, (TaskRunner pid=2049544) 'use_reward_loop': False}, (TaskRunner pid=2049544) 'trainer': {'balance_batch': True, (TaskRunner pid=2049544) 'critic_warmup': 0, (TaskRunner pid=2049544) 'default_hdfs_dir': None, (TaskRunner pid=2049544) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'del_local_ckpt_after_load': False, (TaskRunner pid=2049544) 'device': 'cuda', (TaskRunner pid=2049544) 'esi_redundant_time': 0, (TaskRunner pid=2049544) 'experiment_name': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2049544) 'log_val_generations': 0, (TaskRunner pid=2049544) 'logger': ['console', 'wandb'], (TaskRunner pid=2049544) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2049544) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2049544) 'n_gpus_per_node': 8, (TaskRunner pid=2049544) 'nnodes': 1, (TaskRunner pid=2049544) 'project_name': 'SDPO-root', (TaskRunner pid=2049544) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2049544) 'resume_from_path': None, (TaskRunner pid=2049544) 'resume_mode': 'auto', (TaskRunner pid=2049544) 'rollout_data_dir': None, (TaskRunner pid=2049544) 'save_freq': 10, (TaskRunner pid=2049544) 'test_freq': 10, (TaskRunner pid=2049544) 'total_epochs': 30, (TaskRunner pid=2049544) 'total_training_steps': 100, (TaskRunner pid=2049544) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2049544) 'val_before_train': False, (TaskRunner pid=2049544) 'val_only': False, (TaskRunner pid=2049544) 'validation_data_dir': None}, (TaskRunner pid=2049544) 'transfer_queue': {'enable': False}, (TaskRunner pid=2049544) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2049544) 'dir': '/users/root/SDPO', (TaskRunner pid=2049544) 'log_dir': '/users/root/output', (TaskRunner pid=2049544) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2049544) use_critic=need_critic(config), (TaskRunner pid=2049544) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2049544) Using dataset class: RLHFDataset (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2049544) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2049544) dataset len: 720 (TaskRunner pid=2049544) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2049544) WARNING:2026-07-02 07:32:33,214:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2049544) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 121.99 examples/s] (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2049544) self.use_critic = need_critic(self.config) (WorkerDict pid=2049933) [W702 07:32:42.949021168 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2049934) [W702 07:32:42.120202714 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2049932) Model config after override: Qwen3Config { (WorkerDict pid=2049932) "architectures": [ (WorkerDict pid=2049932) "Qwen3ForCausalLM" (WorkerDict pid=2049932) ], (WorkerDict pid=2049932) "attention_bias": false, (WorkerDict pid=2049932) "attention_dropout": 0.0, (WorkerDict pid=2049932) "dtype": "bfloat16", (WorkerDict pid=2049932) "eos_token_id": 151645, (WorkerDict pid=2049932) "head_dim": 128, (WorkerDict pid=2049932) "hidden_act": "silu", (WorkerDict pid=2049932) "hidden_size": 2560, (WorkerDict pid=2049932) "initializer_range": 0.02, (WorkerDict pid=2049932) "intermediate_size": 9728, (WorkerDict pid=2049932) "layer_types": [ (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention" (WorkerDict pid=2049932) ], (WorkerDict pid=2049932) "max_position_embeddings": 40960, (WorkerDict pid=2049932) "max_window_layers": 36, (WorkerDict pid=2049932) "model_type": "qwen3", (WorkerDict pid=2049932) "num_attention_heads": 32, (WorkerDict pid=2049932) "num_hidden_layers": 36, (WorkerDict pid=2049932) "num_key_value_heads": 8, (WorkerDict pid=2049932) "pad_token_id": 151643, (WorkerDict pid=2049932) "rms_norm_eps": 1e-06, (WorkerDict pid=2049932) "rope_scaling": null, (WorkerDict pid=2049932) "rope_theta": 1000000, (WorkerDict pid=2049932) "sliding_window": null, (WorkerDict pid=2049932) "tie_word_embeddings": true, (WorkerDict pid=2049932) "transformers_version": "4.57.1", (WorkerDict pid=2049932) "use_cache": true, (WorkerDict pid=2049932) "use_sliding_window": false, (WorkerDict pid=2049932) "vocab_size": 151936 (WorkerDict pid=2049932) } (WorkerDict pid=2049932) (WorkerDict pid=2049932) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2049932) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2049932) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2049932) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2049932) Actor use_remove_padding=True (WorkerDict pid=2049932) Actor use_fused_kernels=False (WorkerDict pid=2049932) Actor use_prefix_grouper=False (WorkerDict pid=2049939) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2049939) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2049932) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2049939) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.92s/it] [repeated 12x across cluster] (WorkerDict pid=2049939) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster] (WorkerDict pid=2049938) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.88s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.94s/it] [repeated 2x across cluster] (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2049939) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2049933) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2049933) warnings.warn( (WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2049939) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2049932) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2050805) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2050805) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2050806) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2050806) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2050805) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2050805) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2050803) ['serve', (vLLMHttpServer pid=2050803) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2050803) '--dtype', (vLLMHttpServer pid=2050803) 'bfloat16', (vLLMHttpServer pid=2050803) '--load_format', (vLLMHttpServer pid=2050803) 'dummy', (vLLMHttpServer pid=2050803) '--trust_remote_code', (vLLMHttpServer pid=2050803) '--max_model_len', (vLLMHttpServer pid=2050803) '40960', (vLLMHttpServer pid=2050803) '--max_num_seqs', (vLLMHttpServer pid=2050803) '1024', (vLLMHttpServer pid=2050803) '--enable_chunked_prefill', (vLLMHttpServer pid=2050803) '--max_num_batched_tokens', (vLLMHttpServer pid=2050803) '10240', (vLLMHttpServer pid=2050803) '--enable_prefix_caching', (vLLMHttpServer pid=2050803) '--enable_sleep_mode', (vLLMHttpServer pid=2050803) '--logprobs_mode', (vLLMHttpServer pid=2050803) 'processed_logprobs', (vLLMHttpServer pid=2050803) '--disable_custom_all_reduce', (vLLMHttpServer pid=2050803) '--gpu_memory_utilization', (vLLMHttpServer pid=2050803) '0.8', (vLLMHttpServer pid=2050803) '--disable_log_stats', (vLLMHttpServer pid=2050803) '--tensor_parallel_size', (vLLMHttpServer pid=2050803) '2', (vLLMHttpServer pid=2050803) '--seed', (vLLMHttpServer pid=2050803) '0', (vLLMHttpServer pid=2050803) '--override_generation_config', (vLLMHttpServer pid=2050803) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2050803) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2050803) '--hf_overrides', (vLLMHttpServer pid=2050803) '{}', (vLLMHttpServer pid=2050803) '--scheduling_policy', (vLLMHttpServer pid=2050803) 'fcfs'] (WorkerDict pid=2049938) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2050805) WARNING 07-02 07:33:26 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2049936) WARNING 07-02 07:33:34 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2050803) WARNING 07-02 07:33:27 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2049934) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2049934) 2026-07-02 07:33:49,587 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2049934) 2026-07-02 07:33:49,606 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2050803) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2050803) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2050803) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2050803) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substituting the given values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is B: 56 pC. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2049544) step:10 - global_seqlen/min:15442 - global_seqlen/max:25018 - global_seqlen/minmax_diff:9576 - global_seqlen/balanced_min:21213 - global_seqlen/balanced_max:21224 - global_seqlen/mean:21220.5 - actor/entropy:0.12735222280025482 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40465062856674194 - training/rollout_probs_diff_mean:0.0038589579053223133 - training/rollout_probs_diff_std:0.013698565773665905 - training/rollout_actor_probs_pearson_corr:0.9970982670783997 - rollout_corr/rollout_is_mean:1.0000947713851929 - rollout_corr/rollout_is_ratio_fraction_high:3.965264113503508e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020817638142034411 - rollout_corr/rollout_is_max:3.167215347290039 - rollout_corr/rollout_is_min:0.19278468191623688 - rollout_corr/rollout_is_std:0.03876073658466339 - rollout_corr/rollout_is_eff_sample_size:0.9985000968313013 - rollout_corr/rollout_is_seq_mean:1.000048041343689 - rollout_corr/rollout_is_seq_std:0.002764123724773526 - rollout_corr/rollout_is_seq_max:1.0100802183151245 - rollout_corr/rollout_is_seq_min:0.9903356432914734 - rollout_corr/rollout_is_seq_max_deviation:0.010080218315124512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1353368978016078) - rollout_corr/training_log_ppl:np.float64(0.12389787597203394) - rollout_corr/kl:np.float64(0.000877203190441378) - rollout_corr/k3_kl:np.float64(0.0008888360377170557) - rollout_corr/rollout_ppl:np.float64(1.134331472683698) - rollout_corr/rollout_log_ppl:np.float64(0.12302067201380851) - rollout_corr/log_ppl_diff:np.float64(0.0008772039582254365) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020878618815913796) - rollout_corr/log_ppl_diff_max:np.float64(0.012568935751914978) - rollout_corr/log_ppl_diff_min:np.float64(-0.007140792906284332) - rollout_corr/ppl_ratio:np.float64(1.000881330575794) - rollout_corr/chi2_token:np.float64(0.0018501763697713614) - rollout_corr/chi2_seq:np.float64(0.666111497906968) - actor/pg_loss:np.float64(9.953684639185667e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005019576683480409) - actor/ppo_kl:np.float64(2.3842763525294686e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3116539753973484) - perf/mfu/actor:np.float64(0.0619170865443955) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.6328125) - perf/cpu_memory_used_gb:np.float64(98.92141723632812) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1013687608239918) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.777675) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.047588411442781434) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6271749999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07429798598036827) - val-aux/sciknoweval/score/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/score/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1013687608239918) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.777675) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.047588411442781434) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6271749999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07429798598036827) - val-core/sciknoweval/acc/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1013687608239918) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.777675) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.047588411442781434) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6271749999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07429798598036827) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99609375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.015128841196122723) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0035548642615965665) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.992525) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0202257687927191) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.996175) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.014896175746267976) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.986025) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.026012467615322592) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.998875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.008222303730994859) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9748249999999998) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.030646040504587458) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996625) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0039815903242650785) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9594374999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.02978871055390577) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999500000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0010787124614257437) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008162248879671097 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008162248879671097 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:394.046875 - response_length/max:1270.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.046875 - response_length_non_aborted/max:1270.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013142451643943787 - timing_s/agent_loop/generate_sequences/min:np.float64(0.941465875133872) - timing_s/agent_loop/generate_sequences/max:np.float64(8.937132587656379) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.961100311316841) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.937132587656379) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1270 - timing_s/gen:14.806582853198051 - timing_s/reward:0.059918224811553955 - timing_s/old_log_prob:2.401917176321149 - timing_s/adv:0.15370049141347408 - timing_s/update_actor:9.923850152641535 - timing_s/step:27.43245600350201 - timing_s/testing:88.77899980731308 - timing_s/save_checkpoint:6.549710331484675 - timing_s/stop_profile:0.00011613219976425171 - timing_per_token_ms/update_actor:0.058456740844004235 - timing_per_token_ms/gen:0.14678003542168655 - timing_per_token_ms/adv:0.0009053774146077736 - perf/total_num_tokens:169764 - perf/time_per_step:27.43245600350201 - perf/throughput:773.5545077440753 (TaskRunner pid=2049544) Training Progress: 10%|█ | 10/100 [06:47<1:27:44, 58.49s/it] (TaskRunner pid=2049544) step:11 - global_seqlen/min:16821 - global_seqlen/max:31114 - global_seqlen/minmax_diff:14293 - global_seqlen/balanced_min:23187 - global_seqlen/balanced_max:29684 - global_seqlen/mean:24008.625 - actor/entropy:0.11019539088010788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948736071586609 - training/rollout_probs_diff_mean:0.003139072796329856 - training/rollout_probs_diff_std:0.012617148458957672 - training/rollout_actor_probs_pearson_corr:0.997302770614624 - rollout_corr/rollout_is_mean:1.0000996589660645 - rollout_corr/rollout_is_ratio_fraction_high:7.389344682451338e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.852459334069863e-05 - rollout_corr/rollout_is_max:4.887231826782227 - rollout_corr/rollout_is_min:0.26104938983917236 - rollout_corr/rollout_is_std:0.03467484563589096 - rollout_corr/rollout_is_eff_sample_size:0.9987993368491189 - rollout_corr/rollout_is_seq_mean:1.0002052783966064 - rollout_corr/rollout_is_seq_std:0.0023445014376193285 - rollout_corr/rollout_is_seq_max:1.0106254816055298 - rollout_corr/rollout_is_seq_min:0.99307781457901 - rollout_corr/rollout_is_seq_max_deviation:0.010625481605529785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1195416245609522) - rollout_corr/training_log_ppl:np.float64(0.1096741780929733) - rollout_corr/kl:np.float64(0.0007273018066071835) - rollout_corr/k3_kl:np.float64(0.0009105393462505162) - rollout_corr/rollout_ppl:np.float64(1.1187340491451323) - rollout_corr/rollout_log_ppl:np.float64(0.10894687551262905) - rollout_corr/log_ppl_diff:np.float64(0.0007273025803442579) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019026359186682384) - rollout_corr/log_ppl_diff_max:np.float64(0.022861547768115997) - rollout_corr/log_ppl_diff_min:np.float64(-0.007030755281448364) - rollout_corr/ppl_ratio:np.float64(1.000731429317966) - rollout_corr/chi2_token:np.float64(0.002295879879966378) - rollout_corr/chi2_seq:np.float64(2.1700741790700704) - actor/pg_loss:np.float64(8.732720743864775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011307528586712579) - actor/ppo_kl:np.float64(0.00020700877354862257) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.46738822013139725) - perf/mfu/actor:np.float64(0.06503294604815439) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.65916442871094) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01939600333571434 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01939600333571434 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.76953125 - response_length/max:8192.0 - response_length/min:69.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:475.76953125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001276358962059021 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8453756030648947) - timing_s/agent_loop/generate_sequences/max:np.float64(51.0958620775491) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.681928445308586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.0958620775491) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.71189962141216 - timing_s/reward:0.06205694004893303 - timing_s/old_log_prob:2.802142111584544 - timing_s/adv:0.1481334324926138 - timing_s/update_actor:10.832650827243924 - timing_s/step:70.63796724379063 - timing_s/stop_profile:0.00011838413774967194 - timing_per_token_ms/update_actor:0.05639978771818421 - timing_per_token_ms/gen:0.46562640805120126 - timing_per_token_ms/adv:0.0007712511258590079 - perf/total_num_tokens:192069 - perf/time_per_step:70.63796724379063 - perf/throughput:339.8827278981539 (TaskRunner pid=2049544) Training Progress: 11%|█ | 11/100 [07:58<1:32:18, 62.23s/it] (TaskRunner pid=2049544) step:12 - global_seqlen/min:18163 - global_seqlen/max:27095 - global_seqlen/minmax_diff:8932 - global_seqlen/balanced_min:23218 - global_seqlen/balanced_max:23238 - global_seqlen/mean:23234.125 - actor/entropy:0.11550334095954895 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4828122854232788 - training/rollout_probs_diff_mean:0.0033159777522087097 - training/rollout_probs_diff_std:0.012482362799346447 - training/rollout_actor_probs_pearson_corr:0.9973734021186829 - rollout_corr/rollout_is_mean:0.9999693036079407 - rollout_corr/rollout_is_ratio_fraction_high:5.898561721551232e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010111819574376568 - rollout_corr/rollout_is_max:2.5878732204437256 - rollout_corr/rollout_is_min:0.125954732298851 - rollout_corr/rollout_is_std:0.03412209078669548 - rollout_corr/rollout_is_eff_sample_size:0.9988369179946218 - rollout_corr/rollout_is_seq_mean:1.0000054836273193 - rollout_corr/rollout_is_seq_std:0.0017772279679775238 - rollout_corr/rollout_is_seq_max:1.008151888847351 - rollout_corr/rollout_is_seq_min:0.9952659010887146 - rollout_corr/rollout_is_seq_max_deviation:0.008151888847351074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1121750511229038) - rollout_corr/training_log_ppl:np.float64(0.10409450641964213) - rollout_corr/kl:np.float64(0.0006273885898906428) - rollout_corr/k3_kl:np.float64(0.000663624501786586) - rollout_corr/rollout_ppl:np.float64(1.1114575085230172) - rollout_corr/rollout_log_ppl:np.float64(0.10346711749298265) - rollout_corr/log_ppl_diff:np.float64(0.0006273889266594779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014779563243791927) - rollout_corr/log_ppl_diff_max:np.float64(0.006399750709533691) - rollout_corr/log_ppl_diff_min:np.float64(-0.006532169878482819) - rollout_corr/ppl_ratio:np.float64(1.000629240879789) - rollout_corr/chi2_token:np.float64(0.0014743746723979712) - rollout_corr/chi2_seq:np.float64(1.0935420689638704) - actor/pg_loss:np.float64(0.00015071697998791933) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004757110639275197) - actor/ppo_kl:np.float64(2.2654555305479107e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2736458256840706) - perf/mfu/actor:np.float64(0.0671891188671343) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.26230239868164) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0065000043250620365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0065000043250620365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:463.56640625 - response_length/max:1365.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.56640625 - response_length_non_aborted/max:1365.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.028241038322449e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0592438988387585) - timing_s/agent_loop/generate_sequences/max:np.float64(10.160819062963128) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.8243368406328955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.160819062963128) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1365 - timing_s/gen:15.931704020127654 - timing_s/reward:0.06278514675796032 - timing_s/old_log_prob:2.4699926115572453 - timing_s/adv:0.16309965774416924 - timing_s/update_actor:10.090087844058871 - timing_s/step:28.80461708456278 - timing_s/stop_profile:0.0001237243413925171 - timing_per_token_ms/update_actor:0.054284849569646326 - timing_per_token_ms/gen:0.13424876779155878 - timing_per_token_ms/adv:0.0008774790192452332 - perf/total_num_tokens:185873 - perf/time_per_step:28.80461708456278 - perf/throughput:806.6111391722626 (TaskRunner pid=2049544) Training Progress: 12%|█▏ | 12/100 [08:26<1:16:22, 52.08s/it] (TaskRunner pid=2049544) step:13 - global_seqlen/min:20734 - global_seqlen/max:28493 - global_seqlen/minmax_diff:7759 - global_seqlen/balanced_min:23965 - global_seqlen/balanced_max:23977 - global_seqlen/mean:23970.125 - actor/entropy:0.1322651505470276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5841533541679382 - training/rollout_probs_diff_mean:0.003578777192160487 - training/rollout_probs_diff_std:0.01281787734478712 - training/rollout_actor_probs_pearson_corr:0.997634768486023 - rollout_corr/rollout_is_mean:0.9998666048049927 - rollout_corr/rollout_is_ratio_fraction_high:8.307234566018451e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00010799404844874516 - rollout_corr/rollout_is_max:2.1130197048187256 - rollout_corr/rollout_is_min:0.289154976606369 - rollout_corr/rollout_is_std:0.03564285859465599 - rollout_corr/rollout_is_eff_sample_size:0.9987307228760026 - rollout_corr/rollout_is_seq_mean:0.9998607635498047 - rollout_corr/rollout_is_seq_std:0.0017514645587652922 - rollout_corr/rollout_is_seq_max:1.0048136711120605 - rollout_corr/rollout_is_seq_min:0.9948503971099854 - rollout_corr/rollout_is_seq_max_deviation:0.0051496028900146484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1441420195624232) - rollout_corr/training_log_ppl:np.float64(0.12973640582640655) - rollout_corr/kl:np.float64(0.0009200233825930049) - rollout_corr/k3_kl:np.float64(0.000771395374357553) - rollout_corr/rollout_ppl:np.float64(1.1430599559098482) - rollout_corr/rollout_log_ppl:np.float64(0.12881638239923632) - rollout_corr/log_ppl_diff:np.float64(0.0009200234271702357) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016153893011505716) - rollout_corr/log_ppl_diff_max:np.float64(0.010122627019882202) - rollout_corr/log_ppl_diff_min:np.float64(-0.007734514772891998) - rollout_corr/ppl_ratio:np.float64(1.0009225921239704) - rollout_corr/chi2_token:np.float64(0.0012212155852466822) - rollout_corr/chi2_seq:np.float64(0.878853014903143) - actor/pg_loss:np.float64(-4.01295255869627e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007891653215210681) - actor/ppo_kl:np.float64(0.00011588802720652325) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.40201497077941895) - perf/mfu/actor:np.float64(0.0691041251464859) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.31333541870117) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005807795561850071 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005807795561850071 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.22265625 - response_length/max:1333.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.22265625 - response_length_non_aborted/max:1333.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014058127999305725 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4221638720482588) - timing_s/agent_loop/generate_sequences/max:np.float64(9.880332147702575) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.85110625057132) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.880332147702575) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1333 - timing_s/gen:15.408700393512845 - timing_s/reward:0.062333619222044945 - timing_s/old_log_prob:2.5665055103600025 - timing_s/adv:0.1314475554972887 - timing_s/update_actor:10.066378388553858 - timing_s/step:28.321861196309328 - timing_s/stop_profile:0.00011406466364860535 - timing_per_token_ms/update_actor:0.05249439869709616 - timing_per_token_ms/gen:0.12800369168124182 - timing_per_token_ms/adv:0.0006854759596439772 - perf/total_num_tokens:191761 - perf/time_per_step:28.321861196309328 - perf/throughput:846.3470968187496 (TaskRunner pid=2049544) Training Progress: 13%|█▎ | 13/100 [08:55<1:05:06, 44.90s/it] (TaskRunner pid=2049544) step:14 - global_seqlen/min:17304 - global_seqlen/max:31074 - global_seqlen/minmax_diff:13770 - global_seqlen/balanced_min:23194 - global_seqlen/balanced_max:23602 - global_seqlen/mean:23258.875 - actor/entropy:0.11846989393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45436424016952515 - training/rollout_probs_diff_mean:0.003607378341257572 - training/rollout_probs_diff_std:0.013258584775030613 - training/rollout_actor_probs_pearson_corr:0.9971495866775513 - rollout_corr/rollout_is_mean:0.9998986721038818 - rollout_corr/rollout_is_ratio_fraction_high:5.1646224164869636e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014633095997851342 - rollout_corr/rollout_is_max:2.4299769401550293 - rollout_corr/rollout_is_min:0.09816896170377731 - rollout_corr/rollout_is_std:0.037202417850494385 - rollout_corr/rollout_is_eff_sample_size:0.9986175363719597 - rollout_corr/rollout_is_seq_mean:0.9998643398284912 - rollout_corr/rollout_is_seq_std:0.002263482892885804 - rollout_corr/rollout_is_seq_max:1.00752854347229 - rollout_corr/rollout_is_seq_min:0.9923214316368103 - rollout_corr/rollout_is_seq_max_deviation:0.007678568363189697 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.13216158375144) - rollout_corr/training_log_ppl:np.float64(0.12105548306863056) - rollout_corr/kl:np.float64(0.0011141863628356674) - rollout_corr/k3_kl:np.float64(0.0007862591612095571) - rollout_corr/rollout_ppl:np.float64(1.1308507332578301) - rollout_corr/rollout_log_ppl:np.float64(0.1199412965834199) - rollout_corr/log_ppl_diff:np.float64(0.0011141864852106664) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018289975014340598) - rollout_corr/log_ppl_diff_max:np.float64(0.01189817488193512) - rollout_corr/log_ppl_diff_min:np.float64(-0.006560362875461578) - rollout_corr/ppl_ratio:np.float64(1.0011174657847732) - rollout_corr/chi2_token:np.float64(0.0008673113770782948) - rollout_corr/chi2_seq:np.float64(0.27274479530751705) - actor/pg_loss:np.float64(8.668552618473768e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010782241606648313) - actor/ppo_kl:np.float64(0.00016768844109904535) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.42187249660491943) - perf/mfu/actor:np.float64(0.06630021670401551) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.41584777832031) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009940821677446365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009940821677446365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.80859375 - response_length/max:1963.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.80859375 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013650394976139069 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0465375203639269) - timing_s/agent_loop/generate_sequences/max:np.float64(12.73308464512229) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.650266676486353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.73308464512229) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1963 - timing_s/gen:18.954351030290127 - timing_s/reward:0.06252953223884106 - timing_s/old_log_prob:2.473697155714035 - timing_s/adv:0.14756855927407742 - timing_s/update_actor:9.925982998684049 - timing_s/step:31.649879788979888 - timing_s/stop_profile:0.00011733733117580414 - timing_per_token_ms/update_actor:0.05334513706426068 - timing_per_token_ms/gen:0.16315344119036046 - timing_per_token_ms/adv:0.0007930766173884023 - perf/total_num_tokens:186071 - perf/time_per_step:31.649879788979888 - perf/throughput:734.8803583165097 (TaskRunner pid=2049544) Training Progress: 14%|█▍ | 14/100 [09:26<58:38, 40.91s/it] (TaskRunner pid=2049544) step:15 - global_seqlen/min:15342 - global_seqlen/max:29838 - global_seqlen/minmax_diff:14496 - global_seqlen/balanced_min:23634 - global_seqlen/balanced_max:23672 - global_seqlen/mean:23657.75 - actor/entropy:0.1110260859131813 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9913476705551147 - training/rollout_probs_diff_mean:0.0030704238452017307 - training/rollout_probs_diff_std:0.01288246363401413 - training/rollout_actor_probs_pearson_corr:0.9973083138465881 - rollout_corr/rollout_is_mean:0.9999945163726807 - rollout_corr/rollout_is_ratio_fraction_high:3.303655466879718e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.085052442969754e-05 - rollout_corr/rollout_is_max:3.397390604019165 - rollout_corr/rollout_is_min:0.008578135631978512 - rollout_corr/rollout_is_std:0.03370378911495209 - rollout_corr/rollout_is_eff_sample_size:0.9988653435875123 - rollout_corr/rollout_is_seq_mean:0.9999395608901978 - rollout_corr/rollout_is_seq_std:0.0021285165566951036 - rollout_corr/rollout_is_seq_max:1.016195297241211 - rollout_corr/rollout_is_seq_min:0.9913737177848816 - rollout_corr/rollout_is_seq_max_deviation:0.016195297241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.116935707628727) - rollout_corr/training_log_ppl:np.float64(0.10718949149304535) - rollout_corr/kl:np.float64(0.0008361445555777891) - rollout_corr/k3_kl:np.float64(0.0006998892279455049) - rollout_corr/rollout_ppl:np.float64(1.1159698073752224) - rollout_corr/rollout_log_ppl:np.float64(0.10635334601101931) - rollout_corr/log_ppl_diff:np.float64(0.0008361454820260406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016929644480114803) - rollout_corr/log_ppl_diff_max:np.float64(0.01026906818151474) - rollout_corr/log_ppl_diff_min:np.float64(-0.012994319200515747) - rollout_corr/ppl_ratio:np.float64(1.000839062500745) - rollout_corr/chi2_token:np.float64(0.0011396266054362059) - rollout_corr/chi2_seq:np.float64(1.5898604474496096) - actor/pg_loss:np.float64(-2.7647125534713268e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006954610362299718) - actor/ppo_kl:np.float64(0.00011087026399003719) - actor/pg_clipfrac_lower:np.float64(1.0444518920849077e-05) - actor/grad_norm:np.float64(0.29682732000947) - perf/mfu/actor:np.float64(0.06723820699139785) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.3052020072937) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0049513536505401134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0049513536505401134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.9609375 - response_length/max:1790.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.9609375 - response_length_non_aborted/max:1790.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010203756392002106 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7883450742810965) - timing_s/agent_loop/generate_sequences/max:np.float64(12.01114228926599) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.683357301946671) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.01114228926599) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1790 - timing_s/gen:17.793763728812337 - timing_s/reward:0.06301617436110973 - timing_s/old_log_prob:2.4648690931499004 - timing_s/adv:0.15438295900821686 - timing_s/update_actor:10.23603860847652 - timing_s/step:30.79816811159253 - timing_s/stop_profile:0.00011844001710414886 - timing_per_token_ms/update_actor:0.05408396090327969 - timing_per_token_ms/gen:0.14696116328988204 - timing_per_token_ms/adv:0.0008157102799728253 - perf/total_num_tokens:189262 - perf/time_per_step:30.79816811159253 - perf/throughput:768.1544536765856 (TaskRunner pid=2049544) Training Progress: 15%|█▌ | 15/100 [09:57<53:39, 37.88s/it] (TaskRunner pid=2049544) step:16 - global_seqlen/min:18400 - global_seqlen/max:27667 - global_seqlen/minmax_diff:9267 - global_seqlen/balanced_min:22148 - global_seqlen/balanced_max:22154 - global_seqlen/mean:22151.125 - actor/entropy:0.12429671734571457 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.739967942237854 - training/rollout_probs_diff_mean:0.0033937261905521154 - training/rollout_probs_diff_std:0.012556870467960835 - training/rollout_actor_probs_pearson_corr:0.9975789189338684 - rollout_corr/rollout_is_mean:1.000154972076416 - rollout_corr/rollout_is_ratio_fraction_high:2.7087302441941574e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.126190368784592e-05 - rollout_corr/rollout_is_max:4.322726726531982 - rollout_corr/rollout_is_min:0.12595464289188385 - rollout_corr/rollout_is_std:0.034952208399772644 - rollout_corr/rollout_is_eff_sample_size:0.9987801905807853 - rollout_corr/rollout_is_seq_mean:1.0001484155654907 - rollout_corr/rollout_is_seq_std:0.0019322986481711268 - rollout_corr/rollout_is_seq_max:1.007652997970581 - rollout_corr/rollout_is_seq_min:0.992692768573761 - rollout_corr/rollout_is_seq_max_deviation:0.007652997970581055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.128569976426661) - rollout_corr/training_log_ppl:np.float64(0.11689590534660965) - rollout_corr/kl:np.float64(0.0006918034323035727) - rollout_corr/k3_kl:np.float64(0.0007205118381676812) - rollout_corr/rollout_ppl:np.float64(1.1277636121958494) - rollout_corr/rollout_log_ppl:np.float64(0.11620410031173378) - rollout_corr/log_ppl_diff:np.float64(0.0006918050348758698) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016668791504343972) - rollout_corr/log_ppl_diff_max:np.float64(0.011440090835094452) - rollout_corr/log_ppl_diff_min:np.float64(-0.005701005458831787) - rollout_corr/ppl_ratio:np.float64(1.000694399466738) - rollout_corr/chi2_token:np.float64(0.0014990693889558315) - rollout_corr/chi2_seq:np.float64(0.6086118693929166) - actor/pg_loss:np.float64(3.552588168531656e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008225217134167906) - actor/ppo_kl:np.float64(0.0001884127358140475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3684859052300453) - perf/mfu/actor:np.float64(0.05525549139638902) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(118.55257987976074) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014105712994933128 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014105712994933128 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:432.62890625 - response_length/max:1203.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:432.62890625 - response_length_non_aborted/max:1203.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.934604167938232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0808530896902084) - timing_s/agent_loop/generate_sequences/max:np.float64(8.854041801765561) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.365803626016714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.854041801765561) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1203 - timing_s/gen:14.700905803591013 - timing_s/reward:0.06069855950772762 - timing_s/old_log_prob:2.4508451614528894 - timing_s/adv:0.14457714557647705 - timing_s/update_actor:11.594925744459033 - timing_s/step:29.03880488872528 - timing_s/stop_profile:0.00011275149881839752 - timing_per_token_ms/update_actor:0.0654307949622143 - timing_per_token_ms/gen:0.13273596023214732 - timing_per_token_ms/adv:0.0008158566753182798 - perf/total_num_tokens:177209 - perf/time_per_step:29.03880488872528 - perf/throughput:762.8111792093924 (TaskRunner pid=2049544) Training Progress: 16%|█▌ | 16/100 [10:26<49:20, 35.25s/it] (TaskRunner pid=2049544) step:17 - global_seqlen/min:17199 - global_seqlen/max:28844 - global_seqlen/minmax_diff:11645 - global_seqlen/balanced_min:22404 - global_seqlen/balanced_max:28744 - global_seqlen/mean:23204.125 - actor/entropy:0.12878382205963135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7316876649856567 - training/rollout_probs_diff_mean:0.003660086076706648 - training/rollout_probs_diff_std:0.013964627869427204 - training/rollout_actor_probs_pearson_corr:0.9971086978912354 - rollout_corr/rollout_is_mean:0.9999353289604187 - rollout_corr/rollout_is_ratio_fraction_high:5.108600453240797e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.000221372683881782 - rollout_corr/rollout_is_max:3.039975881576538 - rollout_corr/rollout_is_min:0.002080337144434452 - rollout_corr/rollout_is_std:0.037413306534290314 - rollout_corr/rollout_is_eff_sample_size:0.9986019633462979 - rollout_corr/rollout_is_seq_mean:0.9999227523803711 - rollout_corr/rollout_is_seq_std:0.0023962773848325014 - rollout_corr/rollout_is_seq_max:1.0085341930389404 - rollout_corr/rollout_is_seq_min:0.9915060997009277 - rollout_corr/rollout_is_seq_max_deviation:0.00853419303894043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1416860399767756) - rollout_corr/training_log_ppl:np.float64(0.1297323260405392) - rollout_corr/kl:np.float64(0.0012029753426432421) - rollout_corr/k3_kl:np.float64(0.0010000968625991646) - rollout_corr/rollout_ppl:np.float64(1.1402671793475747) - rollout_corr/rollout_log_ppl:np.float64(0.12852934912734781) - rollout_corr/log_ppl_diff:np.float64(0.0012029769131913781) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022540156060131267) - rollout_corr/log_ppl_diff_max:np.float64(0.022689037024974823) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066199153661727905) - rollout_corr/ppl_ratio:np.float64(1.0012089894153178) - rollout_corr/chi2_token:np.float64(0.0014632882084697485) - rollout_corr/chi2_seq:np.float64(0.5507866519037634) - actor/pg_loss:np.float64(0.0002590372459962964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013125582909196964) - actor/ppo_kl:np.float64(0.00025115833264877097) - actor/pg_clipfrac_lower:np.float64(7.70463520893827e-06) - actor/grad_norm:np.float64(0.3952488452196121) - perf/mfu/actor:np.float64(0.06094650606696392) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(110.15214920043945) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022923780605196953 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022923780605196953 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.78515625 - response_length/max:8192.0 - response_length/min:68.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:458.78515625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017055682837963104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8617520779371262) - timing_s/agent_loop/generate_sequences/max:np.float64(50.32327074185014) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.232911160150252) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.32327074185014) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.11839736439288 - timing_s/reward:0.06247155740857124 - timing_s/old_log_prob:2.831888886168599 - timing_s/adv:0.1662355735898018 - timing_s/update_actor:10.9862146563828 - timing_s/step:71.25081997551024 - timing_s/stop_profile:0.00011354684829711914 - timing_per_token_ms/update_actor:0.05918244415800423 - timing_per_token_ms/gen:0.4863251059131442 - timing_per_token_ms/adv:0.0008955065833650363 - perf/total_num_tokens:185633 - perf/time_per_step:71.25081997551024 - perf/throughput:325.6681818956685 (TaskRunner pid=2049544) Training Progress: 17%|█▋ | 17/100 [11:38<1:03:45, 46.09s/it] (TaskRunner pid=2049544) step:18 - global_seqlen/min:17313 - global_seqlen/max:30331 - global_seqlen/minmax_diff:13018 - global_seqlen/balanced_min:25566 - global_seqlen/balanced_max:25574 - global_seqlen/mean:25570.75 - actor/entropy:0.1224943995475769 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5235333442687988 - training/rollout_probs_diff_mean:0.0032062120735645294 - training/rollout_probs_diff_std:0.012459194287657738 - training/rollout_actor_probs_pearson_corr:0.9975793361663818 - rollout_corr/rollout_is_mean:0.9999059438705444 - rollout_corr/rollout_is_ratio_fraction_high:7.298523996723816e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012407491158228368 - rollout_corr/rollout_is_max:2.7950527667999268 - rollout_corr/rollout_is_min:0.06372947990894318 - rollout_corr/rollout_is_std:0.03364891931414604 - rollout_corr/rollout_is_eff_sample_size:0.9988689117678543 - rollout_corr/rollout_is_seq_mean:0.9998814463615417 - rollout_corr/rollout_is_seq_std:0.0018376336665824056 - rollout_corr/rollout_is_seq_max:1.007022738456726 - rollout_corr/rollout_is_seq_min:0.9937419891357422 - rollout_corr/rollout_is_seq_max_deviation:0.007022738456726074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1279504764825106) - rollout_corr/training_log_ppl:np.float64(0.11757075608329615) - rollout_corr/kl:np.float64(0.0008173328948331005) - rollout_corr/k3_kl:np.float64(0.0006817538003787149) - rollout_corr/rollout_ppl:np.float64(1.1270086439326406) - rollout_corr/rollout_log_ppl:np.float64(0.1167534222076938) - rollout_corr/log_ppl_diff:np.float64(0.000817333875602344) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016419640196545515) - rollout_corr/log_ppl_diff_max:np.float64(0.008472427725791931) - rollout_corr/log_ppl_diff_min:np.float64(-0.006251677870750427) - rollout_corr/ppl_ratio:np.float64(1.0008197950664908) - rollout_corr/chi2_token:np.float64(0.0010528604034334421) - rollout_corr/chi2_seq:np.float64(0.8530804882757366) - actor/pg_loss:np.float64(0.0001363652991130948) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009771198135695158) - actor/ppo_kl:np.float64(8.36708865672442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4684727042913437) - perf/mfu/actor:np.float64(0.0710728954199909) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.35284042358398) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004924679175019264 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004924679175019264 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:535.2109375 - response_length/max:1581.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:535.2109375 - response_length_non_aborted/max:1581.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012665800750255585 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8409140948206186) - timing_s/agent_loop/generate_sequences/max:np.float64(11.426093036308885) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7903019126315485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.426093036308885) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1581 - timing_s/gen:17.257411571219563 - timing_s/reward:0.06468412093818188 - timing_s/old_log_prob:2.512243326753378 - timing_s/adv:0.16402030177414417 - timing_s/update_actor:10.535967061296105 - timing_s/step:30.621809538453817 - timing_s/stop_profile:0.00020102225244045258 - timing_per_token_ms/update_actor:0.05150399900910271 - timing_per_token_ms/gen:0.1259536366445733 - timing_per_token_ms/adv:0.000801796494892329 - perf/total_num_tokens:204566 - perf/time_per_step:30.621809538453817 - perf/throughput:835.0502594527971 (TaskRunner pid=2049544) Training Progress: 18%|█▊ | 18/100 [12:08<56:39, 41.46s/it] (TaskRunner pid=2049544) step:19 - global_seqlen/min:17713 - global_seqlen/max:31095 - global_seqlen/minmax_diff:13382 - global_seqlen/balanced_min:23780 - global_seqlen/balanced_max:23986 - global_seqlen/mean:23818.0 - actor/entropy:0.10652168840169907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43985795974731445 - training/rollout_probs_diff_mean:0.003031388856470585 - training/rollout_probs_diff_std:0.012130273506045341 - training/rollout_actor_probs_pearson_corr:0.9975038766860962 - rollout_corr/rollout_is_mean:1.000024437904358 - rollout_corr/rollout_is_ratio_fraction_high:5.6943903473438695e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010575295891612768 - rollout_corr/rollout_is_max:3.544985771179199 - rollout_corr/rollout_is_min:0.2985078692436218 - rollout_corr/rollout_is_std:0.03386257588863373 - rollout_corr/rollout_is_eff_sample_size:0.9988546391994405 - rollout_corr/rollout_is_seq_mean:1.0000839233398438 - rollout_corr/rollout_is_seq_std:0.002387973479926586 - rollout_corr/rollout_is_seq_max:1.0142525434494019 - rollout_corr/rollout_is_seq_min:0.9915294051170349 - rollout_corr/rollout_is_seq_max_deviation:0.014252543449401855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1093668043613434) - rollout_corr/training_log_ppl:np.float64(0.10132705738578807) - rollout_corr/kl:np.float64(0.0006060766052620181) - rollout_corr/k3_kl:np.float64(0.0007317086579945453) - rollout_corr/rollout_ppl:np.float64(1.1086922818794847) - rollout_corr/rollout_log_ppl:np.float64(0.10072098037198884) - rollout_corr/log_ppl_diff:np.float64(0.0006060770137992222) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016289143422909547) - rollout_corr/log_ppl_diff_max:np.float64(0.014358222484588623) - rollout_corr/log_ppl_diff_min:np.float64(-0.009524352848529816) - rollout_corr/ppl_ratio:np.float64(1.0006090987008065) - rollout_corr/chi2_token:np.float64(0.001838616793975234) - rollout_corr/chi2_seq:np.float64(0.987432076362893) - actor/pg_loss:np.float64(0.0002443152479827404) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007472785102891066) - actor/ppo_kl:np.float64(-8.193314219084868e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4798855558037758) - perf/mfu/actor:np.float64(0.0686439488762442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.47739028930664) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009253384196199477 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009253384196199477 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.1875 - response_length/max:1871.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.1875 - response_length_non_aborted/max:1871.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001460295170545578 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0999032072722912) - timing_s/agent_loop/generate_sequences/max:np.float64(12.776574049144983) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.886515708894876) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.776574049144983) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:1871 - timing_s/gen:18.46205241046846 - timing_s/reward:0.06260044127702713 - timing_s/old_log_prob:2.540384005755186 - timing_s/adv:0.14328701607882977 - timing_s/update_actor:10.04507009498775 - timing_s/step:31.340227657929063 - timing_s/stop_profile:0.00011619366705417633 - timing_per_token_ms/update_actor:0.05271785044392765 - timing_per_token_ms/gen:0.15018590077499397 - timing_per_token_ms/adv:0.0007519891262848989 - perf/total_num_tokens:190544 - perf/time_per_step:31.340227657929063 - perf/throughput:759.9817161498524 (TaskRunner pid=2049544) Training Progress: 19%|█▉ | 19/100 [12:40<51:52, 38.43s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are asked to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2049544) step:20 - global_seqlen/min:17554 - global_seqlen/max:28214 - global_seqlen/minmax_diff:10660 - global_seqlen/balanced_min:22414 - global_seqlen/balanced_max:22431 - global_seqlen/mean:22418.0 - actor/entropy:0.11887378990650177 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7652756571769714 - training/rollout_probs_diff_mean:0.003253082511946559 - training/rollout_probs_diff_std:0.01256453339010477 - training/rollout_actor_probs_pearson_corr:0.9974572062492371 - rollout_corr/rollout_is_mean:0.9998491406440735 - rollout_corr/rollout_is_ratio_fraction_high:2.658820267242845e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014180374273564667 - rollout_corr/rollout_is_max:2.1670210361480713 - rollout_corr/rollout_is_min:0.20142976939678192 - rollout_corr/rollout_is_std:0.03435710072517395 - rollout_corr/rollout_is_eff_sample_size:0.9988207434840839 - rollout_corr/rollout_is_seq_mean:0.9999091625213623 - rollout_corr/rollout_is_seq_std:0.001986477058380842 - rollout_corr/rollout_is_seq_max:1.0120420455932617 - rollout_corr/rollout_is_seq_min:0.9940481185913086 - rollout_corr/rollout_is_seq_max_deviation:0.012042045593261719 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1164336544461548) - rollout_corr/training_log_ppl:np.float64(0.10743120299957809) - rollout_corr/kl:np.float64(0.0007887022689292422) - rollout_corr/k3_kl:np.float64(0.0007427555091403804) - rollout_corr/rollout_ppl:np.float64(1.1155504160560668) - rollout_corr/rollout_log_ppl:np.float64(0.1066425001481548) - rollout_corr/log_ppl_diff:np.float64(0.0007887028514232952) - rollout_corr/log_ppl_abs_diff:np.float64(0.001653945950238267) - rollout_corr/log_ppl_diff_max:np.float64(0.012162841856479645) - rollout_corr/log_ppl_diff_min:np.float64(-0.010080903768539429) - rollout_corr/ppl_ratio:np.float64(1.0007913769222796) - rollout_corr/chi2_token:np.float64(0.0014412584714591503) - rollout_corr/chi2_seq:np.float64(0.749183313222602) - actor/pg_loss:np.float64(0.00014212285168468952) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000699050789080502) - actor/ppo_kl:np.float64(5.158489911716302e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37614476680755615) - perf/mfu/actor:np.float64(0.06503999971026932) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.41239166259766) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09981662846229708) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8320624999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04145535225486681) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6367624999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0671080518080989) - val-aux/sciknoweval/score/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/score/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09981662846229708) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8320624999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04145535225486681) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6367624999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0671080518080989) - val-core/sciknoweval/acc/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09981662846229708) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8320624999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04145535225486681) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6367624999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0671080518080989) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.990625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.030556568705962538) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9989750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010087512303978256) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9817375) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.040950720392224096) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.990325) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03109914326301299) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9670625000000002) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.050240658088205936) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99635) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01916044898238421) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9449) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05467743751202757) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999125) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008915794300729528) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.918325) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.047861241896262015) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999925) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0016371701590179605) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00229323236271739 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00229323236271739 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.75 - response_length/max:1682.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.75 - response_length_non_aborted/max:1682.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.746236562728882e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.339398443698883) - timing_s/agent_loop/generate_sequences/max:np.float64(11.241939969360828) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.378861661745759) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.241939969360828) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:1682 - timing_s/gen:17.086866164579988 - timing_s/reward:0.061425795778632164 - timing_s/old_log_prob:2.430392887443304 - timing_s/adv:0.2020460832864046 - timing_s/update_actor:9.794211328029633 - timing_s/step:29.6614922080189 - timing_s/testing:97.09009108878672 - timing_s/save_checkpoint:7.344661565497518 - timing_s/stop_profile:0.00012345612049102783 - timing_per_token_ms/update_actor:0.054611313052177 - timing_per_token_ms/gen:0.15143634930321173 - timing_per_token_ms/adv:0.001126584013328601 - perf/total_num_tokens:179344 - perf/time_per_step:29.6614922080189 - perf/throughput:755.7947470336425 (TaskRunner pid=2049544) Training Progress: 20%|██ | 20/100 [14:54<1:29:33, 67.17s/it] (TaskRunner pid=2049544) step:21 - global_seqlen/min:20890 - global_seqlen/max:29372 - global_seqlen/minmax_diff:8482 - global_seqlen/balanced_min:23784 - global_seqlen/balanced_max:24194 - global_seqlen/mean:23842.25 - actor/entropy:0.10979560017585754 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417163848876953 - training/rollout_probs_diff_mean:0.0029196166433393955 - training/rollout_probs_diff_std:0.01234018336981535 - training/rollout_actor_probs_pearson_corr:0.9974682927131653 - rollout_corr/rollout_is_mean:1.0000228881835938 - rollout_corr/rollout_is_ratio_fraction_high:1.6375992345274426e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010644395661074668 - rollout_corr/rollout_is_max:2.094970703125 - rollout_corr/rollout_is_min:0.07465586066246033 - rollout_corr/rollout_is_std:0.03303755819797516 - rollout_corr/rollout_is_eff_sample_size:0.9989098287246259 - rollout_corr/rollout_is_seq_mean:0.9999141693115234 - rollout_corr/rollout_is_seq_std:0.0017565247835591435 - rollout_corr/rollout_is_seq_max:1.0051144361495972 - rollout_corr/rollout_is_seq_min:0.9947226643562317 - rollout_corr/rollout_is_seq_max_deviation:0.0052773356437683105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1051466539502144) - rollout_corr/training_log_ppl:np.float64(0.09741823082003975) - rollout_corr/kl:np.float64(0.0006470407322218819) - rollout_corr/k3_kl:np.float64(0.0006291852859376945) - rollout_corr/rollout_ppl:np.float64(1.1044287369586527) - rollout_corr/rollout_log_ppl:np.float64(0.09677118871331913) - rollout_corr/log_ppl_diff:np.float64(0.0006470421067206189) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014577525143977255) - rollout_corr/log_ppl_diff_max:np.float64(0.009566932916641235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005102425813674927) - rollout_corr/ppl_ratio:np.float64(1.0006491616368294) - rollout_corr/chi2_token:np.float64(0.0012274319306015968) - rollout_corr/chi2_seq:np.float64(1.2051882504019886) - actor/pg_loss:np.float64(0.00013444910291582346) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045100462170921674) - actor/ppo_kl:np.float64(-5.319785113400144e-05) - actor/pg_clipfrac_lower:np.float64(2.38095403801708e-05) - actor/grad_norm:np.float64(0.2096654698252678) - perf/mfu/actor:np.float64(0.0669203788543201) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66231536865234) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004104233346879482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004104233346879482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.0703125 - response_length/max:2091.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.0703125 - response_length_non_aborted/max:2091.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015165284276008606 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7176804710179567) - timing_s/agent_loop/generate_sequences/max:np.float64(12.88374364003539) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.737000476088724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.88374364003539) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:2091 - timing_s/gen:19.051039807498455 - timing_s/reward:0.05670631118118763 - timing_s/old_log_prob:2.6326032262295485 - timing_s/adv:0.14211471565067768 - timing_s/update_actor:10.165599260479212 - timing_s/step:32.148173009976745 - timing_s/stop_profile:0.000135812908411026 - timing_per_token_ms/update_actor:0.053296140572299235 - timing_per_token_ms/gen:0.1559898453082654 - timing_per_token_ms/adv:0.000745078147252659 - perf/total_num_tokens:190738 - perf/time_per_step:32.148173009976745 - perf/throughput:741.6362352100347 (TaskRunner pid=2049544) Training Progress: 21%|██ | 21/100 [15:26<1:14:37, 56.68s/it] (TaskRunner pid=2049544) step:22 - global_seqlen/min:16900 - global_seqlen/max:32996 - global_seqlen/minmax_diff:16096 - global_seqlen/balanced_min:23691 - global_seqlen/balanced_max:23913 - global_seqlen/mean:23732.375 - actor/entropy:0.12188427150249481 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5312178134918213 - training/rollout_probs_diff_mean:0.0033468599431216717 - training/rollout_probs_diff_std:0.013181475922465324 - training/rollout_actor_probs_pearson_corr:0.9972920417785645 - rollout_corr/rollout_is_mean:0.999908983707428 - rollout_corr/rollout_is_ratio_fraction_high:2.4285402105306275e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001942832168424502 - rollout_corr/rollout_is_max:3.412278175354004 - rollout_corr/rollout_is_min:0.14518168568611145 - rollout_corr/rollout_is_std:0.03405214846134186 - rollout_corr/rollout_is_eff_sample_size:0.9988416753032981 - rollout_corr/rollout_is_seq_mean:0.999866247177124 - rollout_corr/rollout_is_seq_std:0.00229041394777596 - rollout_corr/rollout_is_seq_max:1.0077131986618042 - rollout_corr/rollout_is_seq_min:0.9842400550842285 - rollout_corr/rollout_is_seq_max_deviation:0.015759944915771484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1243417421355844) - rollout_corr/training_log_ppl:np.float64(0.11367632360634161) - rollout_corr/kl:np.float64(0.0010524871724442164) - rollout_corr/k3_kl:np.float64(0.000926966261786788) - rollout_corr/rollout_ppl:np.float64(1.1231554276309907) - rollout_corr/rollout_log_ppl:np.float64(0.11262383543362375) - rollout_corr/log_ppl_diff:np.float64(0.001052488172717858) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018730249648797326) - rollout_corr/log_ppl_diff_max:np.float64(0.0278562530875206) - rollout_corr/log_ppl_diff_min:np.float64(-0.004908844828605652) - rollout_corr/ppl_ratio:np.float64(1.0010584215633571) - rollout_corr/chi2_token:np.float64(0.0015142448246479034) - rollout_corr/chi2_seq:np.float64(0.5919819474220276) - actor/pg_loss:np.float64(-3.6185141652822495e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007750144300189277) - actor/ppo_kl:np.float64(0.00017040614556229627) - actor/pg_clipfrac_lower:np.float64(5.271592272038106e-06) - actor/grad_norm:np.float64(0.4317408576607704) - perf/mfu/actor:np.float64(0.06862934905562151) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66873407363892) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038765573408454657 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038765573408454657 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:482.54296875 - response_length/max:1955.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:482.54296875 - response_length_non_aborted/max:1955.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013677775859832764 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3632201459258795) - timing_s/agent_loop/generate_sequences/max:np.float64(12.304583990946412) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.897737120372767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.304583990946412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1955 - timing_s/gen:18.77624169550836 - timing_s/reward:0.054932184517383575 - timing_s/old_log_prob:2.5277608297765255 - timing_s/adv:0.14852365665137768 - timing_s/update_actor:10.204811798408628 - timing_s/step:31.823464412242174 - timing_s/stop_profile:0.0001289471983909607 - timing_per_token_ms/update_actor:0.05374942351117739 - timing_per_token_ms/gen:0.15199619282211235 - timing_per_token_ms/adv:0.0007822839931284673 - perf/total_num_tokens:189859 - perf/time_per_step:31.823464412242174 - perf/throughput:745.7508300344066 (TaskRunner pid=2049544) Training Progress: 22%|██▏ | 22/100 [15:58<1:04:00, 49.24s/it] (TaskRunner pid=2049544) step:23 - global_seqlen/min:17606 - global_seqlen/max:33708 - global_seqlen/minmax_diff:16102 - global_seqlen/balanced_min:26856 - global_seqlen/balanced_max:26886 - global_seqlen/mean:26877.375 - actor/entropy:0.1653793752193451 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9992972612380981 - training/rollout_probs_diff_mean:0.003593530971556902 - training/rollout_probs_diff_std:0.01221021730452776 - training/rollout_actor_probs_pearson_corr:0.9982529878616333 - rollout_corr/rollout_is_mean:1.0001059770584106 - rollout_corr/rollout_is_ratio_fraction_high:2.7500669602886774e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.18765116087161e-05 - rollout_corr/rollout_is_max:3.486543655395508 - rollout_corr/rollout_is_min:0.00043065936188213527 - rollout_corr/rollout_is_std:0.034376177936792374 - rollout_corr/rollout_is_eff_sample_size:0.9988199109866871 - rollout_corr/rollout_is_seq_mean:1.0000687837600708 - rollout_corr/rollout_is_seq_std:0.0017359366174787283 - rollout_corr/rollout_is_seq_max:1.0058130025863647 - rollout_corr/rollout_is_seq_min:0.9936341047286987 - rollout_corr/rollout_is_seq_max_deviation:0.0063658952713012695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1735385786741972) - rollout_corr/training_log_ppl:np.float64(0.1555803755109082) - rollout_corr/kl:np.float64(0.0008435647158400172) - rollout_corr/k3_kl:np.float64(0.000810335308230492) - rollout_corr/rollout_ppl:np.float64(1.172548221424222) - rollout_corr/rollout_log_ppl:np.float64(0.15473681054572808) - rollout_corr/log_ppl_diff:np.float64(0.0008435649651801214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016919265181059018) - rollout_corr/log_ppl_diff_max:np.float64(0.013718657195568085) - rollout_corr/log_ppl_diff_min:np.float64(-0.005128346383571625) - rollout_corr/ppl_ratio:np.float64(1.000846435315907) - rollout_corr/chi2_token:np.float64(0.001545611536130309) - rollout_corr/chi2_seq:np.float64(0.7387706565205008) - actor/pg_loss:np.float64(-7.304479368031025e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014301339112989808) - actor/ppo_kl:np.float64(0.00022386275868435668) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4195772558450699) - perf/mfu/actor:np.float64(0.07791174639008124) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.34968948364258) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014247066341340542 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014247066341340542 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.16796875 - response_length/max:1852.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.16796875 - response_length_non_aborted/max:1852.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003154464066028595 - timing_s/agent_loop/generate_sequences/min:np.float64(1.197395345196128) - timing_s/agent_loop/generate_sequences/max:np.float64(13.042586963623762) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.787129377022211) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.042586963623762) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1852 - timing_s/gen:18.867352159693837 - timing_s/reward:0.06311722472310066 - timing_s/old_log_prob:2.6495919916778803 - timing_s/adv:0.15099932067096233 - timing_s/update_actor:9.910499073565006 - timing_s/step:31.743287844583392 - timing_s/stop_profile:0.00010893121361732483 - timing_per_token_ms/update_actor:0.046091271346090375 - timing_per_token_ms/gen:0.12971620793046343 - timing_per_token_ms/adv:0.0007022603615074126 - perf/total_num_tokens:215019 - perf/time_per_step:31.743287844583392 - perf/throughput:846.7104961399359 (TaskRunner pid=2049544) Training Progress: 23%|██▎ | 23/100 [16:30<56:36, 44.11s/it] (TaskRunner pid=2049544) step:24 - global_seqlen/min:18041 - global_seqlen/max:30779 - global_seqlen/minmax_diff:12738 - global_seqlen/balanced_min:24022 - global_seqlen/balanced_max:24069 - global_seqlen/mean:24056.125 - actor/entropy:0.143111914396286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8866641521453857 - training/rollout_probs_diff_mean:0.003361048409715295 - training/rollout_probs_diff_std:0.012565950863063335 - training/rollout_actor_probs_pearson_corr:0.9978638887405396 - rollout_corr/rollout_is_mean:1.0000112056732178 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013656703231390566 - rollout_corr/rollout_is_max:1.9401601552963257 - rollout_corr/rollout_is_min:0.01988193951547146 - rollout_corr/rollout_is_std:0.03432759642601013 - rollout_corr/rollout_is_eff_sample_size:0.9988230031268704 - rollout_corr/rollout_is_seq_mean:1.000041127204895 - rollout_corr/rollout_is_seq_std:0.0024783890694379807 - rollout_corr/rollout_is_seq_max:1.0213310718536377 - rollout_corr/rollout_is_seq_min:0.9930093884468079 - rollout_corr/rollout_is_seq_max_deviation:0.021331071853637695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.144452238921076) - rollout_corr/training_log_ppl:np.float64(0.13059414884628495) - rollout_corr/kl:np.float64(0.00082983940849779) - rollout_corr/k3_kl:np.float64(0.0008679421140129762) - rollout_corr/rollout_ppl:np.float64(1.1434855586849153) - rollout_corr/rollout_log_ppl:np.float64(0.12976430927665206) - rollout_corr/log_ppl_diff:np.float64(0.0008298395696328953) - rollout_corr/log_ppl_abs_diff:np.float64(0.001792395327356644) - rollout_corr/log_ppl_diff_max:np.float64(0.01518431305885315) - rollout_corr/log_ppl_diff_min:np.float64(-0.016911908984184265) - rollout_corr/ppl_ratio:np.float64(1.000833987724036) - rollout_corr/chi2_token:np.float64(0.0022546271793544292) - rollout_corr/chi2_seq:np.float64(1.743649244075641) - actor/pg_loss:np.float64(0.00021742633543908596) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007519598264025262) - actor/ppo_kl:np.float64(9.309058857098762e-05) - actor/pg_clipfrac_lower:np.float64(5.556543328566477e-06) - actor/grad_norm:np.float64(0.3718913719058037) - perf/mfu/actor:np.float64(0.06935155745712235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.33251953125) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010280685499310493 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010280685499310493 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:486.25390625 - response_length/max:1963.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:486.25390625 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013114511966705322 - timing_s/agent_loop/generate_sequences/min:np.float64(1.162827044725418) - timing_s/agent_loop/generate_sequences/max:np.float64(12.901960568502545) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.848180819972185) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.901960568502545) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:257 - timing_s/agent_loop/slowest/response_length:1963 - timing_s/gen:18.82960532233119 - timing_s/reward:0.0641421228647232 - timing_s/old_log_prob:2.5491561722010374 - timing_s/adv:0.1262466385960579 - timing_s/update_actor:10.136537404730916 - timing_s/step:31.79429466277361 - timing_s/stop_profile:0.00012167543172836304 - timing_per_token_ms/update_actor:0.052671291639504056 - timing_per_token_ms/gen:0.15126489442028254 - timing_per_token_ms/adv:0.0006560004915383186 - perf/total_num_tokens:192449 - perf/time_per_step:31.79429466277361 - perf/throughput:756.6176653752331 (TaskRunner pid=2049544) Training Progress: 24%|██▍ | 24/100 [17:02<51:12, 40.43s/it] (TaskRunner pid=2049544) step:25 - global_seqlen/min:17209 - global_seqlen/max:32556 - global_seqlen/minmax_diff:15347 - global_seqlen/balanced_min:25066 - global_seqlen/balanced_max:25097 - global_seqlen/mean:25084.5 - actor/entropy:0.16078905761241913 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7048856616020203 - training/rollout_probs_diff_mean:0.003612553235143423 - training/rollout_probs_diff_std:0.01245187222957611 - training/rollout_actor_probs_pearson_corr:0.9980130791664124 - rollout_corr/rollout_is_mean:1.0001463890075684 - rollout_corr/rollout_is_ratio_fraction_high:3.095304418820888e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.73826104705222e-05 - rollout_corr/rollout_is_max:2.4638538360595703 - rollout_corr/rollout_is_min:0.07782533764839172 - rollout_corr/rollout_is_std:0.03481380268931389 - rollout_corr/rollout_is_eff_sample_size:0.9987899420046099 - rollout_corr/rollout_is_seq_mean:1.0001909732818604 - rollout_corr/rollout_is_seq_std:0.0020113997161388397 - rollout_corr/rollout_is_seq_max:1.0118286609649658 - rollout_corr/rollout_is_seq_min:0.9921811819076538 - rollout_corr/rollout_is_seq_max_deviation:0.01182866096496582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1733281780034304) - rollout_corr/training_log_ppl:np.float64(0.15433276955445763) - rollout_corr/kl:np.float64(0.0005177396068951623) - rollout_corr/k3_kl:np.float64(0.0007405701547043009) - rollout_corr/rollout_ppl:np.float64(1.172693099360913) - rollout_corr/rollout_log_ppl:np.float64(0.15381502895615995) - rollout_corr/log_ppl_diff:np.float64(0.0005177405982976779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015419888804899529) - rollout_corr/log_ppl_diff_max:np.float64(0.017087146639823914) - rollout_corr/log_ppl_diff_min:np.float64(-0.010565042495727539) - rollout_corr/ppl_ratio:np.float64(1.0005205699708313) - rollout_corr/chi2_token:np.float64(0.0020056585781276226) - rollout_corr/chi2_seq:np.float64(0.9553266488946974) - actor/pg_loss:np.float64(7.516134064644575e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005029199624004832) - actor/ppo_kl:np.float64(6.852521258160493e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3206869065761566) - perf/mfu/actor:np.float64(0.07269379031279649) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.39129257202148) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005469983443617821 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005469983443617821 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:504.796875 - response_length/max:1635.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:504.796875 - response_length_non_aborted/max:1635.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001018233597278595 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8177736550569534) - timing_s/agent_loop/generate_sequences/max:np.float64(11.67398145236075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.060982199414866) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.67398145236075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:335 - timing_s/agent_loop/slowest/response_length:1635 - timing_s/gen:17.524463137611747 - timing_s/reward:0.05791650153696537 - timing_s/old_log_prob:2.436915224418044 - timing_s/adv:0.13883326761424541 - timing_s/update_actor:9.980426739901304 - timing_s/step:30.245057743042707 - timing_s/stop_profile:0.00012283958494663239 - timing_per_token_ms/update_actor:0.049734032669085014 - timing_per_token_ms/gen:0.13560887065970026 - timing_per_token_ms/adv:0.0006918279595678876 - perf/total_num_tokens:200676 - perf/time_per_step:30.245057743042707 - perf/throughput:829.3751730651004 (TaskRunner pid=2049544) Training Progress: 25%|██▌ | 25/100 [17:32<46:43, 37.39s/it] (TaskRunner pid=2049544) (TaskRunner pid=2049544) step:26 - global_seqlen/min:19588 - global_seqlen/max:26583 - global_seqlen/minmax_diff:6995 - global_seqlen/balanced_min:22329 - global_seqlen/balanced_max:22375 - global_seqlen/mean:22358.5 - actor/entropy:0.14037299156188965 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3580157160758972 - training/rollout_probs_diff_mean:0.003176919650286436 - training/rollout_probs_diff_std:0.011454894207417965 - training/rollout_actor_probs_pearson_corr:0.9981624484062195 - rollout_corr/rollout_is_mean:1.0003305673599243 - rollout_corr/rollout_is_ratio_fraction_high:2.6683745090849698e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6683745090849698e-05 - rollout_corr/rollout_is_max:3.044771909713745 - rollout_corr/rollout_is_min:0.3202765882015228 - rollout_corr/rollout_is_std:0.03229307755827904 - rollout_corr/rollout_is_eff_sample_size:0.9989589572900974 - rollout_corr/rollout_is_seq_mean:1.0003631114959717 - rollout_corr/rollout_is_seq_std:0.001997264800593257 - rollout_corr/rollout_is_seq_max:1.009385347366333 - rollout_corr/rollout_is_seq_min:0.9931842684745789 - rollout_corr/rollout_is_seq_max_deviation:0.009385347366333008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.137321101501584) - rollout_corr/training_log_ppl:np.float64(0.12318833442259347) - rollout_corr/kl:np.float64(0.00036465673569097135) - rollout_corr/k3_kl:np.float64(0.0006004141492397252) - rollout_corr/rollout_ppl:np.float64(1.136900127865374) - rollout_corr/rollout_log_ppl:np.float64(0.12282367694933782) - rollout_corr/log_ppl_diff:np.float64(0.00036465747325564735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014688569026475307) - rollout_corr/log_ppl_diff_max:np.float64(0.011015359312295914) - rollout_corr/log_ppl_diff_min:np.float64(-0.0049578845500946045) - rollout_corr/ppl_ratio:np.float64(1.0003668149001896) - rollout_corr/chi2_token:np.float64(0.001733903307467699) - rollout_corr/chi2_seq:np.float64(0.5194167881272733) - actor/pg_loss:np.float64(8.537911344319582e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003864719383273041) - actor/ppo_kl:np.float64(0.00016390427055512635) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1358119323849678) - perf/mfu/actor:np.float64(0.06507624815359347) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.42486190795898) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01127944141626358 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01127944141626358 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.171875 - response_length/max:1781.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.171875 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013301707804203033 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8018740545958281) - timing_s/agent_loop/generate_sequences/max:np.float64(11.262868203222752) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.332270985694777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.262868203222752) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:316 - timing_s/agent_loop/slowest/response_length:1781 - timing_s/gen:17.137579491361976 - timing_s/reward:0.06755838170647621 - timing_s/old_log_prob:2.513957444578409 - timing_s/adv:0.14904913492500782 - timing_s/update_actor:10.074241016060114 - timing_s/step:30.039264272898436 - timing_s/stop_profile:0.00011388026177883148 - timing_per_token_ms/update_actor:0.05632220976396065 - timing_per_token_ms/gen:0.15243159614475021 - timing_per_token_ms/adv:0.0008332912255127122 - perf/total_num_tokens:178868 - perf/time_per_step:30.039264272898436 - perf/throughput:744.309174714773 (TaskRunner pid=2049544) Training Progress: 26%|██▌ | 26/100 [18:02<43:24, 35.20s/it] (TaskRunner pid=2049544) step:27 - global_seqlen/min:20390 - global_seqlen/max:35494 - global_seqlen/minmax_diff:15104 - global_seqlen/balanced_min:27757 - global_seqlen/balanced_max:27828 - global_seqlen/mean:27799.5 - actor/entropy:0.13566820323467255 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4101554751396179 - training/rollout_probs_diff_mean:0.002812632592394948 - training/rollout_probs_diff_std:0.010530136525630951 - training/rollout_actor_probs_pearson_corr:0.9983336925506592 - rollout_corr/rollout_is_mean:1.0000394582748413 - rollout_corr/rollout_is_ratio_fraction_high:1.9780045477091335e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.615344005287625e-05 - rollout_corr/rollout_is_max:2.5563464164733887 - rollout_corr/rollout_is_min:0.3321341276168823 - rollout_corr/rollout_is_std:0.03021036833524704 - rollout_corr/rollout_is_eff_sample_size:0.9990882848714958 - rollout_corr/rollout_is_seq_mean:1.0000697374343872 - rollout_corr/rollout_is_seq_std:0.0016781107988208532 - rollout_corr/rollout_is_seq_max:1.0076134204864502 - rollout_corr/rollout_is_seq_min:0.9905708432197571 - rollout_corr/rollout_is_seq_max_deviation:0.00942915678024292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1346074021421373) - rollout_corr/training_log_ppl:np.float64(0.12218492220563348) - rollout_corr/kl:np.float64(0.0005914641850237601) - rollout_corr/k3_kl:np.float64(0.0005361424209464616) - rollout_corr/rollout_ppl:np.float64(1.1339335390366614) - rollout_corr/rollout_log_ppl:np.float64(0.1215934576903237) - rollout_corr/log_ppl_diff:np.float64(0.0005914645153097808) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011890121677424759) - rollout_corr/log_ppl_diff_max:np.float64(0.012660838663578033) - rollout_corr/log_ppl_diff_min:np.float64(-0.006112679839134216) - rollout_corr/ppl_ratio:np.float64(1.0005930345505476) - rollout_corr/chi2_token:np.float64(0.0009534833952784538) - rollout_corr/chi2_seq:np.float64(1.2373452726751566) - actor/pg_loss:np.float64(0.00019052473362535238) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004077799751485145) - actor/ppo_kl:np.float64(0.00015187378462222512) - actor/pg_clipfrac_lower:np.float64(2.895663556046202e-06) - actor/grad_norm:np.float64(0.24880316480994225) - perf/mfu/actor:np.float64(0.08016295921139835) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.3461446762085) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011563085718080401 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011563085718080401 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:592.453125 - response_length/max:2410.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:592.453125 - response_length_non_aborted/max:2410.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010118260979652405 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9887365885078907) - timing_s/agent_loop/generate_sequences/max:np.float64(15.905478810891509) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.845138130702253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.905478810891509) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:2410 - timing_s/gen:21.729817861691117 - timing_s/reward:0.06810113973915577 - timing_s/old_log_prob:2.5614845752716064 - timing_s/adv:0.1540636345744133 - timing_s/update_actor:10.151232639327645 - timing_s/step:34.75130227394402 - timing_s/stop_profile:0.00013370253145694733 - timing_per_token_ms/update_actor:0.04564485260223945 - timing_per_token_ms/gen:0.1432722648264045 - timing_per_token_ms/adv:0.0006927446292847592 - perf/total_num_tokens:222396 - perf/time_per_step:34.75130227394402 - perf/throughput:799.9556327661317 (TaskRunner pid=2049544) Training Progress: 27%|██▋ | 27/100 [18:37<42:40, 35.08s/it] (TaskRunner pid=2049544) step:28 - global_seqlen/min:17957 - global_seqlen/max:34244 - global_seqlen/minmax_diff:16287 - global_seqlen/balanced_min:25513 - global_seqlen/balanced_max:25550 - global_seqlen/mean:25526.75 - actor/entropy:0.16894282400608063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068703174591064 - training/rollout_probs_diff_mean:0.003456175560131669 - training/rollout_probs_diff_std:0.011879513040184975 - training/rollout_actor_probs_pearson_corr:0.9982811212539673 - rollout_corr/rollout_is_mean:0.9999144077301025 - rollout_corr/rollout_is_ratio_fraction_high:1.4930944416846614e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.972377766738646e-05 - rollout_corr/rollout_is_max:2.6947081089019775 - rollout_corr/rollout_is_min:0.39785826206207275 - rollout_corr/rollout_is_std:0.03413606435060501 - rollout_corr/rollout_is_eff_sample_size:0.9988359665383245 - rollout_corr/rollout_is_seq_mean:0.9999093413352966 - rollout_corr/rollout_is_seq_std:0.002044897060841322 - rollout_corr/rollout_is_seq_max:1.0098328590393066 - rollout_corr/rollout_is_seq_min:0.9918521642684937 - rollout_corr/rollout_is_seq_max_deviation:0.00983285903930664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1787242372520268) - rollout_corr/training_log_ppl:np.float64(0.15832054920610972) - rollout_corr/kl:np.float64(0.0006220183064016283) - rollout_corr/k3_kl:np.float64(0.0006294625401039866) - rollout_corr/rollout_ppl:np.float64(1.1779772732406855) - rollout_corr/rollout_log_ppl:np.float64(0.15769852932862705) - rollout_corr/log_ppl_diff:np.float64(0.0006220198774826713) - rollout_corr/log_ppl_abs_diff:np.float64(0.001586154270626139) - rollout_corr/log_ppl_diff_max:np.float64(0.009446874260902405) - rollout_corr/log_ppl_diff_min:np.float64(-0.006108276546001434) - rollout_corr/ppl_ratio:np.float64(1.0006244094111025) - rollout_corr/chi2_token:np.float64(0.0012534137349575758) - rollout_corr/chi2_seq:np.float64(0.5095949505921453) - actor/pg_loss:np.float64(0.0001528579741716385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005431991780824319) - actor/ppo_kl:np.float64(-0.00011134325540240742) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2734988182783127) - perf/mfu/actor:np.float64(0.07474159440863055) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.44351959228516) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018798992037773132 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018798992037773132 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:523.2421875 - response_length/max:1995.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:523.2421875 - response_length_non_aborted/max:1995.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001253858208656311 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8268284611403942) - timing_s/agent_loop/generate_sequences/max:np.float64(13.178542386740446) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.144354823620233) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.178542386740446) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1995 - timing_s/gen:19.040658405050635 - timing_s/reward:0.06457959674298763 - timing_s/old_log_prob:2.444152580574155 - timing_s/adv:0.15133203007280827 - timing_s/update_actor:10.063168801367283 - timing_s/step:31.856078814715147 - timing_s/stop_profile:0.00013601407408714294 - timing_per_token_ms/update_actor:0.04927756569758823 - timing_per_token_ms/gen:0.14214750582344632 - timing_per_token_ms/adv:0.0007410463047235168 - perf/total_num_tokens:204214 - perf/time_per_step:31.856078814715147 - perf/throughput:801.3148808574813 (TaskRunner pid=2049544) Training Progress: 28%|██▊ | 28/100 [19:09<40:57, 34.13s/it] (TaskRunner pid=2049544) step:29 - global_seqlen/min:20947 - global_seqlen/max:33732 - global_seqlen/minmax_diff:12785 - global_seqlen/balanced_min:25386 - global_seqlen/balanced_max:25685 - global_seqlen/mean:25470.375 - actor/entropy:0.13968856632709503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5444822311401367 - training/rollout_probs_diff_mean:0.0031616955529898405 - training/rollout_probs_diff_std:0.011410399340093136 - training/rollout_actor_probs_pearson_corr:0.9981111884117126 - rollout_corr/rollout_is_mean:1.000052809715271 - rollout_corr/rollout_is_ratio_fraction_high:7.336810995184351e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.8694487961474806e-05 - rollout_corr/rollout_is_max:3.6677300930023193 - rollout_corr/rollout_is_min:0.22905343770980835 - rollout_corr/rollout_is_std:0.0324532613158226 - rollout_corr/rollout_is_eff_sample_size:0.9989480129782379 - rollout_corr/rollout_is_seq_mean:1.0001238584518433 - rollout_corr/rollout_is_seq_std:0.002332444768399 - rollout_corr/rollout_is_seq_max:1.0094679594039917 - rollout_corr/rollout_is_seq_min:0.9920924305915833 - rollout_corr/rollout_is_seq_max_deviation:0.0094679594039917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.140123389661312) - rollout_corr/training_log_ppl:np.float64(0.12807749381863687) - rollout_corr/kl:np.float64(0.000638476259126719) - rollout_corr/k3_kl:np.float64(0.0006935356665564996) - rollout_corr/rollout_ppl:np.float64(1.1393897044472396) - rollout_corr/rollout_log_ppl:np.float64(0.1274390167054662) - rollout_corr/log_ppl_diff:np.float64(0.0006384771131706657) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017721431377140107) - rollout_corr/log_ppl_diff_max:np.float64(0.011614277958869934) - rollout_corr/log_ppl_diff_min:np.float64(-0.011640861630439758) - rollout_corr/ppl_ratio:np.float64(1.0006417338736355) - rollout_corr/chi2_token:np.float64(0.0015386808663606644) - rollout_corr/chi2_seq:np.float64(0.6557973187882453) - actor/pg_loss:np.float64(8.236418943852186e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006260939494495688) - actor/ppo_kl:np.float64(0.00012735309167744013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2651471309363842) - perf/mfu/actor:np.float64(0.07338363198697957) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.38971328735352) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009649740532040596 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009649740532040596 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:532.41796875 - response_length/max:2748.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:532.41796875 - response_length_non_aborted/max:2748.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.792375981807709e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8251552730798721) - timing_s/agent_loop/generate_sequences/max:np.float64(16.657433584332466) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.999644991774403) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.657433584332466) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:2748 - timing_s/gen:22.55767191387713 - timing_s/reward:0.06376934424042702 - timing_s/old_log_prob:2.4361399989575148 - timing_s/adv:0.1466422975063324 - timing_s/update_actor:10.275281056761742 - timing_s/step:35.56420364230871 - timing_s/stop_profile:0.00011993944644927979 - timing_per_token_ms/update_actor:0.050427609805321585 - timing_per_token_ms/gen:0.16550137502019185 - timing_per_token_ms/adv:0.000719670879925857 - perf/total_num_tokens:203763 - perf/time_per_step:35.56420364230871 - perf/throughput:716.1801022222058 (TaskRunner pid=2049544) Training Progress: 29%|██▉ | 29/100 [19:45<40:54, 34.57s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2049544) step:30 - global_seqlen/min:16681 - global_seqlen/max:26264 - global_seqlen/minmax_diff:9583 - global_seqlen/balanced_min:22916 - global_seqlen/balanced_max:22938 - global_seqlen/mean:22931.25 - actor/entropy:0.1647506058216095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3170890808105469 - training/rollout_probs_diff_mean:0.0035670390352606773 - training/rollout_probs_diff_std:0.01208495907485485 - training/rollout_actor_probs_pearson_corr:0.9981399178504944 - rollout_corr/rollout_is_mean:0.9999049305915833 - rollout_corr/rollout_is_ratio_fraction_high:8.544525371689815e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00011962335702264681 - rollout_corr/rollout_is_max:2.2645792961120605 - rollout_corr/rollout_is_min:0.1764460951089859 - rollout_corr/rollout_is_std:0.03402237966656685 - rollout_corr/rollout_is_eff_sample_size:0.9988435782394575 - rollout_corr/rollout_is_seq_mean:0.9999919533729553 - rollout_corr/rollout_is_seq_std:0.0021591568365693092 - rollout_corr/rollout_is_seq_max:1.0069876909255981 - rollout_corr/rollout_is_seq_min:0.9930654764175415 - rollout_corr/rollout_is_seq_max_deviation:0.0069876909255981445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1598813771270216) - rollout_corr/training_log_ppl:np.float64(0.14310977509740042) - rollout_corr/kl:np.float64(0.0006737301786081673) - rollout_corr/k3_kl:np.float64(0.0006828237883240718) - rollout_corr/rollout_ppl:np.float64(1.1591023579239845) - rollout_corr/rollout_log_ppl:np.float64(0.14243604317016434) - rollout_corr/log_ppl_diff:np.float64(0.0006737319272360764) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017324861764791422) - rollout_corr/log_ppl_diff_max:np.float64(0.0082569420337677) - rollout_corr/log_ppl_diff_min:np.float64(-0.005524754524230957) - rollout_corr/ppl_ratio:np.float64(1.000676429597661) - rollout_corr/chi2_token:np.float64(0.0013968497514724731) - rollout_corr/chi2_seq:np.float64(0.42728084372356534) - actor/pg_loss:np.float64(2.553162630647421e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004172229162122676) - actor/ppo_kl:np.float64(2.4439394231023925e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29012734070420265) - perf/mfu/actor:np.float64(0.06612954074679292) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.5312614440918) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63125) - val-aux/sciknoweval/reward/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09227713650829415) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.81775) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04889979668850037) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6632125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06499777900917662) - val-aux/sciknoweval/score/mean@16:np.float64(0.63125) - val-aux/sciknoweval/score/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09227713650829415) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.81775) - val-aux/sciknoweval/score/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04889979668850037) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6632125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06499777900917662) - val-core/sciknoweval/acc/mean@16:np.float64(0.63125) - val-aux/sciknoweval/acc/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09227713650829415) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.81775) - val-core/sciknoweval/acc/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04889979668850037) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6632125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06499777900917662) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98359375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04653599667599353) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.997425) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.01734892838049699) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9696374999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.05990914656773903) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9836) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.046352514622771394) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00190372690402996) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9472625000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0713750183504594) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9930749999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.029627882526401583) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9146000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07555851957352466) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9977499999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014792280126255261) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8794875000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06529402915117104) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9997250000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0034755281407842023) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012676871381700039 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012676871381700039 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.1640625 - response_length/max:1567.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:457.1640625 - response_length_non_aborted/max:1567.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010706298053264618 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7780425660312176) - timing_s/agent_loop/generate_sequences/max:np.float64(10.887974064797163) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.538448243663879) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.887974064797163) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1567 - timing_s/gen:16.679681358858943 - timing_s/reward:0.06917422637343407 - timing_s/old_log_prob:2.485142918303609 - timing_s/adv:0.12575341202318668 - timing_s/update_actor:10.15482790954411 - timing_s/step:29.608009880408645 - timing_s/testing:103.1428439784795 - timing_s/save_checkpoint:6.939728271216154 - timing_s/stop_profile:0.0001257825642824173 - timing_per_token_ms/update_actor:0.05535474466908755 - timing_per_token_ms/gen:0.14251996307789996 - timing_per_token_ms/adv:0.0006854914800936859 - perf/total_num_tokens:183450 - perf/time_per_step:29.608009880408645 - perf/throughput:774.4948104456491 (TaskRunner pid=2049544) Training Progress: 30%|███ | 30/100 [22:05<1:17:09, 66.13s/it] (TaskRunner pid=2049544) step:31 - global_seqlen/min:19206 - global_seqlen/max:40437 - global_seqlen/minmax_diff:21231 - global_seqlen/balanced_min:26988 - global_seqlen/balanced_max:27006 - global_seqlen/mean:26994.75 - actor/entropy:0.15669088065624237 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2584149241447449 - training/rollout_probs_diff_mean:0.003061925061047077 - training/rollout_probs_diff_std:0.010589882731437683 - training/rollout_actor_probs_pearson_corr:0.9985439777374268 - rollout_corr/rollout_is_mean:1.0000404119491577 - rollout_corr/rollout_is_ratio_fraction_high:1.3810819837090094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.7621639674180187e-05 - rollout_corr/rollout_is_max:2.6365303993225098 - rollout_corr/rollout_is_min:0.22984786331653595 - rollout_corr/rollout_is_std:0.030734580010175705 - rollout_corr/rollout_is_eff_sample_size:0.9990563960283122 - rollout_corr/rollout_is_seq_mean:1.0000174045562744 - rollout_corr/rollout_is_seq_std:0.0015726868296042085 - rollout_corr/rollout_is_seq_max:1.0063960552215576 - rollout_corr/rollout_is_seq_min:0.9952640533447266 - rollout_corr/rollout_is_seq_max_deviation:0.006396055221557617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1623710673302412) - rollout_corr/training_log_ppl:np.float64(0.14397141717927298) - rollout_corr/kl:np.float64(0.0004526557635822215) - rollout_corr/k3_kl:np.float64(0.0005084346583146271) - rollout_corr/rollout_ppl:np.float64(1.1618167613632977) - rollout_corr/rollout_log_ppl:np.float64(0.1435187609968125) - rollout_corr/log_ppl_diff:np.float64(0.0004526561824604869) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012202383077237755) - rollout_corr/log_ppl_diff_max:np.float64(0.005235195159912109) - rollout_corr/log_ppl_diff_min:np.float64(-0.004862427711486816) - rollout_corr/ppl_ratio:np.float64(1.0004539461806417) - rollout_corr/chi2_token:np.float64(0.0011370060965418816) - rollout_corr/chi2_seq:np.float64(0.43574777944013476) - actor/pg_loss:np.float64(1.602328848093748e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027079670348939544) - actor/ppo_kl:np.float64(-1.6756775252702028e-05) - actor/pg_clipfrac_lower:np.float64(2.4157391180779086e-06) - actor/grad_norm:np.float64(0.20888379216194153) - perf/mfu/actor:np.float64(0.07792531348505069) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.11697006225586) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006074171047657728 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.006074171047657728 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:565.6796875 - response_length/max:1784.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:565.6796875 - response_length_non_aborted/max:1784.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.553879499435425e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7227725125849247) - timing_s/agent_loop/generate_sequences/max:np.float64(13.10448818653822) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.673034100342193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.10448818653822) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:1784 - timing_s/gen:18.842551900073886 - timing_s/reward:0.07317895255982876 - timing_s/old_log_prob:2.6255607213824987 - timing_s/adv:0.14376060664653778 - timing_s/update_actor:10.008040392771363 - timing_s/step:31.798675939440727 - timing_s/stop_profile:0.00011486746370792389 - timing_per_token_ms/update_actor:0.0463425313846737 - timing_per_token_ms/gen:0.13011554062503547 - timing_per_token_ms/adv:0.000665687803399447 - perf/total_num_tokens:215958 - perf/time_per_step:31.798675939440727 - perf/throughput:848.9268563071744 (TaskRunner pid=2049544) Training Progress: 31%|███ | 31/100 [22:36<1:04:13, 55.85s/it] (TaskRunner pid=2049544) step:32 - global_seqlen/min:15472 - global_seqlen/max:26624 - global_seqlen/minmax_diff:11152 - global_seqlen/balanced_min:21157 - global_seqlen/balanced_max:21170 - global_seqlen/mean:21164.125 - actor/entropy:0.16215530037879944 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35835716128349304 - training/rollout_probs_diff_mean:0.0033622696064412594 - training/rollout_probs_diff_std:0.011649704538285732 - training/rollout_actor_probs_pearson_corr:0.9983426928520203 - rollout_corr/rollout_is_mean:0.9998896718025208 - rollout_corr/rollout_is_ratio_fraction_high:1.9218758097849786e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.765627429354936e-05 - rollout_corr/rollout_is_max:2.1169991493225098 - rollout_corr/rollout_is_min:0.016281770542263985 - rollout_corr/rollout_is_std:0.033417847007513046 - rollout_corr/rollout_is_eff_sample_size:0.9988841362904098 - rollout_corr/rollout_is_seq_mean:0.9999381303787231 - rollout_corr/rollout_is_seq_std:0.00247986800968647 - rollout_corr/rollout_is_seq_max:1.011742353439331 - rollout_corr/rollout_is_seq_min:0.9914096593856812 - rollout_corr/rollout_is_seq_max_deviation:0.011742353439331055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1614093114621937) - rollout_corr/training_log_ppl:np.float64(0.1426902024759329) - rollout_corr/kl:np.float64(0.0009178641532645582) - rollout_corr/k3_kl:np.float64(0.0007639187314083529) - rollout_corr/rollout_ppl:np.float64(1.1602927432395518) - rollout_corr/rollout_log_ppl:np.float64(0.1417723370686872) - rollout_corr/log_ppl_diff:np.float64(0.0009178654072456993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020935917491442524) - rollout_corr/log_ppl_diff_max:np.float64(0.03533336520195007) - rollout_corr/log_ppl_diff_min:np.float64(-0.009082436561584473) - rollout_corr/ppl_ratio:np.float64(1.000924628926441) - rollout_corr/chi2_token:np.float64(0.00115761230699718) - rollout_corr/chi2_seq:np.float64(0.4135602100286633) - actor/pg_loss:np.float64(-7.768673822283745e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003810818443525932) - actor/ppo_kl:np.float64(0.00014974098553555137) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26316603645682335) - perf/mfu/actor:np.float64(0.0603751772224912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.58000946044922) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003006534418091178 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003006534418091178 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.50390625 - response_length/max:1261.0 - response_length/min:50.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.50390625 - response_length_non_aborted/max:1261.0 - response_length_non_aborted/min:50.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013849511742591858 - timing_s/agent_loop/generate_sequences/min:np.float64(0.584519986063242) - timing_s/agent_loop/generate_sequences/max:np.float64(9.110312020406127) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.030619656012277) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.110312020406127) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:1261 - timing_s/gen:14.926933655515313 - timing_s/reward:0.05845559947192669 - timing_s/old_log_prob:2.34299498423934 - timing_s/adv:0.13051477633416653 - timing_s/update_actor:10.037976827472448 - timing_s/step:27.581528935581446 - timing_s/stop_profile:0.00012824125587940216 - timing_per_token_ms/update_actor:0.05928650976282063 - timing_per_token_ms/gen:0.14343855912665462 - timing_per_token_ms/adv:0.000770849115745197 - perf/total_num_tokens:169313 - perf/time_per_step:27.581528935581446 - perf/throughput:767.3296520084244 (TaskRunner pid=2049544) Training Progress: 32%|███▏ | 32/100 [23:04<53:42, 47.39s/it] (TaskRunner pid=2049544) step:33 - global_seqlen/min:21009 - global_seqlen/max:35793 - global_seqlen/minmax_diff:14784 - global_seqlen/balanced_min:26523 - global_seqlen/balanced_max:26619 - global_seqlen/mean:26546.375 - actor/entropy:0.1607956439256668 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141917824745178 - training/rollout_probs_diff_mean:0.0031411326490342617 - training/rollout_probs_diff_std:0.01075077150017023 - training/rollout_actor_probs_pearson_corr:0.998579204082489 - rollout_corr/rollout_is_mean:1.0000613927841187 - rollout_corr/rollout_is_ratio_fraction_high:2.17348788282834e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.0714716053334996e-05 - rollout_corr/rollout_is_max:5.24385929107666 - rollout_corr/rollout_is_min:0.3405860662460327 - rollout_corr/rollout_is_std:0.0315140075981617 - rollout_corr/rollout_is_eff_sample_size:0.9990078527425291 - rollout_corr/rollout_is_seq_mean:1.0000804662704468 - rollout_corr/rollout_is_seq_std:0.0017564821755513549 - rollout_corr/rollout_is_seq_max:1.006698489189148 - rollout_corr/rollout_is_seq_min:0.995316743850708 - rollout_corr/rollout_is_seq_max_deviation:0.006698489189147949 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.176677621435374) - rollout_corr/training_log_ppl:np.float64(0.15545551202376373) - rollout_corr/kl:np.float64(0.000548688222613336) - rollout_corr/k3_kl:np.float64(0.0006223271676191189) - rollout_corr/rollout_ppl:np.float64(1.1760094286873937) - rollout_corr/rollout_log_ppl:np.float64(0.1549068232634454) - rollout_corr/log_ppl_diff:np.float64(0.0005486887603183277) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014138846672722138) - rollout_corr/log_ppl_diff_max:np.float64(0.008264932781457901) - rollout_corr/log_ppl_diff_min:np.float64(-0.005941852927207947) - rollout_corr/ppl_ratio:np.float64(1.0005506034940481) - rollout_corr/chi2_token:np.float64(0.0014625547919422388) - rollout_corr/chi2_seq:np.float64(0.45368791883811355) - actor/pg_loss:np.float64(5.0174305215477943e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003640777423470354) - actor/ppo_kl:np.float64(6.6643967455704e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2803301140666008) - perf/mfu/actor:np.float64(0.07602373904107226) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.56497955322266) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006237910129129887 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006237910129129887 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:539.16796875 - response_length/max:2340.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:539.16796875 - response_length_non_aborted/max:2340.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011204741895198822 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2724983282387257) - timing_s/agent_loop/generate_sequences/max:np.float64(14.548386441543698) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.296186965904781) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.548386441543698) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:2340 - timing_s/gen:20.489616226404905 - timing_s/reward:0.06211276724934578 - timing_s/old_log_prob:2.5394117888063192 - timing_s/adv:0.15707547590136528 - timing_s/update_actor:10.15939660742879 - timing_s/step:33.50294020399451 - timing_s/stop_profile:0.00015977025032043457 - timing_per_token_ms/update_actor:0.04783796567058962 - timing_per_token_ms/gen:0.14844643603356522 - timing_per_token_ms/adv:0.0007396277076501277 - perf/total_num_tokens:212371 - perf/time_per_step:33.50294020399451 - perf/throughput:792.3595612314322 (TaskRunner pid=2049544) Training Progress: 33%|███▎ | 33/100 [23:38<48:17, 43.24s/it] (TaskRunner pid=2049544) step:34 - global_seqlen/min:19210 - global_seqlen/max:32362 - global_seqlen/minmax_diff:13152 - global_seqlen/balanced_min:24672 - global_seqlen/balanced_max:24722 - global_seqlen/mean:24706.625 - actor/entropy:0.1634763479232788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.003025032114237547 - training/rollout_probs_diff_std:0.010739969089627266 - training/rollout_actor_probs_pearson_corr:0.998622715473175 - rollout_corr/rollout_is_mean:1.0000046491622925 - rollout_corr/rollout_is_ratio_fraction_high:2.2830181478639133e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.84905462549068e-05 - rollout_corr/rollout_is_max:3.96623158454895 - rollout_corr/rollout_is_min:0.17613501846790314 - rollout_corr/rollout_is_std:0.03082752786576748 - rollout_corr/rollout_is_eff_sample_size:0.9990505658256099 - rollout_corr/rollout_is_seq_mean:1.0001435279846191 - rollout_corr/rollout_is_seq_std:0.001950009143911302 - rollout_corr/rollout_is_seq_max:1.0084269046783447 - rollout_corr/rollout_is_seq_min:0.9940277338027954 - rollout_corr/rollout_is_seq_max_deviation:0.008426904678344727 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1493624411523342) - rollout_corr/training_log_ppl:np.float64(0.13525041530374438) - rollout_corr/kl:np.float64(0.0005815880843990096) - rollout_corr/k3_kl:np.float64(0.0006954628924091821) - rollout_corr/rollout_ppl:np.float64(1.1486861389130354) - rollout_corr/rollout_log_ppl:np.float64(0.13466882749344222) - rollout_corr/log_ppl_diff:np.float64(0.0005815878103021532) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016140752850333229) - rollout_corr/log_ppl_diff_max:np.float64(0.00731133297085762) - rollout_corr/log_ppl_diff_min:np.float64(-0.007330030202865601) - rollout_corr/ppl_ratio:np.float64(1.0005840898957103) - rollout_corr/chi2_token:np.float64(0.0018483358435332775) - rollout_corr/chi2_seq:np.float64(0.8693230981007218) - actor/pg_loss:np.float64(-7.774424739181995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004309596388338832) - actor/ppo_kl:np.float64(0.00016126566817398214) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24917255342006683) - perf/mfu/actor:np.float64(0.07059737130291072) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.48730087280273) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002149841981008649 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002149841981008649 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:513.30078125 - response_length/max:2220.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:513.30078125 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001704581081867218 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9642345681786537) - timing_s/agent_loop/generate_sequences/max:np.float64(14.358597544953227) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.930665408472123) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.358597544953227) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:2220 - timing_s/gen:20.079416058957577 - timing_s/reward:0.055915795266628265 - timing_s/old_log_prob:2.5313415098935366 - timing_s/adv:0.14141708984971046 - timing_s/update_actor:10.167155608534813 - timing_s/step:33.08425388485193 - timing_s/stop_profile:0.0001313425600528717 - timing_per_token_ms/update_actor:0.05143941963205625 - timing_per_token_ms/gen:0.15280557101295672 - timing_per_token_ms/adv:0.000715481626131202 - perf/total_num_tokens:197653 - perf/time_per_step:33.08425388485193 - perf/throughput:746.7789688106661 (TaskRunner pid=2049544) Training Progress: 34%|███▍ | 34/100 [24:11<44:13, 40.21s/it] (TaskRunner pid=2049544) step:35 - global_seqlen/min:18610 - global_seqlen/max:31076 - global_seqlen/minmax_diff:12466 - global_seqlen/balanced_min:24486 - global_seqlen/balanced_max:24516 - global_seqlen/mean:24505.375 - actor/entropy:0.18308240175247192 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6351485848426819 - training/rollout_probs_diff_mean:0.003324724268168211 - training/rollout_probs_diff_std:0.011287217028439045 - training/rollout_actor_probs_pearson_corr:0.998609185218811 - rollout_corr/rollout_is_mean:0.9998504519462585 - rollout_corr/rollout_is_ratio_fraction_high:1.5409863408422098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.852466034004465e-05 - rollout_corr/rollout_is_max:3.141425609588623 - rollout_corr/rollout_is_min:0.22313041985034943 - rollout_corr/rollout_is_std:0.03237971290946007 - rollout_corr/rollout_is_eff_sample_size:0.9989524144661357 - rollout_corr/rollout_is_seq_mean:0.999904215335846 - rollout_corr/rollout_is_seq_std:0.002002937952056527 - rollout_corr/rollout_is_seq_max:1.0084916353225708 - rollout_corr/rollout_is_seq_min:0.9936511516571045 - rollout_corr/rollout_is_seq_max_deviation:0.0084916353225708 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1920651025138795) - rollout_corr/training_log_ppl:np.float64(0.16792560920293909) - rollout_corr/kl:np.float64(0.0008154499521539904) - rollout_corr/k3_kl:np.float64(0.0007078099062596266) - rollout_corr/rollout_ppl:np.float64(1.1910648443736136) - rollout_corr/rollout_log_ppl:np.float64(0.1671101573156193) - rollout_corr/log_ppl_diff:np.float64(0.0008154518873197958) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017030747985700145) - rollout_corr/log_ppl_diff_max:np.float64(0.010384440422058105) - rollout_corr/log_ppl_diff_min:np.float64(-0.007758818566799164) - rollout_corr/ppl_ratio:np.float64(1.0008183948229998) - rollout_corr/chi2_token:np.float64(0.0012777894735336304) - rollout_corr/chi2_seq:np.float64(1.3958141808398068) - actor/pg_loss:np.float64(1.1038966476917267e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003179428899784398) - actor/ppo_kl:np.float64(0.00012983981464032013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2326635681092739) - perf/mfu/actor:np.float64(0.07068896275098592) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.57692337036133) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004130806773900986 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004130806773900986 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:506.98046875 - response_length/max:1617.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:506.98046875 - response_length_non_aborted/max:1617.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.149631321430206e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9240005984902382) - timing_s/agent_loop/generate_sequences/max:np.float64(11.936645423993468) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.090721758017025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.936645423993468) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1617 - timing_s/gen:17.894150488078594 - timing_s/reward:0.06287588179111481 - timing_s/old_log_prob:2.4778820667415857 - timing_s/adv:0.1439337357878685 - timing_s/update_actor:10.013244837522507 - timing_s/step:30.680053589865565 - timing_s/stop_profile:0.00010856986045837402 - timing_per_token_ms/update_actor:0.05107677824519369 - timing_per_token_ms/gen:0.13787321140082284 - timing_per_token_ms/adv:0.0007341947215043052 - perf/total_num_tokens:196043 - perf/time_per_step:30.680053589865565 - perf/throughput:798.7396413184485 (TaskRunner pid=2049544) Training Progress: 35%|███▌ | 35/100 [24:41<40:28, 37.37s/it] (TaskRunner pid=2049544) step:36 - global_seqlen/min:24110 - global_seqlen/max:37995 - global_seqlen/minmax_diff:13885 - global_seqlen/balanced_min:31143 - global_seqlen/balanced_max:31201 - global_seqlen/mean:31181.25 - actor/entropy:0.2063475251197815 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35195159912109375 - training/rollout_probs_diff_mean:0.003259577788412571 - training/rollout_probs_diff_std:0.010389592498540878 - training/rollout_actor_probs_pearson_corr:0.9988572597503662 - rollout_corr/rollout_is_mean:0.9999135732650757 - rollout_corr/rollout_is_ratio_fraction_high:5.54268399355351e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.217073597421404e-05 - rollout_corr/rollout_is_max:5.167435169219971 - rollout_corr/rollout_is_min:0.08687802404165268 - rollout_corr/rollout_is_std:0.03102603740990162 - rollout_corr/rollout_is_eff_sample_size:0.9990380727632393 - rollout_corr/rollout_is_seq_mean:0.9999022483825684 - rollout_corr/rollout_is_seq_std:0.0015286069829016924 - rollout_corr/rollout_is_seq_max:1.0051732063293457 - rollout_corr/rollout_is_seq_min:0.9951693415641785 - rollout_corr/rollout_is_seq_max_deviation:0.005173206329345703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2262746798805892) - rollout_corr/training_log_ppl:np.float64(0.19696971234225202) - rollout_corr/kl:np.float64(0.0006086873604012055) - rollout_corr/k3_kl:np.float64(0.0005889998031136656) - rollout_corr/rollout_ppl:np.float64(1.2254916024394333) - rollout_corr/rollout_log_ppl:np.float64(0.19636102448566817) - rollout_corr/log_ppl_diff:np.float64(0.000608687856583856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012519802548922598) - rollout_corr/log_ppl_diff_max:np.float64(0.006760932505130768) - rollout_corr/log_ppl_diff_min:np.float64(-0.005945771932601929) - rollout_corr/ppl_ratio:np.float64(1.0006101776380092) - rollout_corr/chi2_token:np.float64(0.001163150416687131) - rollout_corr/chi2_seq:np.float64(0.9006679092999548) - actor/pg_loss:np.float64(6.59312354400754e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041949365038362885) - actor/ppo_kl:np.float64(-4.479620235997572e-05) - actor/pg_clipfrac_lower:np.float64(3.426535158723709e-06) - actor/grad_norm:np.float64(0.21542762964963913) - perf/mfu/actor:np.float64(0.08866368508784078) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.68338394165039) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03192170336842537 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03192170336842537 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:704.7578125 - response_length/max:2399.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:704.7578125 - response_length_non_aborted/max:2399.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.613484144210815e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1882787812501192) - timing_s/agent_loop/generate_sequences/max:np.float64(16.427959153428674) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.1280951405206) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.427959153428674) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:2399 - timing_s/gen:22.231742713600397 - timing_s/reward:0.06625362113118172 - timing_s/old_log_prob:2.5154502913355827 - timing_s/adv:0.1720115952193737 - timing_s/update_actor:10.132180897518992 - timing_s/step:35.22234913893044 - timing_s/stop_profile:0.00012091919779777527 - timing_per_token_ms/update_actor:0.04061808337349766 - timing_per_token_ms/gen:0.12322352932412729 - timing_per_token_ms/adv:0.00068956342040238 - perf/total_num_tokens:249450 - perf/time_per_step:35.22234913893044 - perf/throughput:885.2688921175928 (TaskRunner pid=2049544) Training Progress: 36%|███▌ | 36/100 [25:17<39:11, 36.74s/it] (TaskRunner pid=2049544) step:37 - global_seqlen/min:20543 - global_seqlen/max:33018 - global_seqlen/minmax_diff:12475 - global_seqlen/balanced_min:26755 - global_seqlen/balanced_max:26801 - global_seqlen/mean:26782.75 - actor/entropy:0.1873542070388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535153567790985 - training/rollout_probs_diff_mean:0.003138389438390732 - training/rollout_probs_diff_std:0.010506313294172287 - training/rollout_actor_probs_pearson_corr:0.9987807273864746 - rollout_corr/rollout_is_mean:0.9999997019767761 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.388498225831427e-05 - rollout_corr/rollout_is_max:1.984167218208313 - rollout_corr/rollout_is_min:0.38167211413383484 - rollout_corr/rollout_is_std:0.031033720821142197 - rollout_corr/rollout_is_eff_sample_size:0.9990378348031789 - rollout_corr/rollout_is_seq_mean:0.9999757409095764 - rollout_corr/rollout_is_seq_std:0.0020235180854797363 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.991079568862915 - rollout_corr/rollout_is_seq_max_deviation:0.008920431137084961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.17386184213683) - rollout_corr/training_log_ppl:np.float64(0.15399431145488052) - rollout_corr/kl:np.float64(0.000585384931017785) - rollout_corr/k3_kl:np.float64(0.0005648392770467581) - rollout_corr/rollout_ppl:np.float64(1.1731591601856053) - rollout_corr/rollout_log_ppl:np.float64(0.15340892578387866) - rollout_corr/log_ppl_diff:np.float64(0.000585385671001859) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015394459333037958) - rollout_corr/log_ppl_diff_max:np.float64(0.009715557098388672) - rollout_corr/log_ppl_diff_min:np.float64(-0.006565690040588379) - rollout_corr/ppl_ratio:np.float64(1.0005877201911062) - rollout_corr/chi2_token:np.float64(0.0010866590309888124) - rollout_corr/chi2_seq:np.float64(1.2919703791849315) - actor/pg_loss:np.float64(-7.207388989627361e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027252964378021716) - actor/ppo_kl:np.float64(2.565641079144143e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2241645734757185) - perf/mfu/actor:np.float64(0.07756524385091812) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.62493515014648) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012340063229203224 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012340063229203224 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:576.3984375 - response_length/max:2563.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:576.3984375 - response_length_non_aborted/max:2563.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.650814950466156e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7121520359069109) - timing_s/agent_loop/generate_sequences/max:np.float64(16.06510110758245) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.399860835161235) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.06510110758245) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2563 - timing_s/gen:21.823639852926135 - timing_s/reward:0.07287336513400078 - timing_s/old_log_prob:2.61249627918005 - timing_s/adv:0.16705027595162392 - timing_s/update_actor:10.120526600629091 - timing_s/step:34.892802860587835 - timing_s/stop_profile:0.00011398270726203918 - timing_per_token_ms/update_actor:0.04723435140449119 - timing_per_token_ms/gen:0.14789872357260286 - timing_per_token_ms/adv:0.0007796542361763818 - perf/total_num_tokens:214262 - perf/time_per_step:34.892802860587835 - perf/throughput:767.5723302312205 (TaskRunner pid=2049544) Training Progress: 37%|███▋ | 37/100 [25:52<38:00, 36.20s/it] (TaskRunner pid=2049544) step:38 - global_seqlen/min:20896 - global_seqlen/max:36221 - global_seqlen/minmax_diff:15325 - global_seqlen/balanced_min:26629 - global_seqlen/balanced_max:26693 - global_seqlen/mean:26673.0 - actor/entropy:0.17777901887893677 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311151742935181 - training/rollout_probs_diff_mean:0.003014863934367895 - training/rollout_probs_diff_std:0.01037665270268917 - training/rollout_actor_probs_pearson_corr:0.998752772808075 - rollout_corr/rollout_is_mean:1.0000866651535034 - rollout_corr/rollout_is_ratio_fraction_high:2.75694746960653e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.446184564381838e-05 - rollout_corr/rollout_is_max:4.722990036010742 - rollout_corr/rollout_is_min:0.217980295419693 - rollout_corr/rollout_is_std:0.03060242161154747 - rollout_corr/rollout_is_eff_sample_size:0.9990646060209646 - rollout_corr/rollout_is_seq_mean:1.0000120401382446 - rollout_corr/rollout_is_seq_std:0.0017710411921143532 - rollout_corr/rollout_is_seq_max:1.005569338798523 - rollout_corr/rollout_is_seq_min:0.9887515902519226 - rollout_corr/rollout_is_seq_max_deviation:0.011248409748077393 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1705434750765562) - rollout_corr/training_log_ppl:np.float64(0.15039988774515223) - rollout_corr/kl:np.float64(0.0003758689842143781) - rollout_corr/k3_kl:np.float64(0.0005343400661388387) - rollout_corr/rollout_ppl:np.float64(1.170116110239178) - rollout_corr/rollout_log_ppl:np.float64(0.1500240175955696) - rollout_corr/log_ppl_diff:np.float64(0.00037587014958262444) - rollout_corr/log_ppl_abs_diff:np.float64(0.00122680542699527) - rollout_corr/log_ppl_diff_max:np.float64(0.012495182454586029) - rollout_corr/log_ppl_diff_min:np.float64(-0.005860418081283569) - rollout_corr/ppl_ratio:np.float64(1.000377441989258) - rollout_corr/chi2_token:np.float64(0.0015310256276279688) - rollout_corr/chi2_seq:np.float64(0.8941571062896401) - actor/pg_loss:np.float64(7.285538595169783e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002832129719081422) - actor/ppo_kl:np.float64(-9.715641379060003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20185822248458862) - perf/mfu/actor:np.float64(0.07681988024557596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.53835678100586) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021598960738629103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021598960738629103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:566.75 - response_length/max:2166.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:566.75 - response_length_non_aborted/max:2166.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.976459503173828e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1030525546520948) - timing_s/agent_loop/generate_sequences/max:np.float64(14.58826527185738) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.473840224818559) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.58826527185738) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:328 - timing_s/agent_loop/slowest/response_length:2166 - timing_s/gen:20.538045482710004 - timing_s/reward:0.06479933671653271 - timing_s/old_log_prob:2.464496372267604 - timing_s/adv:0.16802607849240303 - timing_s/update_actor:10.160526245832443 - timing_s/step:33.478492356836796 - timing_s/stop_profile:0.00017846189439296722 - timing_per_token_ms/update_actor:0.04761615793982887 - timing_per_token_ms/gen:0.1415557832674653 - timing_per_token_ms/adv:0.0007874352270667109 - perf/total_num_tokens:213384 - perf/time_per_step:33.478492356836796 - perf/throughput:796.720465058606 (TaskRunner pid=2049544) Training Progress: 38%|███▊ | 38/100 [26:25<36:34, 35.40s/it] (TaskRunner pid=2049544) step:39 - global_seqlen/min:15255 - global_seqlen/max:27450 - global_seqlen/minmax_diff:12195 - global_seqlen/balanced_min:22238 - global_seqlen/balanced_max:22271 - global_seqlen/mean:22258.375 - actor/entropy:0.1585860699415207 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.403981477022171 - training/rollout_probs_diff_mean:0.003201692830771208 - training/rollout_probs_diff_std:0.011194097809493542 - training/rollout_actor_probs_pearson_corr:0.9984230995178223 - rollout_corr/rollout_is_mean:1.0000526905059814 - rollout_corr/rollout_is_ratio_fraction_high:4.465760866878554e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.252065213629976e-05 - rollout_corr/rollout_is_max:3.0618841648101807 - rollout_corr/rollout_is_min:0.25154978036880493 - rollout_corr/rollout_is_std:0.032896533608436584 - rollout_corr/rollout_is_eff_sample_size:0.9989191068729061 - rollout_corr/rollout_is_seq_mean:1.0000567436218262 - rollout_corr/rollout_is_seq_std:0.0022173086181282997 - rollout_corr/rollout_is_seq_max:1.0083491802215576 - rollout_corr/rollout_is_seq_min:0.9902656674385071 - rollout_corr/rollout_is_seq_max_deviation:0.00973433256149292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1577396178618073) - rollout_corr/training_log_ppl:np.float64(0.13937123074720148) - rollout_corr/kl:np.float64(0.00047606502702546294) - rollout_corr/k3_kl:np.float64(0.0006253823398196801) - rollout_corr/rollout_ppl:np.float64(1.157200226560235) - rollout_corr/rollout_log_ppl:np.float64(0.1388951658955193) - rollout_corr/log_ppl_diff:np.float64(0.0004760648516821675) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017094570139306597) - rollout_corr/log_ppl_diff_max:np.float64(0.008976280689239502) - rollout_corr/log_ppl_diff_min:np.float64(-0.007652416825294495) - rollout_corr/ppl_ratio:np.float64(1.0004788676742464) - rollout_corr/chi2_token:np.float64(0.001593738328665495) - rollout_corr/chi2_seq:np.float64(1.153579653473571) - actor/pg_loss:np.float64(-4.086887929588556e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021084232048451668) - actor/ppo_kl:np.float64(-5.276580297675082e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22345420345664024) - perf/mfu/actor:np.float64(0.06472844108846296) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.6031494140625) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0039901575073599815 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0039901575073599815 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:437.35546875 - response_length/max:1579.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.35546875 - response_length_non_aborted/max:1579.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015556812286376953 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7400109507143497) - timing_s/agent_loop/generate_sequences/max:np.float64(10.687160091474652) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.344982730093761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.687160091474652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:300 - timing_s/agent_loop/slowest/response_length:1564 - timing_s/gen:16.393727527931333 - timing_s/reward:0.060859134420752525 - timing_s/old_log_prob:2.4244355112314224 - timing_s/adv:0.15019422583281994 - timing_s/update_actor:9.942341027781367 - timing_s/step:29.05965017527342 - timing_s/stop_profile:0.00014510564506053925 - timing_per_token_ms/update_actor:0.05583483198897812 - timing_per_token_ms/gen:0.14642093841654236 - timing_per_token_ms/adv:0.0008434702995660057 - perf/total_num_tokens:178067 - perf/time_per_step:29.05965017527342 - perf/throughput:765.954678247966 (TaskRunner pid=2049544) Training Progress: 39%|███▉ | 39/100 [26:54<34:04, 33.52s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2049544) (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2049544) step:40 - global_seqlen/min:20667 - global_seqlen/max:43425 - global_seqlen/minmax_diff:22758 - global_seqlen/balanced_min:28994 - global_seqlen/balanced_max:29096 - global_seqlen/mean:29057.375 - actor/entropy:0.16825498640537262 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38080501556396484 - training/rollout_probs_diff_mean:0.002904911758378148 - training/rollout_probs_diff_std:0.010255875997245312 - training/rollout_actor_probs_pearson_corr:0.9987383484840393 - rollout_corr/rollout_is_mean:0.9999452829360962 - rollout_corr/rollout_is_ratio_fraction_high:1.811298898246605e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.62259779649321e-05 - rollout_corr/rollout_is_max:2.355332374572754 - rollout_corr/rollout_is_min:0.382403701543808 - rollout_corr/rollout_is_std:0.029906976968050003 - rollout_corr/rollout_is_eff_sample_size:0.9991061339931943 - rollout_corr/rollout_is_seq_mean:0.9999656081199646 - rollout_corr/rollout_is_seq_std:0.0016488394467160106 - rollout_corr/rollout_is_seq_max:1.009485125541687 - rollout_corr/rollout_is_seq_min:0.9944038391113281 - rollout_corr/rollout_is_seq_max_deviation:0.009485125541687012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.157492823433131) - rollout_corr/training_log_ppl:np.float64(0.14261198020540178) - rollout_corr/kl:np.float64(0.0005399936895650326) - rollout_corr/k3_kl:np.float64(0.0005378889725164981) - rollout_corr/rollout_ppl:np.float64(1.156866628676653) - rollout_corr/rollout_log_ppl:np.float64(0.1420719847228611) - rollout_corr/log_ppl_diff:np.float64(0.0005399954825406894) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012537419388536364) - rollout_corr/log_ppl_diff_max:np.float64(0.009929120540618896) - rollout_corr/log_ppl_diff_min:np.float64(-0.007549874484539032) - rollout_corr/ppl_ratio:np.float64(1.0005416304338723) - rollout_corr/chi2_token:np.float64(0.0010685438755899668) - rollout_corr/chi2_seq:np.float64(1.119046370498836) - actor/pg_loss:np.float64(-2.1900166757404804e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024728815151320305) - actor/ppo_kl:np.float64(-7.47417178104115e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1894925720989704) - perf/mfu/actor:np.float64(0.08261510768114952) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.61338424682617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0950390140033621) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8010999999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04192728514733711) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.654725) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07283536359808326) - val-aux/sciknoweval/score/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/score/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.0950390140033621) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8010999999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04192728514733711) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.654725) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07283536359808326) - val-core/sciknoweval/acc/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/acc/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.0950390140033621) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8010999999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04192728514733711) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.654725) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07283536359808326) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9609375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.08980726114743592) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9929124999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.038290550480143085) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9296875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.11086911079562345) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9614625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.08902370132140851) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996375000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0060893990521829235) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8815249999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12288515049099524) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9828125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.060772813601514916) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8237249999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11143746284761198) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9951625) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.03001908595873743) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7747124999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0752713841460063) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99925) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.008297615319292063) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004764470737427473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004764470737427473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:646.98046875 - response_length/max:3478.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.98046875 - response_length_non_aborted/max:3478.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.821169078350067e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9843795355409384) - timing_s/agent_loop/generate_sequences/max:np.float64(20.97976647876203) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.125959783450526) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.97976647876203) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:313 - timing_s/agent_loop/slowest/response_length:3478 - timing_s/gen:26.76995438337326 - timing_s/reward:0.06942468881607056 - timing_s/old_log_prob:2.522913720458746 - timing_s/adv:0.12163343653082848 - timing_s/update_actor:10.376626841723919 - timing_s/step:39.94687524065375 - timing_s/testing:129.93085425533354 - timing_s/save_checkpoint:6.843135168775916 - timing_s/stop_profile:9.070523083209991e-05 - timing_per_token_ms/update_actor:0.044638524822544705 - timing_per_token_ms/gen:0.1616279615242277 - timing_per_token_ms/adv:0.0005232468372092648 - perf/total_num_tokens:232459 - perf/time_per_step:39.94687524065375 - perf/throughput:727.4004493454958 (TaskRunner pid=2049544) Training Progress: 40%|████ | 40/100 [29:51<1:16:29, 76.50s/it] (TaskRunner pid=2049544) step:41 - global_seqlen/min:24357 - global_seqlen/max:35343 - global_seqlen/minmax_diff:10986 - global_seqlen/balanced_min:29480 - global_seqlen/balanced_max:29507 - global_seqlen/mean:29501.125 - actor/entropy:0.1756914258003235 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44652730226516724 - training/rollout_probs_diff_mean:0.00312806130386889 - training/rollout_probs_diff_std:0.010616622865200043 - training/rollout_actor_probs_pearson_corr:0.9987709522247314 - rollout_corr/rollout_is_mean:0.9999130368232727 - rollout_corr/rollout_is_ratio_fraction_high:1.81861159944674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.424815647827927e-05 - rollout_corr/rollout_is_max:3.465665340423584 - rollout_corr/rollout_is_min:0.21781140565872192 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990134444982739 - rollout_corr/rollout_is_seq_mean:0.9999098777770996 - rollout_corr/rollout_is_seq_std:0.0014667019713670015 - rollout_corr/rollout_is_seq_max:1.0048595666885376 - rollout_corr/rollout_is_seq_min:0.9947068691253662 - rollout_corr/rollout_is_seq_max_deviation:0.005293130874633789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1796441837213933) - rollout_corr/training_log_ppl:np.float64(0.16063264349941164) - rollout_corr/kl:np.float64(0.0006431586379722631) - rollout_corr/k3_kl:np.float64(0.0005324390123746525) - rollout_corr/rollout_ppl:np.float64(1.178866429720074) - rollout_corr/rollout_log_ppl:np.float64(0.15998948468040908) - rollout_corr/log_ppl_diff:np.float64(0.000643158819002565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012713416290353052) - rollout_corr/log_ppl_diff_max:np.float64(0.007460832595825195) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031778663396835327) - rollout_corr/ppl_ratio:np.float64(1.0006445937324315) - rollout_corr/chi2_token:np.float64(0.0008295027073472738) - rollout_corr/chi2_seq:np.float64(0.9563508839346468) - actor/pg_loss:np.float64(9.131035767495632e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002985205437653349) - actor/ppo_kl:np.float64(3.691702790131757e-05) - actor/pg_clipfrac_lower:np.float64(5.99118084210204e-06) - actor/grad_norm:np.float64(0.2258460782468319) - perf/mfu/actor:np.float64(0.08484757615781298) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.7967414855957) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01819217950105667 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01819217950105667 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:644.37890625 - response_length/max:1915.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:644.37890625 - response_length_non_aborted/max:1915.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.878321826457977e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2621874269098043) - timing_s/agent_loop/generate_sequences/max:np.float64(14.428985564038157) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.6339426440390525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.428985564038157) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1915 - timing_s/gen:20.114043543115258 - timing_s/reward:0.06382922269403934 - timing_s/old_log_prob:2.600179996341467 - timing_s/adv:0.1322335172444582 - timing_s/update_actor:9.928239937871695 - timing_s/step:32.94103864394128 - timing_s/stop_profile:0.00011893920600414276 - timing_per_token_ms/update_actor:0.04206720903809471 - timing_per_token_ms/gen:0.12193211451867568 - timing_per_token_ms/adv:0.0005602901467505824 - perf/total_num_tokens:236009 - perf/time_per_step:32.94103864394128 - perf/throughput:895.5736131722133 (TaskRunner pid=2049544) Training Progress: 41%|████ | 41/100 [30:24<1:02:23, 63.45s/it] (TaskRunner pid=2049544) step:42 - global_seqlen/min:25542 - global_seqlen/max:34694 - global_seqlen/minmax_diff:9152 - global_seqlen/balanced_min:30320 - global_seqlen/balanced_max:30380 - global_seqlen/mean:30366.25 - actor/entropy:0.19629985094070435 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31755661964416504 - training/rollout_probs_diff_mean:0.0030814865604043007 - training/rollout_probs_diff_std:0.010210192762315273 - training/rollout_actor_probs_pearson_corr:0.9988877177238464 - rollout_corr/rollout_is_mean:0.9999521374702454 - rollout_corr/rollout_is_ratio_fraction_high:1.7170919818454422e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7170919818454422e-05 - rollout_corr/rollout_is_max:2.3355937004089355 - rollout_corr/rollout_is_min:0.20813822746276855 - rollout_corr/rollout_is_std:0.03025965206325054 - rollout_corr/rollout_is_eff_sample_size:0.9990850720973192 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0012307165889069438 - rollout_corr/rollout_is_seq_max:1.006462574005127 - rollout_corr/rollout_is_seq_min:0.9960255026817322 - rollout_corr/rollout_is_seq_max_deviation:0.006462574005126953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1838456057012081) - rollout_corr/training_log_ppl:np.float64(0.161605034765671) - rollout_corr/kl:np.float64(0.00040539363182290344) - rollout_corr/k3_kl:np.float64(0.0005141745206458381) - rollout_corr/rollout_ppl:np.float64(1.1833340604789555) - rollout_corr/rollout_log_ppl:np.float64(0.1611996397114126) - rollout_corr/log_ppl_diff:np.float64(0.00040539505425840616) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010806201898958534) - rollout_corr/log_ppl_diff_max:np.float64(0.005684956908226013) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034256726503372192) - rollout_corr/ppl_ratio:np.float64(1.000406396575272) - rollout_corr/chi2_token:np.float64(0.0012630913406610489) - rollout_corr/chi2_seq:np.float64(1.0963045465759933) - actor/pg_loss:np.float64(-1.263362355530262e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005835516549268505) - actor/ppo_kl:np.float64(-5.1322627289351885e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21340569853782654) - perf/mfu/actor:np.float64(0.0874389401304758) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24371719360352) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002196446730522439 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0002196446730522439 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:682.4765625 - response_length/max:2365.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:682.4765625 - response_length_non_aborted/max:2365.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014196522533893585 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5289256758987904) - timing_s/agent_loop/generate_sequences/max:np.float64(16.370171174407005) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.762006698663754) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.370171174407005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:2365 - timing_s/gen:22.206629561260343 - timing_s/reward:0.06503966823220253 - timing_s/old_log_prob:2.5655108485370874 - timing_s/adv:0.14332416281104088 - timing_s/update_actor:10.220189092680812 - timing_s/step:35.30416659079492 - timing_s/stop_profile:0.00013269484043121338 - timing_per_token_ms/update_actor:0.04207051040497597 - timing_per_token_ms/gen:0.12710274827008908 - timing_per_token_ms/adv:0.000589981323060309 - perf/total_num_tokens:242930 - perf/time_per_step:35.30416659079492 - perf/throughput:860.1321864348324 (TaskRunner pid=2049544) Training Progress: 42%|████▏ | 42/100 [31:00<53:11, 55.03s/it] (TaskRunner pid=2049544) step:43 - global_seqlen/min:22214 - global_seqlen/max:34661 - global_seqlen/minmax_diff:12447 - global_seqlen/balanced_min:28283 - global_seqlen/balanced_max:28875 - global_seqlen/mean:28428.25 - actor/entropy:0.17894256114959717 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9968691468238831 - training/rollout_probs_diff_mean:0.0030561003368347883 - training/rollout_probs_diff_std:0.010964183136820793 - training/rollout_actor_probs_pearson_corr:0.9986260533332825 - rollout_corr/rollout_is_mean:0.9999217391014099 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6834687711670995e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:3.1283660064218566e-05 - rollout_corr/rollout_is_std:0.02998857945203781 - rollout_corr/rollout_is_eff_sample_size:0.9991012551699266 - rollout_corr/rollout_is_seq_mean:0.9999701976776123 - rollout_corr/rollout_is_seq_std:0.0015368059976026416 - rollout_corr/rollout_is_seq_max:1.007220983505249 - rollout_corr/rollout_is_seq_min:0.9942285418510437 - rollout_corr/rollout_is_seq_max_deviation:0.0072209835052490234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1693742866627872) - rollout_corr/training_log_ppl:np.float64(0.15083149004203733) - rollout_corr/kl:np.float64(0.0007233850172774225) - rollout_corr/k3_kl:np.float64(0.0006608226789381888) - rollout_corr/rollout_ppl:np.float64(1.168502108193934) - rollout_corr/rollout_log_ppl:np.float64(0.15010810326202773) - rollout_corr/log_ppl_diff:np.float64(0.000723386780009605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408983988687396) - rollout_corr/log_ppl_diff_max:np.float64(0.025284215807914734) - rollout_corr/log_ppl_diff_min:np.float64(-0.005987722426652908) - rollout_corr/ppl_ratio:np.float64(1.0007265941239893) - rollout_corr/chi2_token:np.float64(0.0009258103091269732) - rollout_corr/chi2_seq:np.float64(1.824508806457743) - actor/pg_loss:np.float64(0.0001151182223111391) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000508402825516896) - actor/ppo_kl:np.float64(5.8405713696974004e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2429964765906334) - perf/mfu/actor:np.float64(0.08157217656976501) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.25969696044922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018738869577646255 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018738869577646255 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:618.5703125 - response_length/max:2843.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:618.5703125 - response_length_non_aborted/max:2843.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.98380172252655e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5165256280452013) - timing_s/agent_loop/generate_sequences/max:np.float64(18.731039391830564) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.20822462200158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.731039391830564) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:2843 - timing_s/gen:24.405709667131305 - timing_s/reward:0.0670168325304985 - timing_s/old_log_prob:2.534782351925969 - timing_s/adv:0.14802170917391777 - timing_s/update_actor:10.064561607316136 - timing_s/step:37.307200910523534 - timing_s/stop_profile:0.00010807998478412628 - timing_per_token_ms/update_actor:0.044254226022161655 - timing_per_token_ms/gen:0.15412120734008175 - timing_per_token_ms/adv:0.0006508565826858748 - perf/total_num_tokens:227426 - perf/time_per_step:37.307200910523534 - perf/throughput:762.0043666149454 (TaskRunner pid=2049544) Training Progress: 43%|████▎ | 43/100 [31:37<47:14, 49.73s/it] (TaskRunner pid=2049544) step:44 - global_seqlen/min:22462 - global_seqlen/max:40198 - global_seqlen/minmax_diff:17736 - global_seqlen/balanced_min:31295 - global_seqlen/balanced_max:31374 - global_seqlen/mean:31349.5 - actor/entropy:0.17300812900066376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6893051266670227 - training/rollout_probs_diff_mean:0.002969349967315793 - training/rollout_probs_diff_std:0.010275507345795631 - training/rollout_actor_probs_pearson_corr:0.9987791776657104 - rollout_corr/rollout_is_mean:0.9999350309371948 - rollout_corr/rollout_is_ratio_fraction_high:1.112074914999539e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.448299659998156e-05 - rollout_corr/rollout_is_max:2.314483165740967 - rollout_corr/rollout_is_min:0.03194784000515938 - rollout_corr/rollout_is_std:0.030064012855291367 - rollout_corr/rollout_is_eff_sample_size:0.9990967333762735 - rollout_corr/rollout_is_seq_mean:0.9999725222587585 - rollout_corr/rollout_is_seq_std:0.0014090986223891377 - rollout_corr/rollout_is_seq_max:1.0068600177764893 - rollout_corr/rollout_is_seq_min:0.9924980998039246 - rollout_corr/rollout_is_seq_max_deviation:0.0075019001960754395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1756953983567655) - rollout_corr/training_log_ppl:np.float64(0.15532817154598888) - rollout_corr/kl:np.float64(0.00048252152897987344) - rollout_corr/k3_kl:np.float64(0.0005270084845747647) - rollout_corr/rollout_ppl:np.float64(1.1751229297369719) - rollout_corr/rollout_log_ppl:np.float64(0.15484564797225175) - rollout_corr/log_ppl_diff:np.float64(0.00048252357373712584) - rollout_corr/log_ppl_abs_diff:np.float64(0.00115150140482001) - rollout_corr/log_ppl_diff_max:np.float64(0.01116035133600235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005915746092796326) - rollout_corr/ppl_ratio:np.float64(1.0004838448949158) - rollout_corr/chi2_token:np.float64(0.0010843609925359488) - rollout_corr/chi2_seq:np.float64(0.9741545785218477) - actor/pg_loss:np.float64(-0.00012030208017677069) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002879910933870633) - actor/ppo_kl:np.float64(-3.409391269992845e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15622974559664726) - perf/mfu/actor:np.float64(0.08872996761202245) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24708938598633) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017787639051675797 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017787639051675797 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:702.515625 - response_length/max:2613.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:702.515625 - response_length_non_aborted/max:2613.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001371428370475769 - timing_s/agent_loop/generate_sequences/min:np.float64(2.046190181747079) - timing_s/agent_loop/generate_sequences/max:np.float64(18.486025908961892) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.959546060519642) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.486025908961892) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:2613 - timing_s/gen:24.597592102363706 - timing_s/reward:0.07184095494449139 - timing_s/old_log_prob:2.5473712403327227 - timing_s/adv:0.13307522423565388 - timing_s/update_actor:10.182054167613387 - timing_s/step:37.62097364850342 - timing_s/stop_profile:0.0001139417290687561 - timing_per_token_ms/update_actor:0.040598949614879774 - timing_per_token_ms/gen:0.13677182503927685 - timing_per_token_ms/adv:0.0005306114301490211 - perf/total_num_tokens:250796 - perf/time_per_step:37.62097364850342 - perf/throughput:833.2984758156863 (TaskRunner pid=2049544) Training Progress: 44%|████▍ | 44/100 [32:15<43:02, 46.11s/it] (TaskRunner pid=2049544) step:45 - global_seqlen/min:20785 - global_seqlen/max:36095 - global_seqlen/minmax_diff:15310 - global_seqlen/balanced_min:29569 - global_seqlen/balanced_max:29783 - global_seqlen/mean:29630.5 - actor/entropy:0.2007351815700531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102352559566498 - training/rollout_probs_diff_mean:0.0033135332632809877 - training/rollout_probs_diff_std:0.01088031567633152 - training/rollout_actor_probs_pearson_corr:0.998772919178009 - rollout_corr/rollout_is_mean:1.0001227855682373 - rollout_corr/rollout_is_ratio_fraction_high:6.0338379626045935e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.033837780705653e-05 - rollout_corr/rollout_is_max:5.347679138183594 - rollout_corr/rollout_is_min:0.0047262562438845634 - rollout_corr/rollout_is_std:0.03222286328673363 - rollout_corr/rollout_is_eff_sample_size:0.998963001987778 - rollout_corr/rollout_is_seq_mean:1.00006902217865 - rollout_corr/rollout_is_seq_std:0.001625648234039545 - rollout_corr/rollout_is_seq_max:1.0073453187942505 - rollout_corr/rollout_is_seq_min:0.9958909749984741 - rollout_corr/rollout_is_seq_max_deviation:0.007345318794250488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.208143400028348) - rollout_corr/training_log_ppl:np.float64(0.17995586975303013) - rollout_corr/kl:np.float64(0.0005010489769254889) - rollout_corr/k3_kl:np.float64(0.0006270660993550337) - rollout_corr/rollout_ppl:np.float64(1.20752296410501) - rollout_corr/rollout_log_ppl:np.float64(0.1794548208345077) - rollout_corr/log_ppl_diff:np.float64(0.0005010489185224287) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013183121991460212) - rollout_corr/log_ppl_diff_max:np.float64(0.006140023469924927) - rollout_corr/log_ppl_diff_min:np.float64(-0.006050072610378265) - rollout_corr/ppl_ratio:np.float64(1.000502604758367) - rollout_corr/chi2_token:np.float64(0.0014851994346827269) - rollout_corr/chi2_seq:np.float64(1.3357745304238051) - actor/pg_loss:np.float64(-6.521574687212706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033872173025883967) - actor/ppo_kl:np.float64(-1.556063463681312e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20266040414571762) - perf/mfu/actor:np.float64(0.08389738845778953) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0823974609375) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005806060507893562 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005806060507893562 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:647.390625 - response_length/max:2551.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:647.390625 - response_length_non_aborted/max:2551.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00031680427491664886 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9299852326512337) - timing_s/agent_loop/generate_sequences/max:np.float64(16.61781131848693) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.397257668249949) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.61781131848693) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:324 - timing_s/agent_loop/slowest/response_length:2551 - timing_s/gen:22.67037677206099 - timing_s/reward:0.06372027657926083 - timing_s/old_log_prob:2.7545996010303497 - timing_s/adv:0.11587526835501194 - timing_s/update_actor:10.353036934509873 - timing_s/step:36.06465018354356 - timing_s/stop_profile:0.00011578015983104706 - timing_per_token_ms/update_actor:0.04367559159696037 - timing_per_token_ms/gen:0.13678937544988892 - timing_per_token_ms/adv:0.0004888344288613588 - perf/total_num_tokens:237044 - perf/time_per_step:36.06465018354356 - perf/throughput:821.5939943740397 (TaskRunner pid=2049544) Training Progress: 45%|████▌ | 45/100 [32:51<39:37, 43.23s/it] (TaskRunner pid=2049544) step:46 - global_seqlen/min:16300 - global_seqlen/max:40451 - global_seqlen/minmax_diff:24151 - global_seqlen/balanced_min:26008 - global_seqlen/balanced_max:26038 - global_seqlen/mean:26025.75 - actor/entropy:0.16465333104133606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42921197414398193 - training/rollout_probs_diff_mean:0.0029608916956931353 - training/rollout_probs_diff_std:0.01070943009108305 - training/rollout_actor_probs_pearson_corr:0.9986132979393005 - rollout_corr/rollout_is_mean:0.9999878406524658 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.257039058255032e-05 - rollout_corr/rollout_is_max:1.9571115970611572 - rollout_corr/rollout_is_min:0.060181617736816406 - rollout_corr/rollout_is_std:0.030463820323348045 - rollout_corr/rollout_is_eff_sample_size:0.9990728161485534 - rollout_corr/rollout_is_seq_mean:1.000013828277588 - rollout_corr/rollout_is_seq_std:0.0019006438087671995 - rollout_corr/rollout_is_seq_max:1.008240818977356 - rollout_corr/rollout_is_seq_min:0.993937611579895 - rollout_corr/rollout_is_seq_max_deviation:0.008240818977355957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.153148335404694) - rollout_corr/training_log_ppl:np.float64(0.136645665843389) - rollout_corr/kl:np.float64(0.0006094530078737392) - rollout_corr/k3_kl:np.float64(0.0006117465548527434) - rollout_corr/rollout_ppl:np.float64(1.152437952812761) - rollout_corr/rollout_log_ppl:np.float64(0.13603621201764327) - rollout_corr/log_ppl_diff:np.float64(0.0006094538257457316) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014239892771001905) - rollout_corr/log_ppl_diff_max:np.float64(0.014275498688220978) - rollout_corr/log_ppl_diff_min:np.float64(-0.006409347057342529) - rollout_corr/ppl_ratio:np.float64(1.0006118323653936) - rollout_corr/chi2_token:np.float64(0.0012153126299381256) - rollout_corr/chi2_seq:np.float64(0.35479475394822657) - actor/pg_loss:np.float64(-5.846214480698109e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001732110890770855) - actor/ppo_kl:np.float64(5.357556297269639e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1466388632543385) - perf/mfu/actor:np.float64(0.07518990372656345) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.19413375854492) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023664643988013268 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023664643988013268 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:561.8671875 - response_length/max:2403.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:561.8671875 - response_length_non_aborted/max:2403.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002136826515197754 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7386901136487722) - timing_s/agent_loop/generate_sequences/max:np.float64(15.293935690075159) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.345510819919582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.293935690075159) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:2403 - timing_s/gen:21.075742207467556 - timing_s/reward:0.06618223898112774 - timing_s/old_log_prob:2.503137046471238 - timing_s/adv:0.11277000606060028 - timing_s/update_actor:10.213601699098945 - timing_s/step:34.05827110260725 - timing_s/stop_profile:0.00012739188969135284 - timing_per_token_ms/update_actor:0.04905527073714948 - timing_per_token_ms/gen:0.14652416056582793 - timing_per_token_ms/adv:0.000541627071557017 - perf/total_num_tokens:208206 - perf/time_per_step:34.05827110260725 - perf/throughput:764.1535861169318 (TaskRunner pid=2049544) Training Progress: 46%|████▌ | 46/100 [33:25<36:26, 40.50s/it] (TaskRunner pid=2049544) step:47 - global_seqlen/min:23716 - global_seqlen/max:42058 - global_seqlen/minmax_diff:18342 - global_seqlen/balanced_min:31374 - global_seqlen/balanced_max:31661 - global_seqlen/mean:31456.375 - actor/entropy:0.14881354570388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4682484567165375 - training/rollout_probs_diff_mean:0.002767270663753152 - training/rollout_probs_diff_std:0.010147105902433395 - training/rollout_actor_probs_pearson_corr:0.9986124634742737 - rollout_corr/rollout_is_mean:1.0000232458114624 - rollout_corr/rollout_is_ratio_fraction_high:5.438477273855824e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.350781819084659e-05 - rollout_corr/rollout_is_max:2.263822555541992 - rollout_corr/rollout_is_min:0.43962690234184265 - rollout_corr/rollout_is_std:0.02890361100435257 - rollout_corr/rollout_is_eff_sample_size:0.999165278640662 - rollout_corr/rollout_is_seq_mean:1.0001194477081299 - rollout_corr/rollout_is_seq_std:0.0015050210058689117 - rollout_corr/rollout_is_seq_max:1.005627155303955 - rollout_corr/rollout_is_seq_min:0.993998646736145 - rollout_corr/rollout_is_seq_max_deviation:0.0060013532638549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1467792866751552) - rollout_corr/training_log_ppl:np.float64(0.13201592379482463) - rollout_corr/kl:np.float64(0.0004759397659170883) - rollout_corr/k3_kl:np.float64(0.0005555009257420807) - rollout_corr/rollout_ppl:np.float64(1.146229465957731) - rollout_corr/rollout_log_ppl:np.float64(0.13153998326015426) - rollout_corr/log_ppl_diff:np.float64(0.00047594053467037156) - rollout_corr/log_ppl_abs_diff:np.float64(0.001261575052922126) - rollout_corr/log_ppl_diff_max:np.float64(0.008257180452346802) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055009350180625916) - rollout_corr/ppl_ratio:np.float64(1.000477674184367) - rollout_corr/chi2_token:np.float64(0.0013230419717729092) - rollout_corr/chi2_seq:np.float64(0.636790500022471) - actor/pg_loss:np.float64(-0.00010656809899955988) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031795622612662555) - actor/ppo_kl:np.float64(0.00011273976376457995) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1950441598892212) - perf/mfu/actor:np.float64(0.0887384677499009) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.28681945800781) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011680489405989647 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011680489405989647 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:718.26171875 - response_length/max:4213.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:718.26171875 - response_length_non_aborted/max:4213.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017112679779529572 - timing_s/agent_loop/generate_sequences/min:np.float64(0.828915087506175) - timing_s/agent_loop/generate_sequences/max:np.float64(25.74352060444653) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.723139668560179) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.74352060444653) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:4213 - timing_s/gen:31.583294926211238 - timing_s/reward:0.0717991292476654 - timing_s/old_log_prob:2.5992581117898226 - timing_s/adv:0.12912281975150108 - timing_s/update_actor:10.13827026821673 - timing_s/step:44.60779097676277 - timing_s/stop_profile:0.00012019835412502289 - timing_per_token_ms/update_actor:0.040287025556094466 - timing_per_token_ms/gen:0.1717650301901359 - timing_per_token_ms/adv:0.0005131027484552061 - perf/total_num_tokens:251651 - perf/time_per_step:44.60779097676277 - perf/throughput:705.1767036925535 (TaskRunner pid=2049544) Training Progress: 47%|████▋ | 47/100 [34:10<36:52, 41.75s/it] (TaskRunner pid=2049544) step:48 - global_seqlen/min:18590 - global_seqlen/max:34823 - global_seqlen/minmax_diff:16233 - global_seqlen/balanced_min:28237 - global_seqlen/balanced_max:28801 - global_seqlen/mean:28407.875 - actor/entropy:0.14581364393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068536281585693 - training/rollout_probs_diff_mean:0.00291546992957592 - training/rollout_probs_diff_std:0.01052320096641779 - training/rollout_actor_probs_pearson_corr:0.9985108971595764 - rollout_corr/rollout_is_mean:0.9999670386314392 - rollout_corr/rollout_is_ratio_fraction_high:1.2597076420206577e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.779122926061973e-05 - rollout_corr/rollout_is_max:2.1663451194763184 - rollout_corr/rollout_is_min:0.35977837443351746 - rollout_corr/rollout_is_std:0.030446205288171768 - rollout_corr/rollout_is_eff_sample_size:0.9990738870447088 - rollout_corr/rollout_is_seq_mean:0.9999531507492065 - rollout_corr/rollout_is_seq_std:0.0016499547054991126 - rollout_corr/rollout_is_seq_max:1.0073652267456055 - rollout_corr/rollout_is_seq_min:0.9916918873786926 - rollout_corr/rollout_is_seq_max_deviation:0.008308112621307373 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1378675876185298) - rollout_corr/training_log_ppl:np.float64(0.12675013709667837) - rollout_corr/kl:np.float64(0.000753273961016987) - rollout_corr/k3_kl:np.float64(0.0006820533353391056) - rollout_corr/rollout_ppl:np.float64(1.1369990683160722) - rollout_corr/rollout_log_ppl:np.float64(0.12599686199246207) - rollout_corr/log_ppl_diff:np.float64(0.0007532751042163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014732455310877413) - rollout_corr/log_ppl_diff_max:np.float64(0.009199053049087524) - rollout_corr/log_ppl_diff_min:np.float64(-0.006944321095943451) - rollout_corr/ppl_ratio:np.float64(1.0007553710602224) - rollout_corr/chi2_token:np.float64(0.0013299805577844381) - rollout_corr/chi2_seq:np.float64(0.6524860400240868) - actor/pg_loss:np.float64(-2.4408800527453423e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820084601917188) - actor/ppo_kl:np.float64(0.00018927762639009416) - actor/pg_clipfrac_lower:np.float64(1.1097301467088982e-05) - actor/grad_norm:np.float64(0.2395218200981617) - perf/mfu/actor:np.float64(0.08095804530180152) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.21407699584961) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01809648796916008 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01809648796916008 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:620.18359375 - response_length/max:3666.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:620.18359375 - response_length_non_aborted/max:3666.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014730170369148254 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9394192546606064) - timing_s/agent_loop/generate_sequences/max:np.float64(21.90096348337829) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.9833613064474775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.90096348337829) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:3666 - timing_s/gen:28.045954892411828 - timing_s/reward:0.0676717646420002 - timing_s/old_log_prob:2.6066879238933325 - timing_s/adv:0.12023183144629002 - timing_s/update_actor:10.316766967996955 - timing_s/step:41.24971695430577 - timing_s/stop_profile:0.00013453327119350433 - timing_per_token_ms/update_actor:0.04539571759589971 - timing_per_token_ms/gen:0.17664851570170015 - timing_per_token_ms/adv:0.0005290427013912956 - perf/total_num_tokens:227263 - perf/time_per_step:41.24971695430577 - perf/throughput:688.680483104132 (TaskRunner pid=2049544) Training Progress: 48%|████▊ | 48/100 [34:51<36:03, 41.61s/it] (TaskRunner pid=2049544) step:49 - global_seqlen/min:18126 - global_seqlen/max:38899 - global_seqlen/minmax_diff:20773 - global_seqlen/balanced_min:26338 - global_seqlen/balanced_max:26710 - global_seqlen/mean:26451.75 - actor/entropy:0.18256059288978577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3656392991542816 - training/rollout_probs_diff_mean:0.0032566494774073362 - training/rollout_probs_diff_std:0.010963589884340763 - training/rollout_actor_probs_pearson_corr:0.9987143278121948 - rollout_corr/rollout_is_mean:1.0001035928726196 - rollout_corr/rollout_is_ratio_fraction_high:6.82920153849409e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.146281521068886e-05 - rollout_corr/rollout_is_max:2.4023168087005615 - rollout_corr/rollout_is_min:0.14749553799629211 - rollout_corr/rollout_is_std:0.03197216987609863 - rollout_corr/rollout_is_eff_sample_size:0.9989789431740315 - rollout_corr/rollout_is_seq_mean:1.0002081394195557 - rollout_corr/rollout_is_seq_std:0.001892627333290875 - rollout_corr/rollout_is_seq_max:1.008901834487915 - rollout_corr/rollout_is_seq_min:0.9917373061180115 - rollout_corr/rollout_is_seq_max_deviation:0.008901834487915039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1714685205370188) - rollout_corr/training_log_ppl:np.float64(0.15115834452444687) - rollout_corr/kl:np.float64(0.0004743365148804912) - rollout_corr/k3_kl:np.float64(0.0006857500739982925) - rollout_corr/rollout_ppl:np.float64(1.1708842911757529) - rollout_corr/rollout_log_ppl:np.float64(0.15068400763266254) - rollout_corr/log_ppl_diff:np.float64(0.0004743368917843327) - rollout_corr/log_ppl_abs_diff:np.float64(0.001488212525146082) - rollout_corr/log_ppl_diff_max:np.float64(0.010617084801197052) - rollout_corr/log_ppl_diff_min:np.float64(-0.015819266438484192) - rollout_corr/ppl_ratio:np.float64(1.0004769298247993) - rollout_corr/chi2_token:np.float64(0.0018537091091275215) - rollout_corr/chi2_seq:np.float64(0.6309683343861252) - actor/pg_loss:np.float64(3.3562071621418e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005490989437930693) - actor/ppo_kl:np.float64(8.648514501530258e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2604895681142807) - perf/mfu/actor:np.float64(0.07564944214215971) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27551984786987) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015421191230416298 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015421191230416298 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.9921875 - response_length/max:3613.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.9921875 - response_length_non_aborted/max:3613.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014311634004116058 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9619488045573235) - timing_s/agent_loop/generate_sequences/max:np.float64(21.043767675757408) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.410715061574592) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.043767675757408) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:3586 - timing_s/gen:27.190765019506216 - timing_s/reward:0.06571215577423573 - timing_s/old_log_prob:2.4588573779910803 - timing_s/adv:0.11915759928524494 - timing_s/update_actor:10.26756145618856 - timing_s/step:40.190888445824385 - timing_s/stop_profile:0.00011413171887397766 - timing_per_token_ms/update_actor:0.048520237111857246 - timing_per_token_ms/gen:0.185691217779869 - timing_per_token_ms/adv:0.0005630893952443834 - perf/total_num_tokens:211614 - perf/time_per_step:40.190888445824385 - perf/throughput:658.1529053694804 (TaskRunner pid=2049544) Training Progress: 49%|████▉ | 49/100 [35:31<35:01, 41.20s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2049544) step:50 - global_seqlen/min:22621 - global_seqlen/max:38182 - global_seqlen/minmax_diff:15561 - global_seqlen/balanced_min:31307 - global_seqlen/balanced_max:32242 - global_seqlen/mean:31432.0 - actor/entropy:0.20185792446136475 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3018432855606079 - training/rollout_probs_diff_mean:0.003106999211013317 - training/rollout_probs_diff_std:0.010115313343703747 - training/rollout_actor_probs_pearson_corr:0.9989494681358337 - rollout_corr/rollout_is_mean:1.0000050067901611 - rollout_corr/rollout_is_ratio_fraction_high:5.458515261125285e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.458515261125285e-06 - rollout_corr/rollout_is_max:2.0969016551971436 - rollout_corr/rollout_is_min:0.33815088868141174 - rollout_corr/rollout_is_std:0.030253741890192032 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:0.9999303817749023 - rollout_corr/rollout_is_seq_std:0.001567038125358522 - rollout_corr/rollout_is_seq_max:1.0050349235534668 - rollout_corr/rollout_is_seq_min:0.993672788143158 - rollout_corr/rollout_is_seq_max_deviation:0.006327211856842041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.191426008939743) - rollout_corr/training_log_ppl:np.float64(0.16859979891887633) - rollout_corr/kl:np.float64(0.0005692955341736194) - rollout_corr/k3_kl:np.float64(0.0005476311655883137) - rollout_corr/rollout_ppl:np.float64(1.1907334956340492) - rollout_corr/rollout_log_ppl:np.float64(0.16803050318412716) - rollout_corr/log_ppl_diff:np.float64(0.0005692957347491756) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012659797939704731) - rollout_corr/log_ppl_diff_max:np.float64(0.008102566003799438) - rollout_corr/log_ppl_diff_min:np.float64(-0.00437454879283905) - rollout_corr/ppl_ratio:np.float64(1.0005708439275622) - rollout_corr/chi2_token:np.float64(0.0010575931519269943) - rollout_corr/chi2_seq:np.float64(1.1911898551043123) - actor/pg_loss:np.float64(6.677815690636635e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036017004458699375) - actor/ppo_kl:np.float64(2.6112116451315615e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17873636819422245) - perf/mfu/actor:np.float64(0.0885810936633662) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.26837921142578) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09966534252010995) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8278625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06058065805524353) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06833156224087977) - val-aux/sciknoweval/score/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/score/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09966534252010995) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8278625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06058065805524353) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06833156224087977) - val-core/sciknoweval/acc/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09966534252010995) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8278625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06058065805524353) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6779125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06833156224087977) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.978125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.05947208208072883) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9967624999999998) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.02126663775002429) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9593875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0761225622474269) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9781625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.059091934676165636) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0017437187447881502) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9295375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08943279499033274) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9918500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03609564352248361) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.890025) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.09147875280510498) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9984124999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014058383946768527) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8474) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07545396833360063) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99985) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002693900935493915) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016236353665590286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016236353665590286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:715.625 - response_length/max:3780.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:715.625 - response_length_non_aborted/max:3780.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014733336865901947 - timing_s/agent_loop/generate_sequences/min:np.float64(0.98030811175704) - timing_s/agent_loop/generate_sequences/max:np.float64(23.301443008705974) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.870220143442566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.301443008705974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:3780 - timing_s/gen:29.260056165978312 - timing_s/reward:0.07223350368440151 - timing_s/old_log_prob:2.538018411025405 - timing_s/adv:0.11729281768202782 - timing_s/update_actor:10.420197248458862 - timing_s/step:42.49460377730429 - timing_s/testing:117.82088489830494 - timing_s/save_checkpoint:6.728960711508989 - timing_s/stop_profile:0.00011482089757919312 - timing_per_token_ms/update_actor:0.04143944566229822 - timing_per_token_ms/gen:0.15971646378809123 - timing_per_token_ms/adv:0.0004664546389110931 - perf/total_num_tokens:251456 - perf/time_per_step:42.49460377730429 - perf/throughput:739.6703864971049 (TaskRunner pid=2049544) Training Progress: 50%|█████ | 50/100 [38:18<1:05:48, 78.97s/it] (TaskRunner pid=2049544) step:51 - global_seqlen/min:18785 - global_seqlen/max:33808 - global_seqlen/minmax_diff:15023 - global_seqlen/balanced_min:26399 - global_seqlen/balanced_max:26471 - global_seqlen/mean:26450.125 - actor/entropy:0.24430422484874725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564788341522217 - training/rollout_probs_diff_mean:0.003582285949960351 - training/rollout_probs_diff_std:0.010883934795856476 - training/rollout_actor_probs_pearson_corr:0.999004602432251 - rollout_corr/rollout_is_mean:0.999866783618927 - rollout_corr/rollout_is_ratio_fraction_high:1.396638253936544e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.189914761809632e-05 - rollout_corr/rollout_is_max:2.2966716289520264 - rollout_corr/rollout_is_min:0.327895849943161 - rollout_corr/rollout_is_std:0.033104244619607925 - rollout_corr/rollout_is_eff_sample_size:0.9989051897151183 - rollout_corr/rollout_is_seq_mean:0.9998487830162048 - rollout_corr/rollout_is_seq_std:0.0018385471776127815 - rollout_corr/rollout_is_seq_max:1.0075657367706299 - rollout_corr/rollout_is_seq_min:0.9936858415603638 - rollout_corr/rollout_is_seq_max_deviation:0.007565736770629883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.222205103840679) - rollout_corr/training_log_ppl:np.float64(0.18761249752424192) - rollout_corr/kl:np.float64(0.0007189881607398974) - rollout_corr/k3_kl:np.float64(0.0005682342473676272) - rollout_corr/rollout_ppl:np.float64(1.2213212260976434) - rollout_corr/rollout_log_ppl:np.float64(0.1868935083039105) - rollout_corr/log_ppl_diff:np.float64(0.000718989220331423) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408860151888803) - rollout_corr/log_ppl_diff_max:np.float64(0.007139421999454498) - rollout_corr/log_ppl_diff_min:np.float64(-0.005957402288913727) - rollout_corr/ppl_ratio:np.float64(1.0007210047915578) - rollout_corr/chi2_token:np.float64(0.0008357972837984562) - rollout_corr/chi2_seq:np.float64(1.0313024234492332) - actor/pg_loss:np.float64(4.522060044109821e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003210717095498694) - actor/ppo_kl:np.float64(-9.128155866733323e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14451957494020462) - perf/mfu/actor:np.float64(0.07523053933355461) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.30613327026367) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011186200194060802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011186200194060802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:559.37890625 - response_length/max:2702.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:559.37890625 - response_length_non_aborted/max:2702.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.59448516368866e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9070874620229006) - timing_s/agent_loop/generate_sequences/max:np.float64(16.53190640732646) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.214220573740022) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.53190640732646) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:2702 - timing_s/gen:22.698202308267355 - timing_s/reward:0.06384330801665783 - timing_s/old_log_prob:2.640516960993409 - timing_s/adv:0.15581872314214706 - timing_s/update_actor:10.303586572408676 - timing_s/step:35.9570118393749 - timing_s/stop_profile:0.00012137554585933685 - timing_per_token_ms/update_actor:0.048693468236958595 - timing_per_token_ms/gen:0.1585058924746849 - timing_per_token_ms/adv:0.0007363798996325493 - perf/total_num_tokens:211601 - perf/time_per_step:35.9570118393749 - perf/throughput:735.6040907447059 (TaskRunner pid=2049544) Training Progress: 51%|█████ | 51/100 [38:54<53:58, 66.09s/it] (TaskRunner pid=2049544) step:52 - global_seqlen/min:23429 - global_seqlen/max:36160 - global_seqlen/minmax_diff:12731 - global_seqlen/balanced_min:30668 - global_seqlen/balanced_max:30765 - global_seqlen/mean:30739.125 - actor/entropy:0.22692662477493286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3938141167163849 - training/rollout_probs_diff_mean:0.0033940812572836876 - training/rollout_probs_diff_std:0.010467931628227234 - training/rollout_actor_probs_pearson_corr:0.9990243315696716 - rollout_corr/rollout_is_mean:1.000043272972107 - rollout_corr/rollout_is_ratio_fraction_high:1.1359441487002186e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2718882974004373e-05 - rollout_corr/rollout_is_max:2.0554285049438477 - rollout_corr/rollout_is_min:0.29720351099967957 - rollout_corr/rollout_is_std:0.03246244043111801 - rollout_corr/rollout_is_eff_sample_size:0.9989472992270986 - rollout_corr/rollout_is_seq_mean:1.0000196695327759 - rollout_corr/rollout_is_seq_std:0.001915202010422945 - rollout_corr/rollout_is_seq_max:1.0059826374053955 - rollout_corr/rollout_is_seq_min:0.9903362989425659 - rollout_corr/rollout_is_seq_max_deviation:0.009663701057434082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.231195269152522) - rollout_corr/training_log_ppl:np.float64(0.19834982651809696) - rollout_corr/kl:np.float64(0.0008127379676388102) - rollout_corr/k3_kl:np.float64(0.0006191496557619303) - rollout_corr/rollout_ppl:np.float64(1.230196955613792) - rollout_corr/rollout_log_ppl:np.float64(0.19753708717325935) - rollout_corr/log_ppl_diff:np.float64(0.0008127393448376097) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014782494035898708) - rollout_corr/log_ppl_diff_max:np.float64(0.011166423559188843) - rollout_corr/log_ppl_diff_min:np.float64(-0.00570167601108551) - rollout_corr/ppl_ratio:np.float64(1.0008151046931744) - rollout_corr/chi2_token:np.float64(0.0008716660086065531) - rollout_corr/chi2_seq:np.float64(0.30706389155238867) - actor/pg_loss:np.float64(2.5613000616431236e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015267897435933264) - actor/ppo_kl:np.float64(0.00024156836637168055) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18445459008216858) - perf/mfu/actor:np.float64(0.08808419007285372) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.37739562988281) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007727969903498888 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007727969903498888 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:687.75390625 - response_length/max:2764.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:687.75390625 - response_length_non_aborted/max:2764.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011474080383777618 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7448907978832722) - timing_s/agent_loop/generate_sequences/max:np.float64(18.379133939743042) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.580681977618951) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.379133939743042) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2764 - timing_s/gen:24.19850930571556 - timing_s/reward:0.07030453719198704 - timing_s/old_log_prob:2.519341915845871 - timing_s/adv:0.11268062144517899 - timing_s/update_actor:10.240152211859822 - timing_s/step:37.230506498366594 - timing_s/stop_profile:0.00011861883103847504 - timing_per_token_ms/update_actor:0.0416413618306467 - timing_per_token_ms/gen:0.13744077077054248 - timing_per_token_ms/adv:0.0004582133577532663 - perf/total_num_tokens:245913 - perf/time_per_step:37.230506498366594 - perf/throughput:825.643481410832 (TaskRunner pid=2049544) Training Progress: 52%|█████▏ | 52/100 [39:32<45:57, 57.45s/it] (TaskRunner pid=2049544) step:53 - global_seqlen/min:19572 - global_seqlen/max:34404 - global_seqlen/minmax_diff:14832 - global_seqlen/balanced_min:27920 - global_seqlen/balanced_max:29013 - global_seqlen/mean:28188.25 - actor/entropy:0.22438420355319977 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30675768852233887 - training/rollout_probs_diff_mean:0.0035303884651511908 - training/rollout_probs_diff_std:0.010939665138721466 - training/rollout_actor_probs_pearson_corr:0.9988002181053162 - rollout_corr/rollout_is_mean:1.000083327293396 - rollout_corr/rollout_is_ratio_fraction_high:1.938861169037409e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.524009636952542e-05 - rollout_corr/rollout_is_max:2.4050400257110596 - rollout_corr/rollout_is_min:0.11861640214920044 - rollout_corr/rollout_is_std:0.03258340060710907 - rollout_corr/rollout_is_eff_sample_size:0.9989399238583818 - rollout_corr/rollout_is_seq_mean:1.0001201629638672 - rollout_corr/rollout_is_seq_std:0.001550388871692121 - rollout_corr/rollout_is_seq_max:1.0046480894088745 - rollout_corr/rollout_is_seq_min:0.994028627872467 - rollout_corr/rollout_is_seq_max_deviation:0.005971372127532959 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2066223253495991) - rollout_corr/training_log_ppl:np.float64(0.17772597989824135) - rollout_corr/kl:np.float64(0.0005446721567352597) - rollout_corr/k3_kl:np.float64(0.0006030060052921726) - rollout_corr/rollout_ppl:np.float64(1.2059485423378646) - rollout_corr/rollout_log_ppl:np.float64(0.17718130597495474) - rollout_corr/log_ppl_diff:np.float64(0.0005446739232866094) - rollout_corr/log_ppl_abs_diff:np.float64(0.001306041274801828) - rollout_corr/log_ppl_diff_max:np.float64(0.006498947739601135) - rollout_corr/log_ppl_diff_min:np.float64(-0.005862616002559662) - rollout_corr/ppl_ratio:np.float64(1.0005462081171572) - rollout_corr/chi2_token:np.float64(0.001310169231146574) - rollout_corr/chi2_seq:np.float64(0.6170898247510195) - actor/pg_loss:np.float64(5.588738713413477e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040137838948339777) - actor/ppo_kl:np.float64(0.00010251342614253645) - actor/pg_clipfrac_lower:np.float64(1.1289739632047713e-06) - actor/grad_norm:np.float64(0.2714943513274193) - perf/mfu/actor:np.float64(0.0789630435605575) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.31622695922852) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02330026589334011 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02330026589334011 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.4140625 - response_length/max:3739.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.4140625 - response_length_non_aborted/max:3739.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001577865332365036 - timing_s/agent_loop/generate_sequences/min:np.float64(1.263271963223815) - timing_s/agent_loop/generate_sequences/max:np.float64(22.0587509367615) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.729596399491129) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.0587509367615) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3739 - timing_s/gen:27.900284377858043 - timing_s/reward:0.0937537681311369 - timing_s/old_log_prob:2.7064144033938646 - timing_s/adv:0.1434136312454939 - timing_s/update_actor:10.502349453046918 - timing_s/step:41.45142296142876 - timing_s/stop_profile:0.00011624768376350403 - timing_per_token_ms/update_actor:0.04657237258896401 - timing_per_token_ms/gen:0.18031593341858748 - timing_per_token_ms/adv:0.0006359637049368704 - perf/total_num_tokens:225506 - perf/time_per_step:41.45142296142876 - perf/throughput:680.030937085794 (TaskRunner pid=2049544) Training Progress: 53%|█████▎ | 53/100 [40:13<41:15, 52.67s/it] (TaskRunner pid=2049544) step:54 - global_seqlen/min:21471 - global_seqlen/max:31367 - global_seqlen/minmax_diff:9896 - global_seqlen/balanced_min:26297 - global_seqlen/balanced_max:26328 - global_seqlen/mean:26318.0 - actor/entropy:0.21818441152572632 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292486190795898 - training/rollout_probs_diff_mean:0.0032305221538990736 - training/rollout_probs_diff_std:0.010491388849914074 - training/rollout_actor_probs_pearson_corr:0.9989482760429382 - rollout_corr/rollout_is_mean:1.0000078678131104 - rollout_corr/rollout_is_ratio_fraction_high:2.0765270164702088e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3843512533640023e-05 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.26139989495277405 - rollout_corr/rollout_is_std:0.03054003044962883 - rollout_corr/rollout_is_eff_sample_size:0.999068175625075 - rollout_corr/rollout_is_seq_mean:0.9999727010726929 - rollout_corr/rollout_is_seq_std:0.001957092434167862 - rollout_corr/rollout_is_seq_max:1.007343053817749 - rollout_corr/rollout_is_seq_min:0.9894382953643799 - rollout_corr/rollout_is_seq_max_deviation:0.010561704635620117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1926028821617365) - rollout_corr/training_log_ppl:np.float64(0.16857526991225313) - rollout_corr/kl:np.float64(0.0005841684274781045) - rollout_corr/k3_kl:np.float64(0.0005794948536674838) - rollout_corr/rollout_ppl:np.float64(1.1918872729875147) - rollout_corr/rollout_log_ppl:np.float64(0.1679911007740884) - rollout_corr/log_ppl_diff:np.float64(0.0005841691381647252) - rollout_corr/log_ppl_abs_diff:np.float64(0.001538367178000044) - rollout_corr/log_ppl_diff_max:np.float64(0.011501207947731018) - rollout_corr/log_ppl_diff_min:np.float64(-0.005205966532230377) - rollout_corr/ppl_ratio:np.float64(1.0005866582505405) - rollout_corr/chi2_token:np.float64(0.0011341231875121593) - rollout_corr/chi2_seq:np.float64(1.4627508698031306) - actor/pg_loss:np.float64(0.00015633960720151663) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00042431396536812827) - actor/ppo_kl:np.float64(3.235917171728886e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16759618930518627) - perf/mfu/actor:np.float64(0.07638956882275981) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.35180282592773) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008146907202899456 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008146907202899456 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:564.34375 - response_length/max:2093.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:564.34375 - response_length_non_aborted/max:2093.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015513785183429718 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6626291759312153) - timing_s/agent_loop/generate_sequences/max:np.float64(14.199840543791652) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.107884573328192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.199840543791652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:2093 - timing_s/gen:19.986812436953187 - timing_s/reward:0.06590797752141953 - timing_s/old_log_prob:2.560090212151408 - timing_s/adv:0.13934014923870564 - timing_s/update_actor:10.21920620650053 - timing_s/step:33.05696338787675 - timing_s/stop_profile:0.00011862441897392273 - timing_per_token_ms/update_actor:0.04853715236007927 - timing_per_token_ms/gen:0.13834384819863493 - timing_per_token_ms/adv:0.0006618101168340378 - perf/total_num_tokens:210544 - perf/time_per_step:33.05696338787675 - perf/throughput:796.1408823670663 (TaskRunner pid=2049544) Training Progress: 54%|█████▍ | 54/100 [40:46<35:52, 46.80s/it] (TaskRunner pid=2049544) step:55 - global_seqlen/min:18191 - global_seqlen/max:33987 - global_seqlen/minmax_diff:15796 - global_seqlen/balanced_min:26610 - global_seqlen/balanced_max:26684 - global_seqlen/mean:26660.125 - actor/entropy:0.1736723929643631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.0030401991680264473 - training/rollout_probs_diff_std:0.01039278693497181 - training/rollout_actor_probs_pearson_corr:0.9987287521362305 - rollout_corr/rollout_is_mean:1.0000579357147217 - rollout_corr/rollout_is_ratio_fraction_high:1.375430656480603e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.375430656480603e-05 - rollout_corr/rollout_is_max:2.335618734359741 - rollout_corr/rollout_is_min:0.392137348651886 - rollout_corr/rollout_is_std:0.030324585735797882 - rollout_corr/rollout_is_eff_sample_size:0.999081502372468 - rollout_corr/rollout_is_seq_mean:1.000092625617981 - rollout_corr/rollout_is_seq_std:0.00183792260941118 - rollout_corr/rollout_is_seq_max:1.0104801654815674 - rollout_corr/rollout_is_seq_min:0.9918577671051025 - rollout_corr/rollout_is_seq_max_deviation:0.010480165481567383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1669259667396545) - rollout_corr/training_log_ppl:np.float64(0.14700542857826804) - rollout_corr/kl:np.float64(0.0007857797350844242) - rollout_corr/k3_kl:np.float64(0.0007165672559068526) - rollout_corr/rollout_ppl:np.float64(1.1659750957041979) - rollout_corr/rollout_log_ppl:np.float64(0.14621964823891176) - rollout_corr/log_ppl_diff:np.float64(0.000785780339356279) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016149269104062114) - rollout_corr/log_ppl_diff_max:np.float64(0.058644767850637436) - rollout_corr/log_ppl_diff_min:np.float64(-0.004740942269563675) - rollout_corr/ppl_ratio:np.float64(1.00079559488222) - rollout_corr/chi2_token:np.float64(0.0010453276336193085) - rollout_corr/chi2_seq:np.float64(0.6579223412554711) - actor/pg_loss:np.float64(-0.00018631992861628532) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005601525168685839) - actor/ppo_kl:np.float64(0.00037079197026113064) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25370683521032333) - perf/mfu/actor:np.float64(0.0767427855529022) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.34751892089844) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002467178856022656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002467178856022656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.00390625 - response_length/max:2618.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.00390625 - response_length_non_aborted/max:2618.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001354403793811798 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8230457752943039) - timing_s/agent_loop/generate_sequences/max:np.float64(16.547815980389714) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.079917931871023) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.547815980389714) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:2618 - timing_s/gen:22.890313317999244 - timing_s/reward:0.06661977060139179 - timing_s/old_log_prob:2.436913600191474 - timing_s/adv:0.1175360195338726 - timing_s/update_actor:10.17019310221076 - timing_s/step:35.7683734241873 - timing_s/stop_profile:0.0001267455518245697 - timing_per_token_ms/update_actor:0.04768447776506468 - timing_per_token_ms/gen:0.15742019626019876 - timing_per_token_ms/adv:0.0005510852796726975 - perf/total_num_tokens:213281 - perf/time_per_step:35.7683734241873 - perf/throughput:745.3546932042451 (TaskRunner pid=2049544) Training Progress: 55%|█████▌ | 55/100 [41:22<32:37, 43.51s/it] (TaskRunner pid=2049544) step:56 - global_seqlen/min:22164 - global_seqlen/max:48288 - global_seqlen/minmax_diff:26124 - global_seqlen/balanced_min:28371 - global_seqlen/balanced_max:28431 - global_seqlen/mean:28409.375 - actor/entropy:0.21692201495170593 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174453377723694 - training/rollout_probs_diff_mean:0.003212681272998452 - training/rollout_probs_diff_std:0.010248677805066109 - training/rollout_actor_probs_pearson_corr:0.9989055395126343 - rollout_corr/rollout_is_mean:1.0000497102737427 - rollout_corr/rollout_is_ratio_fraction_high:1.2312617400311865e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.309416181058623e-05 - rollout_corr/rollout_is_max:2.551107883453369 - rollout_corr/rollout_is_min:0.3708029091358185 - rollout_corr/rollout_is_std:0.031356628984212875 - rollout_corr/rollout_is_eff_sample_size:0.9990177275875637 - rollout_corr/rollout_is_seq_mean:1.0000814199447632 - rollout_corr/rollout_is_seq_std:0.002115494105964899 - rollout_corr/rollout_is_seq_max:1.0103763341903687 - rollout_corr/rollout_is_seq_min:0.9925733208656311 - rollout_corr/rollout_is_seq_max_deviation:0.010376334190368652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2074726885184646) - rollout_corr/training_log_ppl:np.float64(0.18105585960438475) - rollout_corr/kl:np.float64(0.0007089410478542391) - rollout_corr/k3_kl:np.float64(0.0007255429266592728) - rollout_corr/rollout_ppl:np.float64(1.2066095937043428) - rollout_corr/rollout_log_ppl:np.float64(0.18034691788489) - rollout_corr/log_ppl_diff:np.float64(0.00070894171949476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016144337132573128) - rollout_corr/log_ppl_diff_max:np.float64(0.014080479741096497) - rollout_corr/log_ppl_diff_min:np.float64(-0.00835387408733368) - rollout_corr/ppl_ratio:np.float64(1.0007117404602468) - rollout_corr/chi2_token:np.float64(0.0016396085266023874) - rollout_corr/chi2_seq:np.float64(0.7452940770890564) - actor/pg_loss:np.float64(0.00023605907335877419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041431330055274884) - actor/ppo_kl:np.float64(0.00019277200499256963) - actor/pg_clipfrac_lower:np.float64(1.5439723938470706e-05) - actor/grad_norm:np.float64(0.27160290628671646) - perf/mfu/actor:np.float64(0.08149403640203684) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.39385986328125) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020954536739736795 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020954536739736795 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:634.51171875 - response_length/max:2697.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:634.51171875 - response_length_non_aborted/max:2697.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.834898471832275e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9801836125552654) - timing_s/agent_loop/generate_sequences/max:np.float64(17.489308523014188) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.853927971802477) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.489308523014188) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2659 - timing_s/gen:23.2441786210984 - timing_s/reward:0.08269395306706429 - timing_s/old_log_prob:2.530390776693821 - timing_s/adv:0.136021314188838 - timing_s/update_actor:10.16875221952796 - timing_s/step:36.25585407577455 - timing_s/stop_profile:0.00013040564954280853 - timing_per_token_ms/update_actor:0.04474206234529957 - timing_per_token_ms/gen:0.14309833854217624 - timing_per_token_ms/adv:0.0005984877975529117 - perf/total_num_tokens:227275 - perf/time_per_step:36.25585407577455 - perf/throughput:783.5803547924854 (TaskRunner pid=2049544) Training Progress: 56%|█████▌ | 56/100 [41:59<30:19, 41.35s/it] (TaskRunner pid=2049544) step:57 - global_seqlen/min:25396 - global_seqlen/max:39989 - global_seqlen/minmax_diff:14593 - global_seqlen/balanced_min:32645 - global_seqlen/balanced_max:32684 - global_seqlen/mean:32673.25 - actor/entropy:0.20889751613140106 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23668746650218964 - training/rollout_probs_diff_mean:0.0030729498248547316 - training/rollout_probs_diff_std:0.009970693849027157 - training/rollout_actor_probs_pearson_corr:0.9990144371986389 - rollout_corr/rollout_is_mean:0.9999489188194275 - rollout_corr/rollout_is_ratio_fraction_high:5.2733157644979656e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.691321035148576e-05 - rollout_corr/rollout_is_max:2.685802698135376 - rollout_corr/rollout_is_min:0.4615940749645233 - rollout_corr/rollout_is_std:0.03016890585422516 - rollout_corr/rollout_is_eff_sample_size:0.9990906647175409 - rollout_corr/rollout_is_seq_mean:0.9998156428337097 - rollout_corr/rollout_is_seq_std:0.0015230864519253373 - rollout_corr/rollout_is_seq_max:1.0045884847640991 - rollout_corr/rollout_is_seq_min:0.9938079118728638 - rollout_corr/rollout_is_seq_max_deviation:0.0061920881271362305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2169137438759208) - rollout_corr/training_log_ppl:np.float64(0.18850286441738717) - rollout_corr/kl:np.float64(0.0008272747242870082) - rollout_corr/k3_kl:np.float64(0.0005567002334601057) - rollout_corr/rollout_ppl:np.float64(1.2158889258280396) - rollout_corr/rollout_log_ppl:np.float64(0.18767558728723088) - rollout_corr/log_ppl_diff:np.float64(0.0008272771301562898) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012787482482963242) - rollout_corr/log_ppl_diff_max:np.float64(0.00949886441230774) - rollout_corr/log_ppl_diff_min:np.float64(-0.003404974937438965) - rollout_corr/ppl_ratio:np.float64(1.000829064520076) - rollout_corr/chi2_token:np.float64(0.0005316368769854307) - rollout_corr/chi2_seq:np.float64(0.33462699479423463) - actor/pg_loss:np.float64(4.014407750219107e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038230796246807586) - actor/ppo_kl:np.float64(0.00015057841544852124) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21619294956326485) - perf/mfu/actor:np.float64(0.09399730782746046) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.41035079956055) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012718578800559044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012718578800559044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:740.7578125 - response_length/max:2402.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:740.7578125 - response_length_non_aborted/max:2402.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010364875197410583 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0282678361982107) - timing_s/agent_loop/generate_sequences/max:np.float64(16.842207921668887) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.293985813957988) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.842207921668887) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:361 - timing_s/agent_loop/slowest/response_length:2402 - timing_s/gen:22.68095701187849 - timing_s/reward:0.07416719943284988 - timing_s/old_log_prob:2.519518541172147 - timing_s/adv:0.15157914347946644 - timing_s/update_actor:10.281581053510308 - timing_s/step:35.79420953243971 - timing_s/stop_profile:0.00011889636516571045 - timing_per_token_ms/update_actor:0.03933485746562673 - timing_per_token_ms/gen:0.11960385274728419 - timing_per_token_ms/adv:0.0005799053640189851 - perf/total_num_tokens:261386 - perf/time_per_step:35.79420953243971 - perf/throughput:912.8082566089012 (TaskRunner pid=2049544) Training Progress: 57%|█████▋ | 57/100 [42:34<28:26, 39.69s/it] (TaskRunner pid=2049544) step:58 - global_seqlen/min:19068 - global_seqlen/max:41428 - global_seqlen/minmax_diff:22360 - global_seqlen/balanced_min:32655 - global_seqlen/balanced_max:33260 - global_seqlen/mean:32761.875 - actor/entropy:0.25287628173828125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42262715101242065 - training/rollout_probs_diff_mean:0.0032485839910805225 - training/rollout_probs_diff_std:0.009784280322492123 - training/rollout_actor_probs_pearson_corr:0.9992220997810364 - rollout_corr/rollout_is_mean:0.9999250769615173 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.117093729088083e-05 - rollout_corr/rollout_is_max:1.704056739807129 - rollout_corr/rollout_is_min:0.30506160855293274 - rollout_corr/rollout_is_std:0.030359944328665733 - rollout_corr/rollout_is_eff_sample_size:0.9990791225700723 - rollout_corr/rollout_is_seq_mean:0.9998817443847656 - rollout_corr/rollout_is_seq_std:0.0017060940153896809 - rollout_corr/rollout_is_seq_max:1.0051213502883911 - rollout_corr/rollout_is_seq_min:0.9917200803756714 - rollout_corr/rollout_is_seq_max_deviation:0.008279919624328613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.248608773574233) - rollout_corr/training_log_ppl:np.float64(0.2106571579060983) - rollout_corr/kl:np.float64(0.0006302413064700474) - rollout_corr/k3_kl:np.float64(0.0005439822076880318) - rollout_corr/rollout_ppl:np.float64(1.2478208504617214) - rollout_corr/rollout_log_ppl:np.float64(0.21002691524336115) - rollout_corr/log_ppl_diff:np.float64(0.000630242662737146) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013482350041158497) - rollout_corr/log_ppl_diff_max:np.float64(0.007139943540096283) - rollout_corr/log_ppl_diff_min:np.float64(-0.009232334792613983) - rollout_corr/ppl_ratio:np.float64(1.0006320539396256) - rollout_corr/chi2_token:np.float64(0.0009030492510646582) - rollout_corr/chi2_seq:np.float64(2.417929224204272) - actor/pg_loss:np.float64(-9.389116894453764e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003193014416638107) - actor/ppo_kl:np.float64(-1.695539558355108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21667541190981865) - perf/mfu/actor:np.float64(0.09071205098155613) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.45818328857422) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013659363612532616 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013659363612532616 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:751.90234375 - response_length/max:4193.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.90234375 - response_length_non_aborted/max:4193.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.30633395910263e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7540517617017031) - timing_s/agent_loop/generate_sequences/max:np.float64(25.78281068429351) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.984791398346715) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.78281068429351) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:4193 - timing_s/gen:31.48099230416119 - timing_s/reward:0.07489402964711189 - timing_s/old_log_prob:2.5185533091425896 - timing_s/adv:0.12432781234383583 - timing_s/update_actor:10.674950456246734 - timing_s/step:44.96027087979019 - timing_s/stop_profile:0.00012622401118278503 - timing_per_token_ms/update_actor:0.04072931744690564 - timing_per_token_ms/gen:0.16354866720433686 - timing_per_token_ms/adv:0.00047436163354446225 - perf/total_num_tokens:262095 - perf/time_per_step:44.96027087979019 - perf/throughput:728.6850003104983 (TaskRunner pid=2049544) Training Progress: 58%|█████▊ | 58/100 [43:19<28:54, 41.29s/it] (TaskRunner pid=2049544) step:59 - global_seqlen/min:23178 - global_seqlen/max:36252 - global_seqlen/minmax_diff:13074 - global_seqlen/balanced_min:28566 - global_seqlen/balanced_max:28612 - global_seqlen/mean:28596.625 - actor/entropy:0.24240879714488983 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797503709793091 - training/rollout_probs_diff_mean:0.0033322961535304785 - training/rollout_probs_diff_std:0.010173420421779156 - training/rollout_actor_probs_pearson_corr:0.9990172982215881 - rollout_corr/rollout_is_mean:1.0001167058944702 - rollout_corr/rollout_is_ratio_fraction_high:6.2438730310532264e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.873162000265438e-05 - rollout_corr/rollout_is_max:2.413045644760132 - rollout_corr/rollout_is_min:0.38240286707878113 - rollout_corr/rollout_is_std:0.03103756159543991 - rollout_corr/rollout_is_eff_sample_size:0.9990379537831949 - rollout_corr/rollout_is_seq_mean:1.0002449750900269 - rollout_corr/rollout_is_seq_std:0.0013697257963940501 - rollout_corr/rollout_is_seq_max:1.003889560699463 - rollout_corr/rollout_is_seq_min:0.9955812692642212 - rollout_corr/rollout_is_seq_max_deviation:0.004418730735778809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2532247374765575) - rollout_corr/training_log_ppl:np.float64(0.21026221416104818) - rollout_corr/kl:np.float64(0.00036410490782223803) - rollout_corr/k3_kl:np.float64(0.0005213920420086282) - rollout_corr/rollout_ppl:np.float64(1.2527254964224994) - rollout_corr/rollout_log_ppl:np.float64(0.20989810848550405) - rollout_corr/log_ppl_diff:np.float64(0.0003641056755441241) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011843454485642724) - rollout_corr/log_ppl_diff_max:np.float64(0.0051978230476379395) - rollout_corr/log_ppl_diff_min:np.float64(-0.004664614796638489) - rollout_corr/ppl_ratio:np.float64(1.000365313142538) - rollout_corr/chi2_token:np.float64(0.0013532442972064018) - rollout_corr/chi2_seq:np.float64(1.167969731381163) - actor/pg_loss:np.float64(5.0825648941099644e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027953678352332645) - actor/ppo_kl:np.float64(0.0001204940012513589) - actor/pg_clipfrac_lower:np.float64(2.0624340777430916e-06) - actor/grad_norm:np.float64(0.2315312847495079) - perf/mfu/actor:np.float64(0.08222550993023878) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56365966796875) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007726792828179896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007726792828179896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:625.61328125 - response_length/max:2175.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:625.61328125 - response_length_non_aborted/max:2175.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.861107587814331e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0195303186774254) - timing_s/agent_loop/generate_sequences/max:np.float64(15.18119196780026) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.312842677085428) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.18119196780026) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:2156 - timing_s/gen:21.4505631364882 - timing_s/reward:0.06381673738360405 - timing_s/old_log_prob:2.4540651980787516 - timing_s/adv:0.12210911512374878 - timing_s/update_actor:10.219576759263873 - timing_s/step:34.34806403145194 - timing_s/stop_profile:0.00012100301682949066 - timing_per_token_ms/update_actor:0.04467125385978185 - timing_per_token_ms/gen:0.13393459628045104 - timing_per_token_ms/adv:0.0005337566720012797 - perf/total_num_tokens:228773 - perf/time_per_step:34.34806403145194 - perf/throughput:832.5542008368959 (TaskRunner pid=2049544) Training Progress: 59%|█████▉ | 59/100 [43:54<26:48, 39.22s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2049544) step:60 - global_seqlen/min:19702 - global_seqlen/max:41022 - global_seqlen/minmax_diff:21320 - global_seqlen/balanced_min:29482 - global_seqlen/balanced_max:29782 - global_seqlen/mean:29534.0 - actor/entropy:0.2471179962158203 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2885955274105072 - training/rollout_probs_diff_mean:0.0032362197525799274 - training/rollout_probs_diff_std:0.009918652474880219 - training/rollout_actor_probs_pearson_corr:0.9991265535354614 - rollout_corr/rollout_is_mean:1.0000226497650146 - rollout_corr/rollout_is_ratio_fraction_high:1.1968021681241225e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1968021681241225e-05 - rollout_corr/rollout_is_max:2.9771621227264404 - rollout_corr/rollout_is_min:0.3293769657611847 - rollout_corr/rollout_is_std:0.03091634064912796 - rollout_corr/rollout_is_eff_sample_size:0.999045211617763 - rollout_corr/rollout_is_seq_mean:0.9999569654464722 - rollout_corr/rollout_is_seq_std:0.0014956871746107936 - rollout_corr/rollout_is_seq_max:1.0054192543029785 - rollout_corr/rollout_is_seq_min:0.9950917959213257 - rollout_corr/rollout_is_seq_max_deviation:0.005419254302978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2401416734792292) - rollout_corr/training_log_ppl:np.float64(0.19959818758798065) - rollout_corr/kl:np.float64(0.0005349192640053957) - rollout_corr/k3_kl:np.float64(0.0004940067541099324) - rollout_corr/rollout_ppl:np.float64(1.2394655891694129) - rollout_corr/rollout_log_ppl:np.float64(0.19906326734781032) - rollout_corr/log_ppl_diff:np.float64(0.0005349202401703224) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011775598395615816) - rollout_corr/log_ppl_diff_max:np.float64(0.005937367677688599) - rollout_corr/log_ppl_diff_min:np.float64(-0.004511777311563492) - rollout_corr/ppl_ratio:np.float64(1.000536180799827) - rollout_corr/chi2_token:np.float64(0.0009251013398170471) - rollout_corr/chi2_seq:np.float64(1.390581222018227) - actor/pg_loss:np.float64(4.893657751381397e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018130719581677113) - actor/ppo_kl:np.float64(-2.2069814953340483e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15835927985608578) - perf/mfu/actor:np.float64(0.08403507336330182) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.51084518432617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6375) - val-aux/sciknoweval/reward/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10267672443906181) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8312625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.052249733074818304) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6821875000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.076862250314467) - val-aux/sciknoweval/score/mean@16:np.float64(0.6375) - val-aux/sciknoweval/score/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10267672443906181) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8312625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.052249733074818304) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6821875000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.076862250314467) - val-core/sciknoweval/acc/mean@16:np.float64(0.6375) - val-aux/sciknoweval/acc/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10267672443906181) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8312625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.052249733074818304) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6821875000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.076862250314467) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0574668098874074) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9954875000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.022722407353494737) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9575250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.07239005753165481) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9770249999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.05686531821374918) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9995499999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004893080696751963) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9285500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08215960170640288) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9901500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03655386464803364) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8929375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08171927324120949) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971125000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.017114100398696087) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8558125000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06612062868202828) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9996750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00411263922012108) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00966417696326971 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.00966417696326971 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:652.78125 - response_length/max:3139.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:652.78125 - response_length_non_aborted/max:3139.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.66247171163559e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2835856825113297) - timing_s/agent_loop/generate_sequences/max:np.float64(20.483323162421584) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.35434487216844) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.483323162421584) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:3139 - timing_s/gen:26.403195202350616 - timing_s/reward:0.06402513571083546 - timing_s/old_log_prob:2.5449625328183174 - timing_s/adv:0.11957781203091145 - timing_s/update_actor:10.040485488250852 - timing_s/step:39.238921258598566 - timing_s/testing:114.50520241633058 - timing_s/save_checkpoint:6.870979869738221 - timing_s/stop_profile:0.00015047751367092133 - timing_per_token_ms/update_actor:0.04249545222561646 - timing_per_token_ms/gen:0.15799700322149587 - timing_per_token_ms/adv:0.0005061023398071351 - perf/total_num_tokens:236272 - perf/time_per_step:39.238921258598566 - perf/throughput:752.6710483542691 (TaskRunner pid=2049544) Training Progress: 60%|██████ | 60/100 [46:34<50:26, 75.65s/it] (TaskRunner pid=2049544) step:61 - global_seqlen/min:22238 - global_seqlen/max:39486 - global_seqlen/minmax_diff:17248 - global_seqlen/balanced_min:29924 - global_seqlen/balanced_max:30089 - global_seqlen/mean:29970.875 - actor/entropy:0.23904968798160553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5477294921875 - training/rollout_probs_diff_mean:0.003164143767207861 - training/rollout_probs_diff_std:0.00985527690500021 - training/rollout_actor_probs_pearson_corr:0.9991676211357117 - rollout_corr/rollout_is_mean:1.0000258684158325 - rollout_corr/rollout_is_ratio_fraction_high:1.1471639481897e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.720745967759285e-05 - rollout_corr/rollout_is_max:4.02189826965332 - rollout_corr/rollout_is_min:0.26447275280952454 - rollout_corr/rollout_is_std:0.030506834387779236 - rollout_corr/rollout_is_eff_sample_size:0.9990700794243639 - rollout_corr/rollout_is_seq_mean:1.0000176429748535 - rollout_corr/rollout_is_seq_std:0.0016575142508372664 - rollout_corr/rollout_is_seq_max:1.0063189268112183 - rollout_corr/rollout_is_seq_min:0.9920255541801453 - rollout_corr/rollout_is_seq_max_deviation:0.007974445819854736 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.25040641753003) - rollout_corr/training_log_ppl:np.float64(0.20799311240261886) - rollout_corr/kl:np.float64(0.0005623800304297433) - rollout_corr/k3_kl:np.float64(0.0005192289710862497) - rollout_corr/rollout_ppl:np.float64(1.2496482259593904) - rollout_corr/rollout_log_ppl:np.float64(0.20743073029007064) - rollout_corr/log_ppl_diff:np.float64(0.0005623821125482209) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012774796996382065) - rollout_corr/log_ppl_diff_max:np.float64(0.009329631924629211) - rollout_corr/log_ppl_diff_min:np.float64(-0.007050305604934692) - rollout_corr/ppl_ratio:np.float64(1.0005640257149935) - rollout_corr/chi2_token:np.float64(0.0009777885861694813) - rollout_corr/chi2_seq:np.float64(0.3470004736445844) - actor/pg_loss:np.float64(-0.0002099055564031005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002631939267985217) - actor/ppo_kl:np.float64(6.142829495781044e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23834596201777458) - perf/mfu/actor:np.float64(0.08600884737044563) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.00771045684814) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007454414386302233 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.007454414386302233 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:681.02734375 - response_length/max:3128.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:681.02734375 - response_length_non_aborted/max:3128.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015644170343875885 - timing_s/agent_loop/generate_sequences/min:np.float64(0.93056246265769) - timing_s/agent_loop/generate_sequences/max:np.float64(20.83841021731496) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.5293213194963755) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.83841021731496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:3128 - timing_s/gen:26.53979305177927 - timing_s/reward:0.08145093731582165 - timing_s/old_log_prob:2.588765997439623 - timing_s/adv:0.11384812742471695 - timing_s/update_actor:10.279441229999065 - timing_s/step:39.70467999763787 - timing_s/stop_profile:0.0001134183257818222 - timing_per_token_ms/update_actor:0.04287262730066717 - timing_per_token_ms/gen:0.1522274656956647 - timing_per_token_ms/adv:0.0004748281766244602 - perf/total_num_tokens:239767 - perf/time_per_step:39.70467999763787 - perf/throughput:754.8448949036497 (TaskRunner pid=2049544) Training Progress: 61%|██████ | 61/100 [47:14<42:10, 64.88s/it] (TaskRunner pid=2049544) step:62 - global_seqlen/min:24330 - global_seqlen/max:43162 - global_seqlen/minmax_diff:18832 - global_seqlen/balanced_min:34742 - global_seqlen/balanced_max:34794 - global_seqlen/mean:34771.375 - actor/entropy:0.2732798457145691 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4058573246002197 - training/rollout_probs_diff_mean:0.00327512389048934 - training/rollout_probs_diff_std:0.009542555548250675 - training/rollout_actor_probs_pearson_corr:0.9991864562034607 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.467516540287761e-05 - rollout_corr/rollout_is_max:1.9362671375274658 - rollout_corr/rollout_is_min:0.37468504905700684 - rollout_corr/rollout_is_std:0.030330481007695198 - rollout_corr/rollout_is_eff_sample_size:0.9990807884305588 - rollout_corr/rollout_is_seq_mean:0.9999213814735413 - rollout_corr/rollout_is_seq_std:0.0014323026407510042 - rollout_corr/rollout_is_seq_max:1.0065138339996338 - rollout_corr/rollout_is_seq_min:0.9906108975410461 - rollout_corr/rollout_is_seq_max_deviation:0.009389102458953857 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2884436114691198) - rollout_corr/training_log_ppl:np.float64(0.23855835630820366) - rollout_corr/kl:np.float64(0.0005924100263499099) - rollout_corr/k3_kl:np.float64(0.0005057925414178044) - rollout_corr/rollout_ppl:np.float64(1.2876463397406042) - rollout_corr/rollout_log_ppl:np.float64(0.23796594546729466) - rollout_corr/log_ppl_diff:np.float64(0.0005924108409089968) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011562938307179138) - rollout_corr/log_ppl_diff_max:np.float64(0.009115807712078094) - rollout_corr/log_ppl_diff_min:np.float64(-0.0043695345520973206) - rollout_corr/ppl_ratio:np.float64(1.0005937188398093) - rollout_corr/chi2_token:np.float64(0.0008554272353649139) - rollout_corr/chi2_seq:np.float64(0.8644126462750137) - actor/pg_loss:np.float64(-5.7160970754921436e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029845362780633877) - actor/ppo_kl:np.float64(3.140159503445261e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22896912693977356) - perf/mfu/actor:np.float64(0.09839975519482527) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0562973022461) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022515371441841125 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022515371441841125 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:798.54296875 - response_length/max:3135.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:798.54296875 - response_length_non_aborted/max:3135.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014179199934005737 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8601270448416471) - timing_s/agent_loop/generate_sequences/max:np.float64(20.932036239653826) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.75760180906218) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.932036239653826) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:3135 - timing_s/gen:27.040857832878828 - timing_s/reward:0.07787876017391682 - timing_s/old_log_prob:2.5947242341935635 - timing_s/adv:0.12395207956433296 - timing_s/update_actor:10.449679136276245 - timing_s/step:40.37802067771554 - timing_s/stop_profile:0.0001217275857925415 - timing_per_token_ms/update_actor:0.03756566693248486 - timing_per_token_ms/gen:0.1322763521104298 - timing_per_token_ms/adv:0.0004455966997434418 - perf/total_num_tokens:278171 - perf/time_per_step:40.37802067771554 - perf/throughput:861.1460991992154 (TaskRunner pid=2049544) Training Progress: 62%|██████▏ | 62/100 [47:55<36:26, 57.55s/it] (TaskRunner pid=2049544) step:63 - global_seqlen/min:21359 - global_seqlen/max:35870 - global_seqlen/minmax_diff:14511 - global_seqlen/balanced_min:26722 - global_seqlen/balanced_max:27254 - global_seqlen/mean:26812.0 - actor/entropy:0.27676644921302795 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9831006526947021 - training/rollout_probs_diff_mean:0.0035199327394366264 - training/rollout_probs_diff_std:0.010530827566981316 - training/rollout_actor_probs_pearson_corr:0.9990900158882141 - rollout_corr/rollout_is_mean:0.9999789595603943 - rollout_corr/rollout_is_ratio_fraction_high:1.3679142284672707e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.419785571168177e-05 - rollout_corr/rollout_is_max:2.8213930130004883 - rollout_corr/rollout_is_min:0.011052249930799007 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990135634724805 - rollout_corr/rollout_is_seq_mean:1.0000073909759521 - rollout_corr/rollout_is_seq_std:0.001668434706516564 - rollout_corr/rollout_is_seq_max:1.0121815204620361 - rollout_corr/rollout_is_seq_min:0.9929383993148804 - rollout_corr/rollout_is_seq_max_deviation:0.012181520462036133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2705106027424335) - rollout_corr/training_log_ppl:np.float64(0.22211689058167394) - rollout_corr/kl:np.float64(0.0004739185238458177) - rollout_corr/k3_kl:np.float64(0.0005098591802266128) - rollout_corr/rollout_ppl:np.float64(1.2698905346915126) - rollout_corr/rollout_log_ppl:np.float64(0.2216429725958733) - rollout_corr/log_ppl_diff:np.float64(0.0004739179858006537) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012847277976106852) - rollout_corr/log_ppl_diff_max:np.float64(0.0076948776841163635) - rollout_corr/log_ppl_diff_min:np.float64(-0.010769776999950409) - rollout_corr/ppl_ratio:np.float64(1.0004755130503327) - rollout_corr/chi2_token:np.float64(0.001110320445150137) - rollout_corr/chi2_seq:np.float64(0.7298937705345452) - actor/pg_loss:np.float64(-6.4569758251309395e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017891856134610862) - actor/ppo_kl:np.float64(-4.981012998683809e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19124767929315567) - perf/mfu/actor:np.float64(0.07698739538780973) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.1104507446289) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018338599475100636 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0018338599475100636 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:571.125 - response_length/max:3010.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.125 - response_length_non_aborted/max:3010.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001445487141609192 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8224132973700762) - timing_s/agent_loop/generate_sequences/max:np.float64(18.348940243944526) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.530061706544075) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.348940243944526) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:3010 - timing_s/gen:25.11938327550888 - timing_s/reward:0.06868074834346771 - timing_s/old_log_prob:2.4906132221221924 - timing_s/adv:0.12919375859200954 - timing_s/update_actor:10.244162807241082 - timing_s/step:38.14616190828383 - timing_s/stop_profile:0.00011005811393260956 - timing_per_token_ms/update_actor:0.047759225380618206 - timing_per_token_ms/gen:0.17180580594433192 - timing_per_token_ms/adv:0.0006023131368044604 - perf/total_num_tokens:214496 - perf/time_per_step:38.14616190828383 - perf/throughput:702.8754312023591 (TaskRunner pid=2049544) Training Progress: 63%|██████▎ | 63/100 [48:33<31:54, 51.74s/it] (TaskRunner pid=2049544) step:64 - global_seqlen/min:26498 - global_seqlen/max:40322 - global_seqlen/minmax_diff:13824 - global_seqlen/balanced_min:31563 - global_seqlen/balanced_max:31618 - global_seqlen/mean:31602.875 - actor/entropy:0.28700023889541626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535178303718567 - training/rollout_probs_diff_mean:0.0033518956042826176 - training/rollout_probs_diff_std:0.009729590266942978 - training/rollout_actor_probs_pearson_corr:0.9992855787277222 - rollout_corr/rollout_is_mean:1.0000327825546265 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0833825399458874e-05 - rollout_corr/rollout_is_max:1.9838535785675049 - rollout_corr/rollout_is_min:0.4158313572406769 - rollout_corr/rollout_is_std:0.03056929260492325 - rollout_corr/rollout_is_eff_sample_size:0.9990665098066491 - rollout_corr/rollout_is_seq_mean:0.9999979734420776 - rollout_corr/rollout_is_seq_std:0.0013619516976177692 - rollout_corr/rollout_is_seq_max:1.0039030313491821 - rollout_corr/rollout_is_seq_min:0.9954256415367126 - rollout_corr/rollout_is_seq_max_deviation:0.0045743584632873535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2764493175782263) - rollout_corr/training_log_ppl:np.float64(0.22693806317693088) - rollout_corr/kl:np.float64(0.00046152420509582015) - rollout_corr/k3_kl:np.float64(0.0005074531937054871) - rollout_corr/rollout_ppl:np.float64(1.275848121382296) - rollout_corr/rollout_log_ppl:np.float64(0.22647653929016087) - rollout_corr/log_ppl_diff:np.float64(0.00046152388677001) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011273344280198216) - rollout_corr/log_ppl_diff_max:np.float64(0.00530514121055603) - rollout_corr/log_ppl_diff_min:np.float64(-0.005483843386173248) - rollout_corr/ppl_ratio:np.float64(1.0004626805894077) - rollout_corr/chi2_token:np.float64(0.0011098254472017288) - rollout_corr/chi2_seq:np.float64(1.8452809148002416) - actor/pg_loss:np.float64(-0.00011544267181307077) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002127377759961746) - actor/ppo_kl:np.float64(-4.476623224114462e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18739718943834305) - perf/mfu/actor:np.float64(0.08994342294998352) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.98906707763672) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013745415955781937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013745415955781937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:721.12109375 - response_length/max:2725.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:721.12109375 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.722448885440826e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.080433465540409) - timing_s/agent_loop/generate_sequences/max:np.float64(17.78822729177773) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.979315571334155) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.78822729177773) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:2560 - timing_s/gen:23.619467133656144 - timing_s/reward:0.07297545671463013 - timing_s/old_log_prob:2.6043873876333237 - timing_s/adv:0.14066699519753456 - timing_s/update_actor:10.391787450760603 - timing_s/step:36.91719730384648 - timing_s/stop_profile:0.0001245010644197464 - timing_per_token_ms/update_actor:0.041103014562601514 - timing_per_token_ms/gen:0.127944591124151 - timing_per_token_ms/adv:0.0005563852782283833 - perf/total_num_tokens:252823 - perf/time_per_step:36.91719730384648 - perf/throughput:856.047514655378 (TaskRunner pid=2049544) Training Progress: 64%|██████▍ | 64/100 [49:10<28:23, 47.32s/it] (TaskRunner pid=2049544) step:65 - global_seqlen/min:13817 - global_seqlen/max:44402 - global_seqlen/minmax_diff:30585 - global_seqlen/balanced_min:27087 - global_seqlen/balanced_max:27398 - global_seqlen/mean:27214.25 - actor/entropy:0.2703191936016083 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730000019073486 - training/rollout_probs_diff_mean:0.003337751142680645 - training/rollout_probs_diff_std:0.009712927974760532 - training/rollout_actor_probs_pearson_corr:0.999212384223938 - rollout_corr/rollout_is_mean:0.9999141097068787 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3306897017173469e-05 - rollout_corr/rollout_is_max:1.8171900510787964 - rollout_corr/rollout_is_min:0.37625378370285034 - rollout_corr/rollout_is_std:0.030340304598212242 - rollout_corr/rollout_is_eff_sample_size:0.999079955499621 - rollout_corr/rollout_is_seq_mean:1.0000232458114624 - rollout_corr/rollout_is_seq_std:0.001954729435965419 - rollout_corr/rollout_is_seq_max:1.0170458555221558 - rollout_corr/rollout_is_seq_min:0.9947062730789185 - rollout_corr/rollout_is_seq_max_deviation:0.01704585552215576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2311511896550655) - rollout_corr/training_log_ppl:np.float64(0.1959650574644911) - rollout_corr/kl:np.float64(0.0005068062779431948) - rollout_corr/k3_kl:np.float64(0.0005566824616494159) - rollout_corr/rollout_ppl:np.float64(1.2305298056453466) - rollout_corr/rollout_log_ppl:np.float64(0.1954582512989873) - rollout_corr/log_ppl_diff:np.float64(0.0005068061655038036) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013243403445812874) - rollout_corr/log_ppl_diff_max:np.float64(0.007357165217399597) - rollout_corr/log_ppl_diff_min:np.float64(-0.020653128623962402) - rollout_corr/ppl_ratio:np.float64(1.0005090332124382) - rollout_corr/chi2_token:np.float64(0.0013379347510635853) - rollout_corr/chi2_seq:np.float64(0.5852078427560627) - actor/pg_loss:np.float64(5.363975651562214e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001393496023638363) - actor/ppo_kl:np.float64(4.77582690940892e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17264313623309135) - perf/mfu/actor:np.float64(0.07762399981838528) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82825469970703) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034032389521598816 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034032389521598816 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:587.1015625 - response_length/max:3738.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:587.1015625 - response_length_non_aborted/max:3738.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.999820470809937e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0192465409636497) - timing_s/agent_loop/generate_sequences/max:np.float64(20.993049459531903) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.240891806744912) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.993049459531903) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:3738 - timing_s/gen:26.803196189925075 - timing_s/reward:0.06635245122015476 - timing_s/old_log_prob:2.5201703682541847 - timing_s/adv:0.12346276640892029 - timing_s/update_actor:10.165898667648435 - timing_s/step:39.771548522636294 - timing_s/stop_profile:0.00011387281119823456 - timing_per_token_ms/update_actor:0.04669382156245549 - timing_per_token_ms/gen:0.17833368501194344 - timing_per_token_ms/adv:0.0005670869416248853 - perf/total_num_tokens:217714 - perf/time_per_step:39.771548522636294 - perf/throughput:684.2642796397729 (TaskRunner pid=2049544) Training Progress: 65%|██████▌ | 65/100 [49:50<26:17, 45.07s/it] (TaskRunner pid=2049544) step:66 - global_seqlen/min:26398 - global_seqlen/max:47072 - global_seqlen/minmax_diff:20674 - global_seqlen/balanced_min:34031 - global_seqlen/balanced_max:34100 - global_seqlen/mean:34083.75 - actor/entropy:0.32043859362602234 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4526241421699524 - training/rollout_probs_diff_mean:0.003443863708525896 - training/rollout_probs_diff_std:0.009328007698059082 - training/rollout_actor_probs_pearson_corr:0.999345064163208 - rollout_corr/rollout_is_mean:0.999977171421051 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9896735466318205e-05 - rollout_corr/rollout_is_max:1.9577182531356812 - rollout_corr/rollout_is_min:0.13161711394786835 - rollout_corr/rollout_is_std:0.030255712568759918 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:1.0000357627868652 - rollout_corr/rollout_is_seq_std:0.0014041649410501122 - rollout_corr/rollout_is_seq_max:1.0082844495773315 - rollout_corr/rollout_is_seq_min:0.9953852295875549 - rollout_corr/rollout_is_seq_max_deviation:0.008284449577331543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3104135780595243) - rollout_corr/training_log_ppl:np.float64(0.25584780907956883) - rollout_corr/kl:np.float64(0.0005620165548441136) - rollout_corr/k3_kl:np.float64(0.0004995921321437891) - rollout_corr/rollout_ppl:np.float64(1.309668519999832) - rollout_corr/rollout_log_ppl:np.float64(0.2552857908885926) - rollout_corr/log_ppl_diff:np.float64(0.0005620181909762323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011583589657675475) - rollout_corr/log_ppl_diff_max:np.float64(0.005684643983840942) - rollout_corr/log_ppl_diff_min:np.float64(-0.006929904222488403) - rollout_corr/ppl_ratio:np.float64(1.000563226873055) - rollout_corr/chi2_token:np.float64(0.0008636738639324903) - rollout_corr/chi2_seq:np.float64(0.8563104090280831) - actor/pg_loss:np.float64(0.0001647728495299816) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017283655381561402) - actor/ppo_kl:np.float64(0.00013175434309165013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1924263872206211) - perf/mfu/actor:np.float64(0.09599912196819169) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.91796875) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000762293697334826 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000762293697334826 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:785.3046875 - response_length/max:3216.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:785.3046875 - response_length_non_aborted/max:3216.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.582083344459534e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2348940391093493) - timing_s/agent_loop/generate_sequences/max:np.float64(21.131182158365846) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.555415921531676) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.131182158365846) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3216 - timing_s/gen:26.812409596517682 - timing_s/reward:0.11229918338358402 - timing_s/old_log_prob:2.537730697542429 - timing_s/adv:0.12344273924827576 - timing_s/update_actor:10.477383820340037 - timing_s/step:40.15382357686758 - timing_s/stop_profile:0.00011853314936161041 - timing_per_token_ms/update_actor:0.03842514328800395 - timing_per_token_ms/gen:0.1333698584174021 - timing_per_token_ms/adv:0.00045271844811778253 - perf/total_num_tokens:272670 - perf/time_per_step:40.15382357686758 - perf/throughput:848.8295002530091 (TaskRunner pid=2049544) Training Progress: 66%|██████▌ | 66/100 [50:30<24:42, 43.61s/it] (TaskRunner pid=2049544) step:67 - global_seqlen/min:14820 - global_seqlen/max:39555 - global_seqlen/minmax_diff:24735 - global_seqlen/balanced_min:25556 - global_seqlen/balanced_max:26212 - global_seqlen/mean:25672.375 - actor/entropy:0.27359628677368164 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46210977435112 - training/rollout_probs_diff_mean:0.0033997194841504097 - training/rollout_probs_diff_std:0.009928795509040356 - training/rollout_actor_probs_pearson_corr:0.9991419315338135 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1833584469277412e-05 - rollout_corr/rollout_is_max:1.9785306453704834 - rollout_corr/rollout_is_min:0.3678695261478424 - rollout_corr/rollout_is_std:0.031190816313028336 - rollout_corr/rollout_is_eff_sample_size:0.9990281975159966 - rollout_corr/rollout_is_seq_mean:1.0001834630966187 - rollout_corr/rollout_is_seq_std:0.002221266273409128 - rollout_corr/rollout_is_seq_max:1.0159006118774414 - rollout_corr/rollout_is_seq_min:0.9894429445266724 - rollout_corr/rollout_is_seq_max_deviation:0.015900611877441406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3023830708116293) - rollout_corr/training_log_ppl:np.float64(0.2422099069954129) - rollout_corr/kl:np.float64(0.00026683149832429365) - rollout_corr/k3_kl:np.float64(0.000541509634160775) - rollout_corr/rollout_ppl:np.float64(1.301996021065861) - rollout_corr/rollout_log_ppl:np.float64(0.241943074419396) - rollout_corr/log_ppl_diff:np.float64(0.0002668325760168955) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014319048641482368) - rollout_corr/log_ppl_diff_max:np.float64(0.009274959564208984) - rollout_corr/log_ppl_diff_min:np.float64(-0.0138968825340271) - rollout_corr/ppl_ratio:np.float64(1.0002689892426133) - rollout_corr/chi2_token:np.float64(0.0016502379439771175) - rollout_corr/chi2_seq:np.float64(1.019609775627032) - actor/pg_loss:np.float64(-5.0529371947050095e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016554813828406623) - actor/ppo_kl:np.float64(-7.827897409384832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17443475499749184) - perf/mfu/actor:np.float64(0.07407053273139065) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82710266113281) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01963839866220951 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01963839866220951 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:536.73046875 - response_length/max:3340.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:536.73046875 - response_length_non_aborted/max:3340.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023074820637702942 - timing_s/agent_loop/generate_sequences/min:np.float64(1.033557141199708) - timing_s/agent_loop/generate_sequences/max:np.float64(19.718276670202613) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.309584286827885) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.718276670202613) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3340 - timing_s/gen:25.747549824416637 - timing_s/reward:0.05820099823176861 - timing_s/old_log_prob:2.587189307436347 - timing_s/adv:0.1540521327406168 - timing_s/update_actor:10.0051255710423 - timing_s/step:38.650182999670506 - timing_s/stop_profile:0.0001270286738872528 - timing_per_token_ms/update_actor:0.04871542646055487 - timing_per_token_ms/gen:0.18738710089602584 - timing_per_token_ms/adv:0.00075008707190422 - perf/total_num_tokens:205379 - perf/time_per_step:38.650182999670506 - perf/throughput:664.2238925548905 (TaskRunner pid=2049544) Training Progress: 67%|██████▋ | 67/100 [51:09<23:14, 42.26s/it] (TaskRunner pid=2049544) step:68 - global_seqlen/min:21420 - global_seqlen/max:36370 - global_seqlen/minmax_diff:14950 - global_seqlen/balanced_min:26998 - global_seqlen/balanced_max:27019 - global_seqlen/mean:27007.25 - actor/entropy:0.2508779466152191 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4620983600616455 - training/rollout_probs_diff_mean:0.003281368175521493 - training/rollout_probs_diff_std:0.009971853345632553 - training/rollout_actor_probs_pearson_corr:0.9991183876991272 - rollout_corr/rollout_is_mean:0.9999003410339355 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.37541350745596e-05 - rollout_corr/rollout_is_max:1.727583885192871 - rollout_corr/rollout_is_min:0.3678779602050781 - rollout_corr/rollout_is_std:0.03009769693017006 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:0.999991238117218 - rollout_corr/rollout_is_seq_std:0.0015605590306222439 - rollout_corr/rollout_is_seq_max:1.005854606628418 - rollout_corr/rollout_is_seq_min:0.9957244396209717 - rollout_corr/rollout_is_seq_max_deviation:0.005854606628417969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2427316303364933) - rollout_corr/training_log_ppl:np.float64(0.2052579416122171) - rollout_corr/kl:np.float64(0.0006245356830429927) - rollout_corr/k3_kl:np.float64(0.0004809370777678623) - rollout_corr/rollout_ppl:np.float64(1.2419258821755648) - rollout_corr/rollout_log_ppl:np.float64(0.20463340327114565) - rollout_corr/log_ppl_diff:np.float64(0.0006245383410714567) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013257119280751795) - rollout_corr/log_ppl_diff_max:np.float64(0.005434125661849976) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038042664527893066) - rollout_corr/ppl_ratio:np.float64(1.0006259840447456) - rollout_corr/chi2_token:np.float64(0.0006594203878194094) - rollout_corr/chi2_seq:np.float64(0.44312575249932706) - actor/pg_loss:np.float64(-5.7110912166535854e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002576871158908034) - actor/ppo_kl:np.float64(0.000146509498037517) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1650708168745041) - perf/mfu/actor:np.float64(0.07864794728282518) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82149505615234) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015104975318536162 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015104975318536162 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:578.6328125 - response_length/max:1942.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:578.6328125 - response_length_non_aborted/max:1942.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.873881936073303e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0787797346711159) - timing_s/agent_loop/generate_sequences/max:np.float64(13.899392530322075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.687431053593173) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.899392530322075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:1942 - timing_s/gen:19.621694235131145 - timing_s/reward:0.06307606771588326 - timing_s/old_log_prob:2.414205303415656 - timing_s/adv:0.1221406627446413 - timing_s/update_actor:10.004864446818829 - timing_s/step:32.31770410388708 - timing_s/stop_profile:4.9777328968048096e-05 - timing_per_token_ms/update_actor:0.04630638276212327 - timing_per_token_ms/gen:0.13246266276332375 - timing_per_token_ms/adv:0.0005653142338846111 - perf/total_num_tokens:216058 - perf/time_per_step:32.31770410388708 - perf/throughput:835.6797225812722 (TaskRunner pid=2049544) Training Progress: 68%|██████▊ | 68/100 [51:41<20:57, 39.29s/it] (TaskRunner pid=2049544) step:69 - global_seqlen/min:23146 - global_seqlen/max:36753 - global_seqlen/minmax_diff:13607 - global_seqlen/balanced_min:29343 - global_seqlen/balanced_max:29374 - global_seqlen/mean:29363.125 - actor/entropy:0.2744224965572357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491748213768005 - training/rollout_probs_diff_mean:0.0033163181506097317 - training/rollout_probs_diff_std:0.00957618560642004 - training/rollout_actor_probs_pearson_corr:0.9992418885231018 - rollout_corr/rollout_is_mean:0.9999679327011108 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9769123792648315 - rollout_corr/rollout_is_min:0.547244668006897 - rollout_corr/rollout_is_std:0.030115514993667603 - rollout_corr/rollout_is_eff_sample_size:0.9990938775276863 - rollout_corr/rollout_is_seq_mean:1.000019907951355 - rollout_corr/rollout_is_seq_std:0.001533227856270969 - rollout_corr/rollout_is_seq_max:1.0057690143585205 - rollout_corr/rollout_is_seq_min:0.990231990814209 - rollout_corr/rollout_is_seq_max_deviation:0.009768009185791016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2841168828308582) - rollout_corr/training_log_ppl:np.float64(0.23235505366756115) - rollout_corr/kl:np.float64(0.0005125461427646361) - rollout_corr/k3_kl:np.float64(0.0005065118215270559) - rollout_corr/rollout_ppl:np.float64(1.2834058650769293) - rollout_corr/rollout_log_ppl:np.float64(0.2318425054399995) - rollout_corr/log_ppl_diff:np.float64(0.0005125482275616378) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012335704232100397) - rollout_corr/log_ppl_diff_max:np.float64(0.011537343263626099) - rollout_corr/log_ppl_diff_min:np.float64(-0.005053341388702393) - rollout_corr/ppl_ratio:np.float64(1.0005140437278897) - rollout_corr/chi2_token:np.float64(0.00102224200963974) - rollout_corr/chi2_seq:np.float64(0.8634938539471477) - actor/pg_loss:np.float64(-5.3855590522289276e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000181908647164164) - actor/ppo_kl:np.float64(5.1798504737021744e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18398911505937576) - perf/mfu/actor:np.float64(0.08486034559374442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82257843017578) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015627924352884293 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015627924352884293 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:646.87890625 - response_length/max:2353.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.87890625 - response_length_non_aborted/max:2353.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.917755722999573e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.203237522393465) - timing_s/agent_loop/generate_sequences/max:np.float64(16.094254648312926) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.346816844648856) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.094254648312926) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:2353 - timing_s/gen:22.094875816255808 - timing_s/reward:0.05950486660003662 - timing_s/old_log_prob:2.4449626710265875 - timing_s/adv:0.12635387293994427 - timing_s/update_actor:10.193446822464466 - timing_s/step:34.9527602866292 - timing_s/stop_profile:2.9163435101509094e-05 - timing_per_token_ms/update_actor:0.04339391167690967 - timing_per_token_ms/gen:0.13342235745107703 - timing_per_token_ms/adv:0.0005378935013726581 - perf/total_num_tokens:234905 - perf/time_per_step:34.9527602866292 - perf/throughput:840.0802900603117 (TaskRunner pid=2049544) Training Progress: 69%|██████▉ | 69/100 [52:16<19:37, 37.99s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2049544) step:70 - global_seqlen/min:18463 - global_seqlen/max:39933 - global_seqlen/minmax_diff:21470 - global_seqlen/balanced_min:27119 - global_seqlen/balanced_max:27177 - global_seqlen/mean:27165.625 - actor/entropy:0.34878426790237427 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35282185673713684 - training/rollout_probs_diff_mean:0.003718312829732895 - training/rollout_probs_diff_std:0.009631498716771603 - training/rollout_actor_probs_pearson_corr:0.999345600605011 - rollout_corr/rollout_is_mean:1.000012993812561 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0160612621111795e-05 - rollout_corr/rollout_is_max:1.975199818611145 - rollout_corr/rollout_is_min:0.19730398058891296 - rollout_corr/rollout_is_std:0.03194605931639671 - rollout_corr/rollout_is_eff_sample_size:0.9989806087004571 - rollout_corr/rollout_is_seq_mean:1.0000489950180054 - rollout_corr/rollout_is_seq_std:0.0017589139752089977 - rollout_corr/rollout_is_seq_max:1.0087381601333618 - rollout_corr/rollout_is_seq_min:0.992647647857666 - rollout_corr/rollout_is_seq_max_deviation:0.008738160133361816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3483932986855507) - rollout_corr/training_log_ppl:np.float64(0.2762820063435356) - rollout_corr/kl:np.float64(0.0006076907076355553) - rollout_corr/k3_kl:np.float64(0.0005228528713416836) - rollout_corr/rollout_ppl:np.float64(1.347519775852561) - rollout_corr/rollout_log_ppl:np.float64(0.27567431312490953) - rollout_corr/log_ppl_diff:np.float64(0.0006076932186260819) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012910107034258544) - rollout_corr/log_ppl_diff_max:np.float64(0.009554415941238403) - rollout_corr/log_ppl_diff_min:np.float64(-0.007596790790557861) - rollout_corr/ppl_ratio:np.float64(1.0006093357224017) - rollout_corr/chi2_token:np.float64(0.0008673274423927069) - rollout_corr/chi2_seq:np.float64(0.4545943159610033) - actor/pg_loss:np.float64(4.701910074800253e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021670004468887782) - actor/ppo_kl:np.float64(0.0001372517156355002) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2546451985836029) - perf/mfu/actor:np.float64(0.07849234372480743) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.89488983154297) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09773915943586196) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8506) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07051862491369472) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6758124999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06817587534083876) - val-aux/sciknoweval/score/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/score/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09773915943586196) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8506) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07051862491369472) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6758124999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06817587534083876) - val-core/sciknoweval/acc/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09773915943586196) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8506) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07051862491369472) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6758124999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06817587534083876) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.041294709008662854) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9961125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.016882665811088163) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.966925) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04991881605672479) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9813124999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04115652748695172) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004110435910939457) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9449500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05482807368910221) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9914250000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.027274560942092924) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.92125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05037233272292474) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.01160302730045884) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8982125) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.037001893862708676) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9991249999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.005012406355781736) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005754174781031907 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005754174781031907 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:581.26953125 - response_length/max:2622.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:581.26953125 - response_length_non_aborted/max:2622.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7524307370185852e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.118723189458251) - timing_s/agent_loop/generate_sequences/max:np.float64(16.3854166790843) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.583953251349158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.3854166790843) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:2622 - timing_s/gen:22.14735895767808 - timing_s/reward:0.0681693684309721 - timing_s/old_log_prob:2.4311879333108664 - timing_s/adv:0.11663812026381493 - timing_s/update_actor:9.943649336695671 - timing_s/step:34.796333791688085 - timing_s/testing:115.27732796221972 - timing_s/save_checkpoint:6.6401550360023975 - timing_s/stop_profile:0.000131901353597641 - timing_per_token_ms/update_actor:0.04575474214515436 - timing_per_token_ms/gen:0.1488347767728106 - timing_per_token_ms/adv:0.0005366990464227075 - perf/total_num_tokens:217325 - perf/time_per_step:34.796333791688085 - perf/throughput:780.7036558112666 (TaskRunner pid=2049544) Training Progress: 70%|███████ | 70/100 [54:53<36:48, 73.62s/it] (TaskRunner pid=2049544) step:71 - global_seqlen/min:20204 - global_seqlen/max:38376 - global_seqlen/minmax_diff:18172 - global_seqlen/balanced_min:28759 - global_seqlen/balanced_max:29243 - global_seqlen/mean:28856.5 - actor/entropy:0.32371971011161804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35350361466407776 - training/rollout_probs_diff_mean:0.003485669381916523 - training/rollout_probs_diff_std:0.009408628568053246 - training/rollout_actor_probs_pearson_corr:0.9993824362754822 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:6.257665518205613e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.257665518205613e-06 - rollout_corr/rollout_is_max:3.121652126312256 - rollout_corr/rollout_is_min:0.4736731946468353 - rollout_corr/rollout_is_std:0.03072100132703781 - rollout_corr/rollout_is_eff_sample_size:0.9990571099353601 - rollout_corr/rollout_is_seq_mean:1.0000032186508179 - rollout_corr/rollout_is_seq_std:0.0015029723290354013 - rollout_corr/rollout_is_seq_max:1.0061728954315186 - rollout_corr/rollout_is_seq_min:0.9937267303466797 - rollout_corr/rollout_is_seq_max_deviation:0.0062732696533203125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3052765680477023) - rollout_corr/training_log_ppl:np.float64(0.24597317739971913) - rollout_corr/kl:np.float64(0.00043000688371730433) - rollout_corr/k3_kl:np.float64(0.000540082649621354) - rollout_corr/rollout_ppl:np.float64(1.3047043732367456) - rollout_corr/rollout_log_ppl:np.float64(0.2455431708949618) - rollout_corr/log_ppl_diff:np.float64(0.0004300065047573298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012972943659406155) - rollout_corr/log_ppl_diff_max:np.float64(0.00797504186630249) - rollout_corr/log_ppl_diff_min:np.float64(-0.0062088742852211) - rollout_corr/ppl_ratio:np.float64(1.0004315802361816) - rollout_corr/chi2_token:np.float64(0.0013474705629050732) - rollout_corr/chi2_seq:np.float64(1.4930782679002732) - actor/pg_loss:np.float64(0.0001349709928035736) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038389957057916035) - actor/ppo_kl:np.float64(-3.772802561208266e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17576251924037933) - perf/mfu/actor:np.float64(0.08244675822096596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2421646118164) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009262127801775932 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009262127801775932 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:624.234375 - response_length/max:3404.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:624.234375 - response_length_non_aborted/max:3404.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012946873903274536 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2765609622001648) - timing_s/agent_loop/generate_sequences/max:np.float64(20.10214041545987) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.958212114004709) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.10214041545987) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:3404 - timing_s/gen:25.802896425127983 - timing_s/reward:0.07092655450105667 - timing_s/old_log_prob:2.582106838002801 - timing_s/adv:0.1316581778228283 - timing_s/update_actor:10.102314312011003 - timing_s/step:38.781780099496245 - timing_s/stop_profile:0.00012562982738018036 - timing_per_token_ms/update_actor:0.04376099974014089 - timing_per_token_ms/gen:0.16146589838256853 - timing_per_token_ms/adv:0.0005703142178661146 - perf/total_num_tokens:230852 - perf/time_per_step:38.781780099496245 - perf/throughput:744.0736326689355 (TaskRunner pid=2049544) Training Progress: 71%|███████ | 71/100 [55:32<30:32, 63.19s/it] (TaskRunner pid=2049544) step:72 - global_seqlen/min:24019 - global_seqlen/max:35372 - global_seqlen/minmax_diff:11353 - global_seqlen/balanced_min:30681 - global_seqlen/balanced_max:31067 - global_seqlen/mean:30756.75 - actor/entropy:0.2652339041233063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5334677696228027 - training/rollout_probs_diff_mean:0.003048401791602373 - training/rollout_probs_diff_std:0.009212449193000793 - training/rollout_actor_probs_pearson_corr:0.9992727637290955 - rollout_corr/rollout_is_mean:1.0000463724136353 - rollout_corr/rollout_is_ratio_fraction_high:1.1159842870256398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.6739762941142544e-05 - rollout_corr/rollout_is_max:2.703545570373535 - rollout_corr/rollout_is_min:0.2786954641342163 - rollout_corr/rollout_is_std:0.02862177975475788 - rollout_corr/rollout_is_eff_sample_size:0.9991814643117487 - rollout_corr/rollout_is_seq_mean:1.0000512599945068 - rollout_corr/rollout_is_seq_std:0.0012803305871784687 - rollout_corr/rollout_is_seq_max:1.005553960800171 - rollout_corr/rollout_is_seq_min:0.9965124726295471 - rollout_corr/rollout_is_seq_max_deviation:0.0055539608001708984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.272761202417314) - rollout_corr/training_log_ppl:np.float64(0.2287760855979286) - rollout_corr/kl:np.float64(0.0003352667156821809) - rollout_corr/k3_kl:np.float64(0.0004365081685619998) - rollout_corr/rollout_ppl:np.float64(1.2723114336840808) - rollout_corr/rollout_log_ppl:np.float64(0.22844081881339662) - rollout_corr/log_ppl_diff:np.float64(0.00033526678453199565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010565653501544148) - rollout_corr/log_ppl_diff_max:np.float64(0.00969955325126648) - rollout_corr/log_ppl_diff_min:np.float64(-0.005003869533538818) - rollout_corr/ppl_ratio:np.float64(1.0003363261930645) - rollout_corr/chi2_token:np.float64(0.0010922125075012445) - rollout_corr/chi2_seq:np.float64(1.312244726344943) - actor/pg_loss:np.float64(8.467817679047585e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000129777238043971) - actor/ppo_kl:np.float64(-3.5065210759910315e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1640554666519165) - perf/mfu/actor:np.float64(0.08815191325739583) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.06222915649414) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006141400896012783 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006141400896012783 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:700.0546875 - response_length/max:3313.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:700.0546875 - response_length_non_aborted/max:3313.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013216771185398102 - timing_s/agent_loop/generate_sequences/min:np.float64(2.07568240724504) - timing_s/agent_loop/generate_sequences/max:np.float64(20.67868852801621) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.946097736770753) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.67868852801621) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:3313 - timing_s/gen:26.445967487990856 - timing_s/reward:0.0720598753541708 - timing_s/old_log_prob:2.519180826842785 - timing_s/adv:0.12123813852667809 - timing_s/update_actor:10.210091643035412 - timing_s/step:39.45813396573067 - timing_s/stop_profile:0.00012371689081192017 - timing_per_token_ms/update_actor:0.04149532884259314 - timing_per_token_ms/gen:0.14756641494520994 - timing_per_token_ms/adv:0.0004927298012902781 - perf/total_num_tokens:246054 - perf/time_per_step:39.45813396573067 - perf/throughput:779.4780672272083 (TaskRunner pid=2049544) Training Progress: 72%|███████▏ | 72/100 [56:11<26:10, 56.08s/it] (TaskRunner pid=2049544) step:73 - global_seqlen/min:21925 - global_seqlen/max:46347 - global_seqlen/minmax_diff:24422 - global_seqlen/balanced_min:36563 - global_seqlen/balanced_max:36628 - global_seqlen/mean:36610.375 - actor/entropy:0.3390262722969055 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43396425247192383 - training/rollout_probs_diff_mean:0.0032661668956279755 - training/rollout_probs_diff_std:0.008885001763701439 - training/rollout_actor_probs_pearson_corr:0.9994308948516846 - rollout_corr/rollout_is_mean:0.9999864101409912 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7025687813875265e-05 - rollout_corr/rollout_is_max:1.9699597358703613 - rollout_corr/rollout_is_min:0.2820616662502289 - rollout_corr/rollout_is_std:0.029777150601148605 - rollout_corr/rollout_is_eff_sample_size:0.9991141068069422 - rollout_corr/rollout_is_seq_mean:0.9999426007270813 - rollout_corr/rollout_is_seq_std:0.0015595207223668694 - rollout_corr/rollout_is_seq_max:1.0064419507980347 - rollout_corr/rollout_is_seq_min:0.992620587348938 - rollout_corr/rollout_is_seq_max_deviation:0.007379412651062012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4027120932005346) - rollout_corr/training_log_ppl:np.float64(0.31371801071509253) - rollout_corr/kl:np.float64(0.000663529713321509) - rollout_corr/k3_kl:np.float64(0.0005334890627182176) - rollout_corr/rollout_ppl:np.float64(1.4016525098122656) - rollout_corr/rollout_log_ppl:np.float64(0.3130544779851334) - rollout_corr/log_ppl_diff:np.float64(0.0006635327299591154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013385674392338842) - rollout_corr/log_ppl_diff_max:np.float64(0.008049964904785156) - rollout_corr/log_ppl_diff_min:np.float64(-0.005723848938941956) - rollout_corr/ppl_ratio:np.float64(1.0006653110031039) - rollout_corr/chi2_token:np.float64(0.0007862187922000885) - rollout_corr/chi2_seq:np.float64(0.7408852751832455) - actor/pg_loss:np.float64(-9.118067100644112e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025507958866910485) - actor/ppo_kl:np.float64(9.864521248204028e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22031916305422783) - perf/mfu/actor:np.float64(0.10239718785831436) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18194198608398) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006957425735890865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006957425735890865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:867.23046875 - response_length/max:3153.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:867.23046875 - response_length_non_aborted/max:3153.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010944902896881104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9543074388056993) - timing_s/agent_loop/generate_sequences/max:np.float64(21.861840976402164) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.284751043880533) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.861840976402164) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:3153 - timing_s/gen:27.61041198298335 - timing_s/reward:0.07980250380933285 - timing_s/old_log_prob:2.6316978093236685 - timing_s/adv:0.14758859016001225 - timing_s/update_actor:10.437388250604272 - timing_s/step:40.99476144276559 - timing_s/stop_profile:0.00011257268488407135 - timing_per_token_ms/update_actor:0.03563671585788274 - timing_per_token_ms/gen:0.12436506291572648 - timing_per_token_ms/adv:0.00050391654742683 - perf/total_num_tokens:292883 - perf/time_per_step:40.99476144276559 - perf/throughput:893.0500803404648 (TaskRunner pid=2049544) Training Progress: 73%|███████▎ | 73/100 [56:52<23:12, 51.57s/it] (TaskRunner pid=2049544) step:74 - global_seqlen/min:18401 - global_seqlen/max:45796 - global_seqlen/minmax_diff:27395 - global_seqlen/balanced_min:30702 - global_seqlen/balanced_max:30981 - global_seqlen/mean:30746.0 - actor/entropy:0.31715160608291626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4399150013923645 - training/rollout_probs_diff_mean:0.0034521878696978092 - training/rollout_probs_diff_std:0.009627916850149632 - training/rollout_actor_probs_pearson_corr:0.9993271231651306 - rollout_corr/rollout_is_mean:1.0001097917556763 - rollout_corr/rollout_is_ratio_fraction_high:1.1016612916137092e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.652492028370034e-05 - rollout_corr/rollout_is_max:2.2787067890167236 - rollout_corr/rollout_is_min:0.41993895173072815 - rollout_corr/rollout_is_std:0.030713239684700966 - rollout_corr/rollout_is_eff_sample_size:0.9990577048586795 - rollout_corr/rollout_is_seq_mean:1.000206708908081 - rollout_corr/rollout_is_seq_std:0.002247885102406144 - rollout_corr/rollout_is_seq_max:1.0164759159088135 - rollout_corr/rollout_is_seq_min:0.9938144683837891 - rollout_corr/rollout_is_seq_max_deviation:0.016475915908813477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3208666159771383) - rollout_corr/training_log_ppl:np.float64(0.2606007744325325) - rollout_corr/kl:np.float64(0.0003702222969650393) - rollout_corr/k3_kl:np.float64(0.000720645587279023) - rollout_corr/rollout_ppl:np.float64(1.3202885040082037) - rollout_corr/rollout_log_ppl:np.float64(0.26023055279802065) - rollout_corr/log_ppl_diff:np.float64(0.0003702216345118359) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016211537149501964) - rollout_corr/log_ppl_diff_max:np.float64(0.01012013852596283) - rollout_corr/log_ppl_diff_min:np.float64(-0.012066826224327087) - rollout_corr/ppl_ratio:np.float64(1.0003729804884642) - rollout_corr/chi2_token:np.float64(0.0021484598983079195) - rollout_corr/chi2_seq:np.float64(2.6817988054826856) - actor/pg_loss:np.float64(-0.00016311672516167164) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024571236758674786) - actor/ppo_kl:np.float64(-5.575577448624358e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21021737158298492) - perf/mfu/actor:np.float64(0.08564326289574913) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.05369186401367) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008819486945867538 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008819486945867538 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:709.15625 - response_length/max:4610.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:709.15625 - response_length_non_aborted/max:4610.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.643472731113434e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.46839356422424316) - timing_s/agent_loop/generate_sequences/max:np.float64(26.743163952603936) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.783247412800847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.743163952603936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:4610 - timing_s/gen:33.11821131967008 - timing_s/reward:0.07101857103407383 - timing_s/old_log_prob:2.5942629985511303 - timing_s/adv:0.12228076718747616 - timing_s/update_actor:10.489771209657192 - timing_s/step:46.487874053418636 - timing_s/stop_profile:0.00010906346142292023 - timing_per_token_ms/update_actor:0.042646893944160186 - timing_per_token_ms/gen:0.18242525954958622 - timing_per_token_ms/adv:0.0004971409581225044 - perf/total_num_tokens:245968 - perf/time_per_step:46.487874053418636 - perf/throughput:661.3767703093963 (TaskRunner pid=2049544) Training Progress: 74%|███████▍ | 74/100 [57:39<21:41, 50.06s/it] (TaskRunner pid=2049544) step:75 - global_seqlen/min:19402 - global_seqlen/max:28818 - global_seqlen/minmax_diff:9416 - global_seqlen/balanced_min:23785 - global_seqlen/balanced_max:23996 - global_seqlen/mean:23814.375 - actor/entropy:0.25116294622421265 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135130643844604 - training/rollout_probs_diff_mean:0.0031402192544192076 - training/rollout_probs_diff_std:0.009609016589820385 - training/rollout_actor_probs_pearson_corr:0.9991532564163208 - rollout_corr/rollout_is_mean:1.0000470876693726 - rollout_corr/rollout_is_ratio_fraction_high:8.058602361415979e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4175806174753234e-05 - rollout_corr/rollout_is_max:2.1670117378234863 - rollout_corr/rollout_is_min:0.4087337255477905 - rollout_corr/rollout_is_std:0.03037172183394432 - rollout_corr/rollout_is_eff_sample_size:0.9990784086315643 - rollout_corr/rollout_is_seq_mean:1.0000663995742798 - rollout_corr/rollout_is_seq_std:0.0019097818294540048 - rollout_corr/rollout_is_seq_max:1.0100950002670288 - rollout_corr/rollout_is_seq_min:0.9921054244041443 - rollout_corr/rollout_is_seq_max_deviation:0.010095000267028809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2536835842765868) - rollout_corr/training_log_ppl:np.float64(0.20439182232803432) - rollout_corr/kl:np.float64(0.00044191136634275097) - rollout_corr/k3_kl:np.float64(0.0004956063722261206) - rollout_corr/rollout_ppl:np.float64(1.2531235865317285) - rollout_corr/rollout_log_ppl:np.float64(0.20394990891509224) - rollout_corr/log_ppl_diff:np.float64(0.0004419134129420854) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013838859085808508) - rollout_corr/log_ppl_diff_max:np.float64(0.009681947529315948) - rollout_corr/log_ppl_diff_min:np.float64(-0.005931958556175232) - rollout_corr/ppl_ratio:np.float64(1.0004437156021595) - rollout_corr/chi2_token:np.float64(0.0011102964635938406) - rollout_corr/chi2_seq:np.float64(0.39504921343177557) - actor/pg_loss:np.float64(1.7428305000066757e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018432266847412393) - actor/ppo_kl:np.float64(3.7105337884923983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16733924113214016) - perf/mfu/actor:np.float64(0.06918613903772251) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.97526931762695) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008511898340657353 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008511898340657353 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:484.73046875 - response_length/max:2220.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:484.73046875 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016090832650661469 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2748903073370457) - timing_s/agent_loop/generate_sequences/max:np.float64(13.663885425776243) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.775116956734564) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.663885425776243) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2220 - timing_s/gen:19.10645161010325 - timing_s/reward:0.06184021010994911 - timing_s/old_log_prob:2.374053070321679 - timing_s/adv:0.11829666048288345 - timing_s/update_actor:9.632479302585125 - timing_s/step:31.360355062410235 - timing_s/stop_profile:3.0426308512687683e-05 - timing_per_token_ms/update_actor:0.05056021469482783 - timing_per_token_ms/gen:0.15397129211710156 - timing_per_token_ms/adv:0.0006209309528534942 - perf/total_num_tokens:190515 - perf/time_per_step:31.360355062410235 - perf/throughput:759.3783601176395 (TaskRunner pid=2049544) Training Progress: 75%|███████▌ | 75/100 [58:10<18:31, 44.46s/it] (TaskRunner pid=2049544) step:76 - global_seqlen/min:25932 - global_seqlen/max:42011 - global_seqlen/minmax_diff:16079 - global_seqlen/balanced_min:31987 - global_seqlen/balanced_max:32015 - global_seqlen/mean:32005.0 - actor/entropy:0.35323843359947205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8143891096115112 - training/rollout_probs_diff_mean:0.003405920695513487 - training/rollout_probs_diff_std:0.009273294359445572 - training/rollout_actor_probs_pearson_corr:0.9994474053382874 - rollout_corr/rollout_is_mean:1.0001150369644165 - rollout_corr/rollout_is_ratio_fraction_high:1.081595564755844e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.081595564755844e-05 - rollout_corr/rollout_is_max:3.5232980251312256 - rollout_corr/rollout_is_min:0.16788354516029358 - rollout_corr/rollout_is_std:0.030408985912799835 - rollout_corr/rollout_is_eff_sample_size:0.9990763858113328 - rollout_corr/rollout_is_seq_mean:1.0001401901245117 - rollout_corr/rollout_is_seq_std:0.0016239474061876535 - rollout_corr/rollout_is_seq_max:1.010636329650879 - rollout_corr/rollout_is_seq_min:0.9931896328926086 - rollout_corr/rollout_is_seq_max_deviation:0.010636329650878906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3853302095085382) - rollout_corr/training_log_ppl:np.float64(0.3024687277356861) - rollout_corr/kl:np.float64(0.0005689695157187202) - rollout_corr/k3_kl:np.float64(0.0005752426737899441) - rollout_corr/rollout_ppl:np.float64(1.38448647223413) - rollout_corr/rollout_log_ppl:np.float64(0.30189975767279975) - rollout_corr/log_ppl_diff:np.float64(0.0005689700628863648) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012543359625851735) - rollout_corr/log_ppl_diff_max:np.float64(0.007834061980247498) - rollout_corr/log_ppl_diff_min:np.float64(-0.006428062915802002) - rollout_corr/ppl_ratio:np.float64(1.0005704772192985) - rollout_corr/chi2_token:np.float64(0.0012044101022183895) - rollout_corr/chi2_seq:np.float64(1.3872064049355686) - actor/pg_loss:np.float64(5.75545709580183e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002793574865336268) - actor/ppo_kl:np.float64(0.00019198946114684645) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3208663538098335) - perf/mfu/actor:np.float64(0.09134199443488283) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.03634643554688) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008177538402378559 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008177538402378559 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:722.3125 - response_length/max:2396.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:722.3125 - response_length_non_aborted/max:2396.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.119371831417084e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.99646220728755) - timing_s/agent_loop/generate_sequences/max:np.float64(16.732603766024113) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.14587117084011) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.732603766024113) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:306 - timing_s/agent_loop/slowest/response_length:2371 - timing_s/gen:22.66927962191403 - timing_s/reward:0.07421145401895046 - timing_s/old_log_prob:2.5292600374668837 - timing_s/adv:0.1157302763313055 - timing_s/update_actor:10.305034121498466 - timing_s/step:35.780901934951544 - timing_s/stop_profile:0.00012325309216976166 - timing_per_token_ms/update_actor:0.040247750826036814 - timing_per_token_ms/gen:0.12259496204634653 - timing_per_token_ms/adv:0.00045200076679934975 - perf/total_num_tokens:256040 - perf/time_per_step:35.780901934951544 - perf/throughput:894.471583141867 (TaskRunner pid=2049544) Training Progress: 76%|███████▌ | 76/100 [58:46<16:44, 41.87s/it] (TaskRunner pid=2049544) step:77 - global_seqlen/min:16244 - global_seqlen/max:41465 - global_seqlen/minmax_diff:25221 - global_seqlen/balanced_min:29420 - global_seqlen/balanced_max:29517 - global_seqlen/mean:29492.5 - actor/entropy:0.3045775294303894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915552258491516 - training/rollout_probs_diff_mean:0.0032992891501635313 - training/rollout_probs_diff_std:0.009267176501452923 - training/rollout_actor_probs_pearson_corr:0.9993472695350647 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:5.8924742916133255e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.140592098236084 - rollout_corr/rollout_is_min:0.5226767659187317 - rollout_corr/rollout_is_std:0.029582349583506584 - rollout_corr/rollout_is_eff_sample_size:0.9991255307618554 - rollout_corr/rollout_is_seq_mean:1.0000417232513428 - rollout_corr/rollout_is_seq_std:0.0017432396998628974 - rollout_corr/rollout_is_seq_max:1.008344054222107 - rollout_corr/rollout_is_seq_min:0.9936603307723999 - rollout_corr/rollout_is_seq_max_deviation:0.008344054222106934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2967352382838726) - rollout_corr/training_log_ppl:np.float64(0.24739519167633262) - rollout_corr/kl:np.float64(0.0005334063558146696) - rollout_corr/k3_kl:np.float64(0.0005630067024355867) - rollout_corr/rollout_ppl:np.float64(1.2960236202925444) - rollout_corr/rollout_log_ppl:np.float64(0.24686178589763585) - rollout_corr/log_ppl_diff:np.float64(0.0005334057786967605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014061912952456623) - rollout_corr/log_ppl_diff_max:np.float64(0.01065683364868164) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076304227113723755) - rollout_corr/ppl_ratio:np.float64(1.0005355728790164) - rollout_corr/chi2_token:np.float64(0.0011709912214428186) - rollout_corr/chi2_seq:np.float64(0.44916587066836655) - actor/pg_loss:np.float64(2.6725465431809425e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018346831632243266) - actor/ppo_kl:np.float64(9.181251817746983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16028263419866562) - perf/mfu/actor:np.float64(0.08482457572663438) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.09542083740234) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006819802336394787 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006819802336394787 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:662.921875 - response_length/max:2972.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:662.921875 - response_length_non_aborted/max:2972.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013144128024578094 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8779960200190544) - timing_s/agent_loop/generate_sequences/max:np.float64(18.480581056326628) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.09187524789013) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.480581056326628) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2972 - timing_s/gen:24.529541220515966 - timing_s/reward:0.07079425640404224 - timing_s/old_log_prob:2.487271063029766 - timing_s/adv:0.11753782816231251 - timing_s/update_actor:10.21386044844985 - timing_s/step:37.50457299686968 - timing_s/stop_profile:2.83215194940567e-05 - timing_per_token_ms/update_actor:0.043290075648257396 - timing_per_token_ms/gen:0.1445396871126639 - timing_per_token_ms/adv:0.0004981682977126071 - perf/total_num_tokens:235940 - perf/time_per_step:37.50457299686968 - perf/throughput:786.3707714379681 (TaskRunner pid=2049544) Training Progress: 77%|███████▋ | 77/100 [59:24<15:33, 40.57s/it] (TaskRunner pid=2049544) step:78 - global_seqlen/min:16539 - global_seqlen/max:34942 - global_seqlen/minmax_diff:18403 - global_seqlen/balanced_min:26643 - global_seqlen/balanced_max:26991 - global_seqlen/mean:26772.625 - actor/entropy:0.324239581823349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30065640807151794 - training/rollout_probs_diff_mean:0.003476528450846672 - training/rollout_probs_diff_std:0.009609517641365528 - training/rollout_actor_probs_pearson_corr:0.9993136525154114 - rollout_corr/rollout_is_mean:0.9999270439147949 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.365252956020413e-05 - rollout_corr/rollout_is_max:1.927759051322937 - rollout_corr/rollout_is_min:0.17178574204444885 - rollout_corr/rollout_is_std:0.031049082055687904 - rollout_corr/rollout_is_eff_sample_size:0.9990367639843101 - rollout_corr/rollout_is_seq_mean:0.9997465014457703 - rollout_corr/rollout_is_seq_std:0.0017120024422183633 - rollout_corr/rollout_is_seq_max:1.0067331790924072 - rollout_corr/rollout_is_seq_min:0.9950670599937439 - rollout_corr/rollout_is_seq_max_deviation:0.0067331790924072266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026048908941448) - rollout_corr/training_log_ppl:np.float64(0.24134253649390303) - rollout_corr/kl:np.float64(0.0007427987794557112) - rollout_corr/k3_kl:np.float64(0.0005388006466660045) - rollout_corr/rollout_ppl:np.float64(1.301613979972899) - rollout_corr/rollout_log_ppl:np.float64(0.24059973818657454) - rollout_corr/log_ppl_diff:np.float64(0.000742798307328485) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014510368491755798) - rollout_corr/log_ppl_diff_max:np.float64(0.0068367719650268555) - rollout_corr/log_ppl_diff_min:np.float64(-0.005617767572402954) - rollout_corr/ppl_ratio:np.float64(1.0007447020616382) - rollout_corr/chi2_token:np.float64(0.0006613847799599171) - rollout_corr/chi2_seq:np.float64(0.6144026103429496) - actor/pg_loss:np.float64(5.164649337530136e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011791538452143868) - actor/ppo_kl:np.float64(-3.060101226282086e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16195594891905785) - perf/mfu/actor:np.float64(0.07693655141257658) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.07821655273438) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012876042164862156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012876042164862156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:572.23828125 - response_length/max:3226.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:572.23828125 - response_length_non_aborted/max:3226.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.397596836090088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2444690354168415) - timing_s/agent_loop/generate_sequences/max:np.float64(19.064789429306984) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.27938631172583) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.064789429306984) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:3226 - timing_s/gen:24.83233668655157 - timing_s/reward:0.06715011782944202 - timing_s/old_log_prob:2.454452695325017 - timing_s/adv:0.12591312639415264 - timing_s/update_actor:10.206827508285642 - timing_s/step:37.77338271215558 - timing_s/stop_profile:0.00012371130287647247 - timing_per_token_ms/update_actor:0.04765514918823631 - timing_per_token_ms/gen:0.1695121042408277 - timing_per_token_ms/adv:0.0005878818681122632 - perf/total_num_tokens:214181 - perf/time_per_step:37.77338271215558 - perf/throughput:708.7695905875142 (TaskRunner pid=2049544) Training Progress: 78%|███████▊ | 78/100 [1:00:02<14:34, 39.74s/it] (TaskRunner pid=2049544) step:79 - global_seqlen/min:20249 - global_seqlen/max:46165 - global_seqlen/minmax_diff:25916 - global_seqlen/balanced_min:30262 - global_seqlen/balanced_max:30358 - global_seqlen/mean:30336.375 - actor/entropy:0.27736997604370117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4035380482673645 - training/rollout_probs_diff_mean:0.0030377230141311884 - training/rollout_probs_diff_std:0.009076771326363087 - training/rollout_actor_probs_pearson_corr:0.9993468523025513 - rollout_corr/rollout_is_mean:0.9998963475227356 - rollout_corr/rollout_is_ratio_fraction_high:5.790153409179766e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.211137977312319e-05 - rollout_corr/rollout_is_max:4.029329299926758 - rollout_corr/rollout_is_min:0.07581690698862076 - rollout_corr/rollout_is_std:0.02898186817765236 - rollout_corr/rollout_is_eff_sample_size:0.9991605182489611 - rollout_corr/rollout_is_seq_mean:0.9999415874481201 - rollout_corr/rollout_is_seq_std:0.0017486511496827006 - rollout_corr/rollout_is_seq_max:1.0085891485214233 - rollout_corr/rollout_is_seq_min:0.9941185712814331 - rollout_corr/rollout_is_seq_max_deviation:0.00858914852142334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2839201726019382) - rollout_corr/training_log_ppl:np.float64(0.2279481466684956) - rollout_corr/kl:np.float64(0.0005260294094213691) - rollout_corr/k3_kl:np.float64(0.0005059725925065095) - rollout_corr/rollout_ppl:np.float64(1.2831925540231168) - rollout_corr/rollout_log_ppl:np.float64(0.2274221143743489) - rollout_corr/log_ppl_diff:np.float64(0.0005260322941467166) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012799371324945241) - rollout_corr/log_ppl_diff_max:np.float64(0.008957505226135254) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052579790353775024) - rollout_corr/ppl_ratio:np.float64(1.0005276040174067) - rollout_corr/chi2_token:np.float64(0.0009765762370079756) - rollout_corr/chi2_seq:np.float64(0.7638932466506958) - actor/pg_loss:np.float64(0.00014414661563932896) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017305412563928257) - actor/ppo_kl:np.float64(-2.993323226707645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20632774755358696) - perf/mfu/actor:np.float64(0.08763345813710535) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.01940155029297) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008221293799579144 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008221293799579144 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:674.63671875 - response_length/max:2725.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:674.63671875 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010620616376399994 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1851273197680712) - timing_s/agent_loop/generate_sequences/max:np.float64(17.865634333342314) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.507378077250905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.865634333342314) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:2725 - timing_s/gen:23.788950365036726 - timing_s/reward:0.07290799915790558 - timing_s/old_log_prob:2.4474152121692896 - timing_s/adv:0.1101104523986578 - timing_s/update_actor:10.210453910753131 - timing_s/step:36.71885127387941 - timing_s/stop_profile:0.00012296251952648163 - timing_per_token_ms/update_actor:0.04207182759456729 - timing_per_token_ms/gen:0.13774166863553142 - timing_per_token_ms/adv:0.0004537063689986765 - perf/total_num_tokens:242691 - perf/time_per_step:36.71885127387941 - perf/throughput:826.1798489752948 (TaskRunner pid=2049544) Training Progress: 79%|███████▉ | 79/100 [1:00:38<13:35, 38.85s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) - $ E = 2.0 \, \text{N/C} $ (TaskRunner pid=2049544) - $ r = 50 \, \text{cm} = 0.50 \, \text{m} $ (TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2049544) step:80 - global_seqlen/min:18167 - global_seqlen/max:36780 - global_seqlen/minmax_diff:18613 - global_seqlen/balanced_min:28127 - global_seqlen/balanced_max:28499 - global_seqlen/mean:28202.25 - actor/entropy:0.23095937073230743 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44294559955596924 - training/rollout_probs_diff_mean:0.002918842015787959 - training/rollout_probs_diff_std:0.0094361687079072 - training/rollout_actor_probs_pearson_corr:0.9991447329521179 - rollout_corr/rollout_is_mean:0.9999209046363831 - rollout_corr/rollout_is_ratio_fraction_high:1.9175455236108974e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9175455236108974e-05 - rollout_corr/rollout_is_max:2.320603370666504 - rollout_corr/rollout_is_min:0.33252665400505066 - rollout_corr/rollout_is_std:0.028494464233517647 - rollout_corr/rollout_is_eff_sample_size:0.999188605215701 - rollout_corr/rollout_is_seq_mean:0.9999192953109741 - rollout_corr/rollout_is_seq_std:0.001614381093531847 - rollout_corr/rollout_is_seq_max:1.0071684122085571 - rollout_corr/rollout_is_seq_min:0.9931057691574097 - rollout_corr/rollout_is_seq_max_deviation:0.007168412208557129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2288768324069679) - rollout_corr/training_log_ppl:np.float64(0.19351409275259357) - rollout_corr/kl:np.float64(0.0005485934744235976) - rollout_corr/k3_kl:np.float64(0.0005014402207166313) - rollout_corr/rollout_ppl:np.float64(1.2281699823215604) - rollout_corr/rollout_log_ppl:np.float64(0.1929654997366015) - rollout_corr/log_ppl_diff:np.float64(0.0005485930159920827) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012800928816432133) - rollout_corr/log_ppl_diff_max:np.float64(0.00843721628189087) - rollout_corr/log_ppl_diff_min:np.float64(-0.006566546857357025) - rollout_corr/ppl_ratio:np.float64(1.0005502616986632) - rollout_corr/chi2_token:np.float64(0.0008651481475681067) - rollout_corr/chi2_seq:np.float64(0.5386446032207459) - actor/pg_loss:np.float64(-2.616771962493658e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001737377756398928) - actor/ppo_kl:np.float64(2.498516215898583e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1959109902381897) - perf/mfu/actor:np.float64(0.08114905418503308) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.12442779541016) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10302193213109104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8416625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04059727090755087) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.683725) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05843863672565752) - val-aux/sciknoweval/score/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/score/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10302193213109104) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8416625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04059727090755087) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.683725) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05843863672565752) - val-core/sciknoweval/acc/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/acc/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10302193213109104) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8416625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04059727090755087) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.683725) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05843863672565752) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.975) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.06354165489638078) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.996175) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.023966893234937735) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9539) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.08044933488834552) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9746749999999998) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06396939198866787) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998125000000002) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003145810047315943) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9212250000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.09233866134852486) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9907999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03956667018273212) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8777874999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08983250151225204) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9978250000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.018387073504573324) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8370125000000002) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0672232524014538) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998249999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004527947017243354) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006327900337055326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006327900337055326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:611.1328125 - response_length/max:2845.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:611.1328125 - response_length_non_aborted/max:2845.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000109061598777771 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1131607126444578) - timing_s/agent_loop/generate_sequences/max:np.float64(17.91330724582076) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.142933668168553) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.91330724582076) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:2845 - timing_s/gen:23.700068602338433 - timing_s/reward:0.06908616237342358 - timing_s/old_log_prob:2.398565139621496 - timing_s/adv:0.14442053809762 - timing_s/update_actor:10.117256434634328 - timing_s/step:36.51560699753463 - timing_s/testing:115.6058545038104 - timing_s/save_checkpoint:6.825648196041584 - timing_s/stop_profile:0.00023227371275424957 - timing_per_token_ms/update_actor:0.04484241698195325 - timing_per_token_ms/gen:0.15148653628851666 - timing_per_token_ms/adv:0.0006401108869754187 - perf/total_num_tokens:225618 - perf/time_per_step:36.51560699753463 - perf/throughput:772.3341419986278 (TaskRunner pid=2049544) Training Progress: 80%|████████ | 80/100 [1:03:17<24:57, 74.90s/it] (TaskRunner pid=2049544) step:81 - global_seqlen/min:22002 - global_seqlen/max:45026 - global_seqlen/minmax_diff:23024 - global_seqlen/balanced_min:31986 - global_seqlen/balanced_max:32078 - global_seqlen/mean:32047.5 - actor/entropy:0.30386513471603394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135500192642212 - training/rollout_probs_diff_mean:0.003210825379937887 - training/rollout_probs_diff_std:0.00900829304009676 - training/rollout_actor_probs_pearson_corr:0.9993411898612976 - rollout_corr/rollout_is_mean:1.0000028610229492 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263490493234713e-06 - rollout_corr/rollout_is_max:1.9500842094421387 - rollout_corr/rollout_is_min:0.46151214838027954 - rollout_corr/rollout_is_std:0.029131613671779633 - rollout_corr/rollout_is_eff_sample_size:0.9991520686654056 - rollout_corr/rollout_is_seq_mean:0.9999109506607056 - rollout_corr/rollout_is_seq_std:0.0019725700840353966 - rollout_corr/rollout_is_seq_max:1.0081653594970703 - rollout_corr/rollout_is_seq_min:0.9903681874275208 - rollout_corr/rollout_is_seq_max_deviation:0.009631812572479248 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3244363577105105) - rollout_corr/training_log_ppl:np.float64(0.2665938182544778) - rollout_corr/kl:np.float64(0.0010818528244627146) - rollout_corr/k3_kl:np.float64(0.0008180101519030814) - rollout_corr/rollout_ppl:np.float64(1.3230207040905952) - rollout_corr/rollout_log_ppl:np.float64(0.26551196482614614) - rollout_corr/log_ppl_diff:np.float64(0.0010818534283316694) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018223601437057368) - rollout_corr/log_ppl_diff_max:np.float64(0.01393115520477295) - rollout_corr/log_ppl_diff_min:np.float64(-0.010075211524963379) - rollout_corr/ppl_ratio:np.float64(1.0010864462237805) - rollout_corr/chi2_token:np.float64(0.0015508977230638266) - rollout_corr/chi2_seq:np.float64(1.0036336749326438) - actor/pg_loss:np.float64(7.214734796434641e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002760089199682625) - actor/ppo_kl:np.float64(0.0004848889264348344) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21540894359350204) - perf/mfu/actor:np.float64(0.0899665554810477) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27264404296875) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004168026614934206 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004168026614934206 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:742.140625 - response_length/max:3366.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:742.140625 - response_length_non_aborted/max:3366.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.953603148460388e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6883454918861389) - timing_s/agent_loop/generate_sequences/max:np.float64(21.4869614392519) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.608749296545284) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.4869614392519) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:3366 - timing_s/gen:27.376880433410406 - timing_s/reward:0.07466047257184982 - timing_s/old_log_prob:2.649170648306608 - timing_s/adv:0.12651739083230495 - timing_s/update_actor:10.34231029637158 - timing_s/step:40.673489751294255 - timing_s/stop_profile:0.00011551380157470703 - timing_per_token_ms/update_actor:0.04033977024873851 - timing_per_token_ms/gen:0.1440979453092322 - timing_per_token_ms/adv:0.000493476054420411 - perf/total_num_tokens:256380 - perf/time_per_step:40.673489751294255 - perf/throughput:787.921080683278 (TaskRunner pid=2049544) Training Progress: 81%|████████ | 81/100 [1:03:58<20:28, 64.65s/it] (TaskRunner pid=2049544) step:82 - global_seqlen/min:25371 - global_seqlen/max:44114 - global_seqlen/minmax_diff:18743 - global_seqlen/balanced_min:33520 - global_seqlen/balanced_max:33584 - global_seqlen/mean:33563.875 - actor/entropy:0.2920665442943573 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.913862407207489 - training/rollout_probs_diff_mean:0.0031529199331998825 - training/rollout_probs_diff_std:0.009444354102015495 - training/rollout_actor_probs_pearson_corr:0.9993178248405457 - rollout_corr/rollout_is_mean:0.9999819993972778 - rollout_corr/rollout_is_ratio_fraction_high:2.0209470676491037e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.547131175058894e-05 - rollout_corr/rollout_is_max:2.713327169418335 - rollout_corr/rollout_is_min:0.049278318881988525 - rollout_corr/rollout_is_std:0.02968292124569416 - rollout_corr/rollout_is_eff_sample_size:0.9991195807527394 - rollout_corr/rollout_is_seq_mean:0.9999279379844666 - rollout_corr/rollout_is_seq_std:0.0012865856988355517 - rollout_corr/rollout_is_seq_max:1.004623532295227 - rollout_corr/rollout_is_seq_min:0.9942681193351746 - rollout_corr/rollout_is_seq_max_deviation:0.0057318806648254395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3072700020857155) - rollout_corr/training_log_ppl:np.float64(0.24861305610102136) - rollout_corr/kl:np.float64(0.0007281105056762271) - rollout_corr/k3_kl:np.float64(0.0005667303261134293) - rollout_corr/rollout_ppl:np.float64(1.306269062217325) - rollout_corr/rollout_log_ppl:np.float64(0.2478849448962137) - rollout_corr/log_ppl_diff:np.float64(0.0007281112048076466) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011902832047780976) - rollout_corr/log_ppl_diff_max:np.float64(0.012343257665634155) - rollout_corr/log_ppl_diff_min:np.float64(-0.00298440083861351) - rollout_corr/ppl_ratio:np.float64(1.0007296884432435) - rollout_corr/chi2_token:np.float64(0.0007541878148913383) - rollout_corr/chi2_seq:np.float64(0.7926566984970123) - actor/pg_loss:np.float64(0.0001337927533313632) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021164456336464355) - actor/ppo_kl:np.float64(0.00017838788005519746) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1409779703244567) - perf/mfu/actor:np.float64(0.09575702121449964) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.184974193573) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00978454202413559 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00978454202413559 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:773.15234375 - response_length/max:2703.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:773.15234375 - response_length_non_aborted/max:2703.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.613321602344513e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2882147133350372) - timing_s/agent_loop/generate_sequences/max:np.float64(18.499180192127824) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.74611435400584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.499180192127824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:240 - timing_s/agent_loop/slowest/response_length:2703 - timing_s/gen:24.018707640469074 - timing_s/reward:0.07196270488202572 - timing_s/old_log_prob:2.514760909602046 - timing_s/adv:0.12197888456285 - timing_s/update_actor:10.373511025682092 - timing_s/step:37.19780015014112 - timing_s/stop_profile:3.86945903301239e-05 - timing_per_token_ms/update_actor:0.03863346762584063 - timing_per_token_ms/gen:0.12135134489215253 - timing_per_token_ms/adv:0.0004542789105952829 - perf/total_num_tokens:268511 - perf/time_per_step:37.19780015014112 - perf/throughput:902.3080629641123 (TaskRunner pid=2049544) Training Progress: 82%|████████▏ | 82/100 [1:04:35<16:55, 56.42s/it] (TaskRunner pid=2049544) step:83 - global_seqlen/min:19276 - global_seqlen/max:41937 - global_seqlen/minmax_diff:22661 - global_seqlen/balanced_min:29583 - global_seqlen/balanced_max:29645 - global_seqlen/mean:29615.125 - actor/entropy:0.34415289759635925 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5622131824493408 - training/rollout_probs_diff_mean:0.003517036559060216 - training/rollout_probs_diff_std:0.009729686193168163 - training/rollout_actor_probs_pearson_corr:0.9993311762809753 - rollout_corr/rollout_is_mean:0.9999920129776001 - rollout_corr/rollout_is_ratio_fraction_high:1.2048701137246098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0121751478873193e-05 - rollout_corr/rollout_is_max:8.23111629486084 - rollout_corr/rollout_is_min:0.2471064180135727 - rollout_corr/rollout_is_std:0.03159145638346672 - rollout_corr/rollout_is_eff_sample_size:0.9990024989924954 - rollout_corr/rollout_is_seq_mean:0.9999005794525146 - rollout_corr/rollout_is_seq_std:0.0016829889500513673 - rollout_corr/rollout_is_seq_max:1.004620909690857 - rollout_corr/rollout_is_seq_min:0.9928184747695923 - rollout_corr/rollout_is_seq_max_deviation:0.007181525230407715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.372836111113429) - rollout_corr/training_log_ppl:np.float64(0.2916473155346466) - rollout_corr/kl:np.float64(0.0007178986236571205) - rollout_corr/k3_kl:np.float64(0.0006351841151968074) - rollout_corr/rollout_ppl:np.float64(1.3718020445667207) - rollout_corr/rollout_log_ppl:np.float64(0.2909294152777875) - rollout_corr/log_ppl_diff:np.float64(0.0007179002568591386) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014387322589755058) - rollout_corr/log_ppl_diff_max:np.float64(0.009183712303638458) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071634650230407715) - rollout_corr/ppl_ratio:np.float64(1.0007200299296528) - rollout_corr/chi2_token:np.float64(0.0011453672777861357) - rollout_corr/chi2_seq:np.float64(0.5700722692999989) - actor/pg_loss:np.float64(0.00010708440095186234) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003794109481987107) - actor/ppo_kl:np.float64(5.79457945377726e-05) - actor/pg_clipfrac_lower:np.float64(4.787071247847052e-06) - actor/grad_norm:np.float64(0.27837222814559937) - perf/mfu/actor:np.float64(0.08514566987298042) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15073013305664) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003308121347799897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003308121347799897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.41015625 - response_length/max:2853.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.41015625 - response_length_non_aborted/max:2853.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.170570850372314e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8073275871574879) - timing_s/agent_loop/generate_sequences/max:np.float64(18.502459589391947) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.3145277952207834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.502459589391947) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:2853 - timing_s/gen:24.35667880065739 - timing_s/reward:0.10609085857868195 - timing_s/old_log_prob:2.4029372967779636 - timing_s/adv:0.11332383565604687 - timing_s/update_actor:10.112380102276802 - timing_s/step:37.18459020368755 - timing_s/stop_profile:3.2028183341026306e-05 - timing_per_token_ms/update_actor:0.042682497973066134 - timing_per_token_ms/gen:0.14673316827009206 - timing_per_token_ms/adv:0.0004783190838129455 - perf/total_num_tokens:236921 - perf/time_per_step:37.18459020368755 - perf/throughput:796.4354276267675 (TaskRunner pid=2049544) Training Progress: 83%|████████▎ | 83/100 [1:05:12<14:21, 50.66s/it] (TaskRunner pid=2049544) step:84 - global_seqlen/min:25752 - global_seqlen/max:42945 - global_seqlen/minmax_diff:17193 - global_seqlen/balanced_min:33932 - global_seqlen/balanced_max:34032 - global_seqlen/mean:33969.0 - actor/entropy:0.3499566912651062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6138913631439209 - training/rollout_probs_diff_mean:0.0033732259180396795 - training/rollout_probs_diff_std:0.009146650321781635 - training/rollout_actor_probs_pearson_corr:0.9994776844978333 - rollout_corr/rollout_is_mean:0.9999618530273438 - rollout_corr/rollout_is_ratio_fraction_high:4.901384272670839e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4506920453859493e-05 - rollout_corr/rollout_is_max:2.404789447784424 - rollout_corr/rollout_is_min:0.2902381718158722 - rollout_corr/rollout_is_std:0.03017878346145153 - rollout_corr/rollout_is_eff_sample_size:0.9990900697549667 - rollout_corr/rollout_is_seq_mean:1.0001087188720703 - rollout_corr/rollout_is_seq_std:0.001655240310356021 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9946045279502869 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3845429411157966) - rollout_corr/training_log_ppl:np.float64(0.29742183088092133) - rollout_corr/kl:np.float64(0.00039445633141355074) - rollout_corr/k3_kl:np.float64(0.0005463047522766828) - rollout_corr/rollout_ppl:np.float64(1.3839370189234614) - rollout_corr/rollout_log_ppl:np.float64(0.29702737282786984) - rollout_corr/log_ppl_diff:np.float64(0.00039445805305149406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012936670245835558) - rollout_corr/log_ppl_diff_max:np.float64(0.00665166974067688) - rollout_corr/log_ppl_diff_min:np.float64(-0.005310043692588806) - rollout_corr/ppl_ratio:np.float64(1.000396022805944) - rollout_corr/chi2_token:np.float64(0.0013724297750741243) - rollout_corr/chi2_seq:np.float64(1.7039004366379231) - actor/pg_loss:np.float64(0.00012169475667178631) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021371153513882746) - actor/ppo_kl:np.float64(1.5052716766383867e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2066413126885891) - perf/mfu/actor:np.float64(0.09535977782956703) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16190719604492) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017204470932483673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017204470932483673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:796.96875 - response_length/max:3594.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:796.96875 - response_length_non_aborted/max:3594.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7237460017204285e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9937344156205654) - timing_s/agent_loop/generate_sequences/max:np.float64(23.47761689312756) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.72609610089421) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.47761689312756) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:3594 - timing_s/gen:29.4396906606853 - timing_s/reward:0.07473043724894524 - timing_s/old_log_prob:2.594569843262434 - timing_s/adv:0.12372583709657192 - timing_s/update_actor:10.489614827558398 - timing_s/step:42.81442074663937 - timing_s/stop_profile:2.9224902391433716e-05 - timing_per_token_ms/update_actor:0.038599954471571135 - timing_per_token_ms/gen:0.14429523321121682 - timing_per_token_ms/adv:0.000455289518003812 - perf/total_num_tokens:271752 - perf/time_per_step:42.81442074663937 - perf/throughput:793.4009010892977 (TaskRunner pid=2049544) Training Progress: 84%|████████▍ | 84/100 [1:05:55<12:52, 48.31s/it] (TaskRunner pid=2049544) step:85 - global_seqlen/min:20755 - global_seqlen/max:54433 - global_seqlen/minmax_diff:33678 - global_seqlen/balanced_min:33282 - global_seqlen/balanced_max:33493 - global_seqlen/mean:33418.125 - actor/entropy:0.29106423258781433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967511296272278 - training/rollout_probs_diff_mean:0.003082117298617959 - training/rollout_probs_diff_std:0.009240406565368176 - training/rollout_actor_probs_pearson_corr:0.9993290901184082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:4.992785306967562e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.9971141227870248e-05 - rollout_corr/rollout_is_max:2.3519351482391357 - rollout_corr/rollout_is_min:0.26859748363494873 - rollout_corr/rollout_is_std:0.029412612318992615 - rollout_corr/rollout_is_eff_sample_size:0.9991358839466532 - rollout_corr/rollout_is_seq_mean:0.9999825954437256 - rollout_corr/rollout_is_seq_std:0.0015941840829327703 - rollout_corr/rollout_is_seq_max:1.0068665742874146 - rollout_corr/rollout_is_seq_min:0.9917056560516357 - rollout_corr/rollout_is_seq_max_deviation:0.008294343948364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2718443693593144) - rollout_corr/training_log_ppl:np.float64(0.22443578000093112) - rollout_corr/kl:np.float64(0.0005144908149414817) - rollout_corr/k3_kl:np.float64(0.0004981835849378058) - rollout_corr/rollout_ppl:np.float64(1.2711914060637355) - rollout_corr/rollout_log_ppl:np.float64(0.22392128901265096) - rollout_corr/log_ppl_diff:np.float64(0.0005144909882801585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011518053579493426) - rollout_corr/log_ppl_diff_max:np.float64(0.009792208671569824) - rollout_corr/log_ppl_diff_min:np.float64(-0.005565732717514038) - rollout_corr/ppl_ratio:np.float64(1.000515874940902) - rollout_corr/chi2_token:np.float64(0.000967925414443016) - rollout_corr/chi2_seq:np.float64(1.0396802069153637) - actor/pg_loss:np.float64(-0.0001277067931368947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000293373741897085) - actor/ppo_kl:np.float64(1.70920517623685e-05) - actor/pg_clipfrac_lower:np.float64(5.778476406703703e-06) - actor/grad_norm:np.float64(0.20695016533136368) - perf/mfu/actor:np.float64(0.09336689611238912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0907974243164) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014600777067244053 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014600777067244053 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:782.37890625 - response_length/max:4261.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:782.37890625 - response_length_non_aborted/max:4261.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.2804364562034607e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7515706215053797) - timing_s/agent_loop/generate_sequences/max:np.float64(26.178023107349873) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.071110863718786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.178023107349873) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:4261 - timing_s/gen:31.903686491772532 - timing_s/reward:0.07636177726089954 - timing_s/old_log_prob:2.5928792282938957 - timing_s/adv:0.12288173846900463 - timing_s/update_actor:10.525410840287805 - timing_s/step:45.31749305874109 - timing_s/stop_profile:0.00011776015162467957 - timing_per_token_ms/update_actor:0.039370142850204064 - timing_per_token_ms/gen:0.15928826092183063 - timing_per_token_ms/adv:0.0004596373168340707 - perf/total_num_tokens:267345 - perf/time_per_step:45.31749305874109 - perf/throughput:737.4221905144448 (TaskRunner pid=2049544) Training Progress: 85%|████████▌ | 85/100 [1:06:41<11:51, 47.43s/it] (TaskRunner pid=2049544) step:86 - global_seqlen/min:22104 - global_seqlen/max:49662 - global_seqlen/minmax_diff:27558 - global_seqlen/balanced_min:36725 - global_seqlen/balanced_max:36775 - global_seqlen/mean:36762.375 - actor/entropy:0.3210241496562958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24132895469665527 - training/rollout_probs_diff_mean:0.003310859901830554 - training/rollout_probs_diff_std:0.009120251052081585 - training/rollout_actor_probs_pearson_corr:0.9993811845779419 - rollout_corr/rollout_is_mean:0.9998584389686584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2271813577390276e-05 - rollout_corr/rollout_is_max:1.9132159948349 - rollout_corr/rollout_is_min:0.41626274585723877 - rollout_corr/rollout_is_std:0.029449068009853363 - rollout_corr/rollout_is_eff_sample_size:0.9991331468769348 - rollout_corr/rollout_is_seq_mean:0.9998147487640381 - rollout_corr/rollout_is_seq_std:0.0013719926355406642 - rollout_corr/rollout_is_seq_max:1.0041395425796509 - rollout_corr/rollout_is_seq_min:0.9918246865272522 - rollout_corr/rollout_is_seq_max_deviation:0.008175313472747803 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.318041059654206) - rollout_corr/training_log_ppl:np.float64(0.26210865678149275) - rollout_corr/kl:np.float64(0.0005536363921763154) - rollout_corr/k3_kl:np.float64(0.0004948169941201286) - rollout_corr/rollout_ppl:np.float64(1.317284729797393) - rollout_corr/rollout_log_ppl:np.float64(0.2615550198825076) - rollout_corr/log_ppl_diff:np.float64(0.0005536368989851326) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010979093494825065) - rollout_corr/log_ppl_diff_max:np.float64(0.0066015273332595825) - rollout_corr/log_ppl_diff_min:np.float64(-0.002927526831626892) - rollout_corr/ppl_ratio:np.float64(1.0005547057371587) - rollout_corr/chi2_token:np.float64(0.0008581595029681921) - rollout_corr/chi2_seq:np.float64(0.48298407695256174) - actor/pg_loss:np.float64(0.00010274478700011969) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012036708881169034) - actor/ppo_kl:np.float64(-9.579434273687681e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1470967587083578) - perf/mfu/actor:np.float64(0.1030748564778111) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2135009765625) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007343573961406946 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007343573961406946 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:876.94921875 - response_length/max:3278.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:876.94921875 - response_length_non_aborted/max:3278.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010039471089839935 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4529647156596184) - timing_s/agent_loop/generate_sequences/max:np.float64(22.549825344234705) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.41551528852142) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.549825344234705) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:3278 - timing_s/gen:28.685686687007546 - timing_s/reward:0.07788657583296299 - timing_s/old_log_prob:2.5779322646558285 - timing_s/adv:0.1303798407316208 - timing_s/update_actor:10.655327502638102 - timing_s/step:42.21963391080499 - timing_s/stop_profile:0.00011177733540534973 - timing_per_token_ms/update_actor:0.036230410516996325 - timing_per_token_ms/gen:0.1277764564074118 - timing_per_token_ms/adv:0.0004433195649479284 - perf/total_num_tokens:294099 - perf/time_per_step:42.21963391080499 - perf/throughput:870.7412072228237 (TaskRunner pid=2049544) Training Progress: 86%|████████▌ | 86/100 [1:07:23<10:42, 45.88s/it] (TaskRunner pid=2049544) step:87 - global_seqlen/min:29440 - global_seqlen/max:47768 - global_seqlen/minmax_diff:18328 - global_seqlen/balanced_min:35517 - global_seqlen/balanced_max:36384 - global_seqlen/mean:35692.75 - actor/entropy:0.3466551899909973 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292303800582886 - training/rollout_probs_diff_mean:0.003489995375275612 - training/rollout_probs_diff_std:0.00928904302418232 - training/rollout_actor_probs_pearson_corr:0.9994551539421082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.662482751882635e-06 - rollout_corr/rollout_is_max:1.702067494392395 - rollout_corr/rollout_is_min:0.26140108704566956 - rollout_corr/rollout_is_std:0.03009967692196369 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:1.0000789165496826 - rollout_corr/rollout_is_seq_std:0.001175415120087564 - rollout_corr/rollout_is_seq_max:1.0036451816558838 - rollout_corr/rollout_is_seq_min:0.9961031079292297 - rollout_corr/rollout_is_seq_max_deviation:0.0038968920707702637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.384202916175127) - rollout_corr/training_log_ppl:np.float64(0.2980515792296501) - rollout_corr/kl:np.float64(0.00042411336519165843) - rollout_corr/k3_kl:np.float64(0.00048466515880818406) - rollout_corr/rollout_ppl:np.float64(1.3836216540075839) - rollout_corr/rollout_log_ppl:np.float64(0.2976274649699917) - rollout_corr/log_ppl_diff:np.float64(0.00042411425965838134) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010522096417844296) - rollout_corr/log_ppl_diff_max:np.float64(0.004691094160079956) - rollout_corr/log_ppl_diff_min:np.float64(-0.005741387605667114) - rollout_corr/ppl_ratio:np.float64(1.0004250849597156) - rollout_corr/chi2_token:np.float64(0.001095710089430213) - rollout_corr/chi2_seq:np.float64(1.8486225868109614) - actor/pg_loss:np.float64(7.82626448199153e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016853076499501185) - actor/ppo_kl:np.float64(3.361720549453773e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18693841621279716) - perf/mfu/actor:np.float64(0.0996124497047341) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18145751953125) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006155060138553381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006155060138553381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:837.8046875 - response_length/max:4550.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:837.8046875 - response_length_non_aborted/max:4550.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001092962920665741 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4342339634895325) - timing_s/agent_loop/generate_sequences/max:np.float64(28.061727035790682) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.031970594347513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.061727035790682) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:239 - timing_s/agent_loop/slowest/response_length:4550 - timing_s/gen:33.89907926507294 - timing_s/reward:0.07478075660765171 - timing_s/old_log_prob:2.563651405274868 - timing_s/adv:0.12060143612325191 - timing_s/update_actor:10.524738831445575 - timing_s/step:47.28383187763393 - timing_s/stop_profile:0.00016075000166893005 - timing_per_token_ms/update_actor:0.036858811773558965 - timing_per_token_ms/gen:0.15805387622540745 - timing_per_token_ms/adv:0.00042235970933611136 - perf/total_num_tokens:285542 - perf/time_per_step:47.28383187763393 - perf/throughput:754.8616214601526 (TaskRunner pid=2049544) Training Progress: 87%|████████▋ | 87/100 [1:08:10<10:02, 46.32s/it] (TaskRunner pid=2049544) step:88 - global_seqlen/min:22456 - global_seqlen/max:46812 - global_seqlen/minmax_diff:24356 - global_seqlen/balanced_min:35428 - global_seqlen/balanced_max:35596 - global_seqlen/mean:35553.625 - actor/entropy:0.3236216604709625 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21511626243591309 - training/rollout_probs_diff_mean:0.0032852201256901026 - training/rollout_probs_diff_std:0.009024620987474918 - training/rollout_actor_probs_pearson_corr:0.999454140663147 - rollout_corr/rollout_is_mean:1.0000885725021362 - rollout_corr/rollout_is_ratio_fraction_high:4.604963123711059e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.604963123711059e-06 - rollout_corr/rollout_is_max:2.0104920864105225 - rollout_corr/rollout_is_min:0.23549965023994446 - rollout_corr/rollout_is_std:0.029910963028669357 - rollout_corr/rollout_is_eff_sample_size:0.999106252989479 - rollout_corr/rollout_is_seq_mean:1.0000383853912354 - rollout_corr/rollout_is_seq_std:0.0014727648813277483 - rollout_corr/rollout_is_seq_max:1.0041821002960205 - rollout_corr/rollout_is_seq_min:0.9893348813056946 - rollout_corr/rollout_is_seq_max_deviation:0.01066511869430542 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338182927109301) - rollout_corr/training_log_ppl:np.float64(0.2705517132417299) - rollout_corr/kl:np.float64(0.0005242268059881816) - rollout_corr/k3_kl:np.float64(0.000508854052498009) - rollout_corr/rollout_ppl:np.float64(1.3374784495681524) - rollout_corr/rollout_log_ppl:np.float64(0.27002748603990767) - rollout_corr/log_ppl_diff:np.float64(0.0005242272018222138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011741442867787555) - rollout_corr/log_ppl_diff_max:np.float64(0.013874843716621399) - rollout_corr/log_ppl_diff_min:np.float64(-0.0045333802700042725) - rollout_corr/ppl_ratio:np.float64(1.0005257281009108) - rollout_corr/chi2_token:np.float64(0.0009958110749721527) - rollout_corr/chi2_seq:np.float64(1.3633345067501068) - actor/pg_loss:np.float64(1.946359407156706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016658261313295952) - actor/ppo_kl:np.float64(7.023494554836418e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13375208340585232) - perf/mfu/actor:np.float64(0.09886611268499279) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15729522705078) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02385486103594303 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02385486103594303 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:848.26953125 - response_length/max:4442.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:848.26953125 - response_length_non_aborted/max:4442.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.980984032154083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.990953391417861) - timing_s/agent_loop/generate_sequences/max:np.float64(25.68886986002326) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.6535395380997215) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.68886986002326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:4442 - timing_s/gen:31.64418187364936 - timing_s/reward:0.07790577039122581 - timing_s/old_log_prob:2.6047119572758675 - timing_s/adv:0.11337987333536148 - timing_s/update_actor:10.512858772650361 - timing_s/step:45.041199665516615 - timing_s/stop_profile:0.00016839616000652313 - timing_per_token_ms/update_actor:0.036961276004381975 - timing_per_token_ms/gen:0.1457202939516081 - timing_per_token_ms/adv:0.00039862276116486536 - perf/total_num_tokens:284429 - perf/time_per_step:45.041199665516615 - perf/throughput:789.3578604483693 (TaskRunner pid=2049544) Training Progress: 88%|████████▊ | 88/100 [1:08:55<09:11, 45.95s/it] (TaskRunner pid=2049544) step:89 - global_seqlen/min:23526 - global_seqlen/max:43993 - global_seqlen/minmax_diff:20467 - global_seqlen/balanced_min:32615 - global_seqlen/balanced_max:32696 - global_seqlen/mean:32672.25 - actor/entropy:0.3472384810447693 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7165358662605286 - training/rollout_probs_diff_mean:0.003400100627914071 - training/rollout_probs_diff_std:0.009368252009153366 - training/rollout_actor_probs_pearson_corr:0.9993821382522583 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:5.2063269322388805e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.123795977444388e-05 - rollout_corr/rollout_is_max:2.0695598125457764 - rollout_corr/rollout_is_min:0.2369716465473175 - rollout_corr/rollout_is_std:0.03049706295132637 - rollout_corr/rollout_is_eff_sample_size:0.9990706743631297 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0016654012724757195 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.9951695799827576 - rollout_corr/rollout_is_seq_max_deviation:0.007972240447998047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3569685937836766) - rollout_corr/training_log_ppl:np.float64(0.27856632304610685) - rollout_corr/kl:np.float64(0.0005558686453759343) - rollout_corr/k3_kl:np.float64(0.0005486493795388014) - rollout_corr/rollout_ppl:np.float64(1.3561613205820322) - rollout_corr/rollout_log_ppl:np.float64(0.27801045119122136) - rollout_corr/log_ppl_diff:np.float64(0.0005558718548854813) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011855626216856763) - rollout_corr/log_ppl_diff_max:np.float64(0.005568467080593109) - rollout_corr/log_ppl_diff_min:np.float64(-0.006579384207725525) - rollout_corr/ppl_ratio:np.float64(1.0005573213566095) - rollout_corr/chi2_token:np.float64(0.0010650090407580137) - rollout_corr/chi2_seq:np.float64(0.8179540059063584) - actor/pg_loss:np.float64(3.63390427082777e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022707523396547913) - actor/ppo_kl:np.float64(6.390776004217003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3162117004394531) - perf/mfu/actor:np.float64(0.09205693492850642) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16839599609375) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01907663233578205 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01907663233578205 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:750.2890625 - response_length/max:3252.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:750.2890625 - response_length_non_aborted/max:3252.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003261677920818329 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3473277054727077) - timing_s/agent_loop/generate_sequences/max:np.float64(20.700606567785144) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.879736434209917) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.700606567785144) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:3252 - timing_s/gen:26.722555236890912 - timing_s/reward:0.0684677753597498 - timing_s/old_log_prob:2.669492883607745 - timing_s/adv:0.1653036493808031 - timing_s/update_actor:10.382515193894506 - timing_s/step:40.15039100125432 - timing_s/stop_profile:0.00012563727796077728 - timing_per_token_ms/update_actor:0.03972222296403869 - timing_per_token_ms/gen:0.13912635357669914 - timing_per_token_ms/adv:0.0006324313805324209 - perf/total_num_tokens:261378 - perf/time_per_step:40.15039100125432 - perf/throughput:813.7467453051529 (TaskRunner pid=2049544) Training Progress: 89%|████████▉ | 89/100 [1:09:36<08:07, 44.35s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2049544) (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \text{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \text{cm} = 0.50 \, \text{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2049544) step:90 - global_seqlen/min:22189 - global_seqlen/max:43070 - global_seqlen/minmax_diff:20881 - global_seqlen/balanced_min:29230 - global_seqlen/balanced_max:29298 - global_seqlen/mean:29287.75 - actor/entropy:0.3101608455181122 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3454935550689697 - training/rollout_probs_diff_mean:0.0033272632863372564 - training/rollout_probs_diff_std:0.009299682453274727 - training/rollout_actor_probs_pearson_corr:0.999302327632904 - rollout_corr/rollout_is_mean:0.9999826550483704 - rollout_corr/rollout_is_ratio_fraction_high:1.2050661098328419e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2050661098328419e-05 - rollout_corr/rollout_is_max:2.8188319206237793 - rollout_corr/rollout_is_min:0.3406546115875244 - rollout_corr/rollout_is_std:0.03000447526574135 - rollout_corr/rollout_is_eff_sample_size:0.9991005411997865 - rollout_corr/rollout_is_seq_mean:1.0000338554382324 - rollout_corr/rollout_is_seq_std:0.0013818239094689488 - rollout_corr/rollout_is_seq_max:1.0056010484695435 - rollout_corr/rollout_is_seq_min:0.9940690398216248 - rollout_corr/rollout_is_seq_max_deviation:0.005930960178375244 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3156746202148497) - rollout_corr/training_log_ppl:np.float64(0.2531014566193335) - rollout_corr/kl:np.float64(0.0005285690323812986) - rollout_corr/k3_kl:np.float64(0.0004731722787880699) - rollout_corr/rollout_ppl:np.float64(1.3148961896076798) - rollout_corr/rollout_log_ppl:np.float64(0.25257288754801266) - rollout_corr/log_ppl_diff:np.float64(0.0005285690713208169) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010851562838070095) - rollout_corr/log_ppl_diff_max:np.float64(0.006567090749740601) - rollout_corr/log_ppl_diff_min:np.float64(-0.00448918342590332) - rollout_corr/ppl_ratio:np.float64(1.0005297421012074) - rollout_corr/chi2_token:np.float64(0.0008521552663296461) - rollout_corr/chi2_seq:np.float64(0.22603251412510872) - actor/pg_loss:np.float64(0.00011028861626982689) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012377322673273738) - actor/ppo_kl:np.float64(0.00010799788028137414) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14029540959745646) - perf/mfu/actor:np.float64(0.08421706394502831) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.20226669311523) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10800278634919543) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8915625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04701965592013402) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.693225) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0730352339040406) - val-aux/sciknoweval/score/mean@16:np.float64(0.6625) - val-aux/sciknoweval/score/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10800278634919543) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8915625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04701965592013402) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.693225) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0730352339040406) - val-core/sciknoweval/acc/mean@16:np.float64(0.6625) - val-aux/sciknoweval/acc/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10800278634919543) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8915625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04701965592013402) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.693225) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0730352339040406) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9890625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03660810518441162) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9990500000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010677570009542227) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9789875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.048719255966368245) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.989175) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.036500484252753186) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9618874999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06036258960895739) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9962249999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021651315045931742) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9349000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06689717910953397) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9992000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009789340111118103) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9021374999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05987592930423645) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0017437187447881502) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01157616637647152 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01157616637647152 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.3046875 - response_length/max:2634.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.3046875 - response_length_non_aborted/max:2634.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011180341243743896 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8763771429657936) - timing_s/agent_loop/generate_sequences/max:np.float64(17.981070213019848) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.232629431753594) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.981070213019848) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:2634 - timing_s/gen:23.51352482289076 - timing_s/reward:0.0758508276194334 - timing_s/old_log_prob:2.507504915818572 - timing_s/adv:0.1161189116537571 - timing_s/update_actor:10.261307056993246 - timing_s/step:36.5611352995038 - timing_s/testing:112.82421230524778 - timing_s/save_checkpoint:6.677826223894954 - timing_s/stop_profile:0.0001322384923696518 - timing_per_token_ms/update_actor:0.04379521752692357 - timing_per_token_ms/gen:0.14167675802809465 - timing_per_token_ms/adv:0.0004955950510612675 - perf/total_num_tokens:234302 - perf/time_per_step:36.5611352995038 - perf/throughput:801.0623783993241 (TaskRunner pid=2049544) Training Progress: 90%|█████████ | 90/100 [1:12:12<12:58, 77.88s/it] (TaskRunner pid=2049544) step:91 - global_seqlen/min:21497 - global_seqlen/max:45454 - global_seqlen/minmax_diff:23957 - global_seqlen/balanced_min:32904 - global_seqlen/balanced_max:33139 - global_seqlen/mean:33002.875 - actor/entropy:0.26041534543037415 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34585317969322205 - training/rollout_probs_diff_mean:0.002946822438389063 - training/rollout_probs_diff_std:0.008950787596404552 - training/rollout_actor_probs_pearson_corr:0.9992419481277466 - rollout_corr/rollout_is_mean:0.9999063611030579 - rollout_corr/rollout_is_ratio_fraction_high:5.014869202568661e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5044607607705984e-05 - rollout_corr/rollout_is_max:5.096095561981201 - rollout_corr/rollout_is_min:0.37255313992500305 - rollout_corr/rollout_is_std:0.028246553614735603 - rollout_corr/rollout_is_eff_sample_size:0.9992028873298154 - rollout_corr/rollout_is_seq_mean:1.0000227689743042 - rollout_corr/rollout_is_seq_std:0.001477584009990096 - rollout_corr/rollout_is_seq_max:1.0067633390426636 - rollout_corr/rollout_is_seq_min:0.995822548866272 - rollout_corr/rollout_is_seq_max_deviation:0.006763339042663574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2352091469801962) - rollout_corr/training_log_ppl:np.float64(0.20221140759531409) - rollout_corr/kl:np.float64(0.00045496249386900445) - rollout_corr/k3_kl:np.float64(0.00047493476955651204) - rollout_corr/rollout_ppl:np.float64(1.2346553695388138) - rollout_corr/rollout_log_ppl:np.float64(0.2017564445413882) - rollout_corr/log_ppl_diff:np.float64(0.0004549630539258942) - rollout_corr/log_ppl_abs_diff:np.float64(0.001167977330624126) - rollout_corr/log_ppl_diff_max:np.float64(0.009916573762893677) - rollout_corr/log_ppl_diff_min:np.float64(-0.006213173270225525) - rollout_corr/ppl_ratio:np.float64(1.0004563292022794) - rollout_corr/chi2_token:np.float64(0.0009643365629017353) - rollout_corr/chi2_seq:np.float64(0.6764775053597987) - actor/pg_loss:np.float64(5.630298983305693e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001349058953792337) - actor/ppo_kl:np.float64(3.4980036296872186e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12929855473339558) - perf/mfu/actor:np.float64(0.09015558565254489) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.73992156982422) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012762214988470078 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012762214988470078 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:778.93359375 - response_length/max:5011.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:778.93359375 - response_length_non_aborted/max:5011.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.5048614740371704e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9828272927552462) - timing_s/agent_loop/generate_sequences/max:np.float64(30.033628037199378) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.231723692668311) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.033628037199378) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:5011 - timing_s/gen:35.68446674197912 - timing_s/reward:0.0753241665661335 - timing_s/old_log_prob:2.7556536067277193 - timing_s/adv:0.12121403589844704 - timing_s/update_actor:10.860587490722537 - timing_s/step:49.55868074670434 - timing_s/stop_profile:0.00011924095451831818 - timing_per_token_ms/update_actor:0.04113500524849175 - timing_per_token_ms/gen:0.17895292914480998 - timing_per_token_ms/adv:0.0004591040776691691 - perf/total_num_tokens:264023 - perf/time_per_step:49.55868074670434 - perf/throughput:665.9353013991337 (TaskRunner pid=2049544) Training Progress: 91%|█████████ | 91/100 [1:13:02<10:24, 69.39s/it] (TaskRunner pid=2049544) step:92 - global_seqlen/min:27995 - global_seqlen/max:54733 - global_seqlen/minmax_diff:26738 - global_seqlen/balanced_min:38799 - global_seqlen/balanced_max:39948 - global_seqlen/mean:39032.875 - actor/entropy:0.365276038646698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4553599953651428 - training/rollout_probs_diff_mean:0.0034760674461722374 - training/rollout_probs_diff_std:0.009343450888991356 - training/rollout_actor_probs_pearson_corr:0.999370813369751 - rollout_corr/rollout_is_mean:0.9999127984046936 - rollout_corr/rollout_is_ratio_fraction_high:8.297963177028578e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.0744908397318795e-05 - rollout_corr/rollout_is_max:3.3402388095855713 - rollout_corr/rollout_is_min:0.3921363651752472 - rollout_corr/rollout_is_std:0.03065885230898857 - rollout_corr/rollout_is_eff_sample_size:0.999060798471362 - rollout_corr/rollout_is_seq_mean:0.9999949932098389 - rollout_corr/rollout_is_seq_std:0.001438560662791133 - rollout_corr/rollout_is_seq_max:1.005057454109192 - rollout_corr/rollout_is_seq_min:0.9950283765792847 - rollout_corr/rollout_is_seq_max_deviation:0.0050574541091918945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4253517426550388) - rollout_corr/training_log_ppl:np.float64(0.3268485037915525) - rollout_corr/kl:np.float64(0.0007143817989492263) - rollout_corr/k3_kl:np.float64(0.0007667658379375553) - rollout_corr/rollout_ppl:np.float64(1.4242633627727628) - rollout_corr/rollout_log_ppl:np.float64(0.3261341213074047) - rollout_corr/log_ppl_diff:np.float64(0.0007143824841477908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013714263404835947) - rollout_corr/log_ppl_diff_max:np.float64(0.013571739196777344) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037727057933807373) - rollout_corr/ppl_ratio:np.float64(1.000716715119779) - rollout_corr/chi2_token:np.float64(0.001422470435500145) - rollout_corr/chi2_seq:np.float64(0.832664096262306) - actor/pg_loss:np.float64(2.3467233404517174e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434544945548623) - actor/ppo_kl:np.float64(0.00016380211024369373) - actor/pg_clipfrac_lower:np.float64(2.0755844616360264e-06) - actor/grad_norm:np.float64(0.215729471296072) - perf/mfu/actor:np.float64(0.10647986423451218) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.4896354675293) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011413848027586937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011413848027586937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:941.49609375 - response_length/max:5473.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:941.49609375 - response_length_non_aborted/max:5473.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.439482629299164e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7557445913553238) - timing_s/agent_loop/generate_sequences/max:np.float64(32.87127552181482) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.810325469457894) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.87127552181482) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:5473 - timing_s/gen:38.770385617390275 - timing_s/reward:0.08092910423874855 - timing_s/old_log_prob:2.7198645491153 - timing_s/adv:0.12597152031958103 - timing_s/update_actor:10.861796030774713 - timing_s/step:52.657433899119496 - timing_s/stop_profile:0.00011220574378967285 - timing_per_token_ms/update_actor:0.034784127580836384 - timing_per_token_ms/gen:0.1608576178098782 - timing_per_token_ms/adv:0.00040341481481821745 - perf/total_num_tokens:312263 - perf/time_per_step:52.657433899119496 - perf/throughput:741.2604851724968 (TaskRunner pid=2049544) Training Progress: 92%|█████████▏| 92/100 [1:13:54<08:35, 64.39s/it] (TaskRunner pid=2049544) step:93 - global_seqlen/min:17843 - global_seqlen/max:51915 - global_seqlen/minmax_diff:34072 - global_seqlen/balanced_min:30929 - global_seqlen/balanced_max:31113 - global_seqlen/mean:31065.0 - actor/entropy:0.4133561849594116 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311214327812195 - training/rollout_probs_diff_mean:0.0036772009916603565 - training/rollout_probs_diff_std:0.009407786652445793 - training/rollout_actor_probs_pearson_corr:0.9994404911994934 - rollout_corr/rollout_is_mean:0.9999762177467346 - rollout_corr/rollout_is_ratio_fraction_high:5.474292720464291e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6422878616140224e-05 - rollout_corr/rollout_is_max:2.111367702484131 - rollout_corr/rollout_is_min:0.21797895431518555 - rollout_corr/rollout_is_std:0.031457215547561646 - rollout_corr/rollout_is_eff_sample_size:0.9990114219410976 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0018004179000854492 - rollout_corr/rollout_is_seq_max:1.007155179977417 - rollout_corr/rollout_is_seq_min:0.9935778975486755 - rollout_corr/rollout_is_seq_max_deviation:0.007155179977416992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4254543129354715) - rollout_corr/training_log_ppl:np.float64(0.3189181691413978) - rollout_corr/kl:np.float64(0.0006560000657124254) - rollout_corr/k3_kl:np.float64(0.0006174342987037562) - rollout_corr/rollout_ppl:np.float64(1.4244641666300595) - rollout_corr/rollout_log_ppl:np.float64(0.31826216776971705) - rollout_corr/log_ppl_diff:np.float64(0.0006560013716807589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014309252292150632) - rollout_corr/log_ppl_diff_max:np.float64(0.00973115861415863) - rollout_corr/log_ppl_diff_min:np.float64(-0.004590287804603577) - rollout_corr/ppl_ratio:np.float64(1.0006579444743693) - rollout_corr/chi2_token:np.float64(0.0011531137861311436) - rollout_corr/chi2_seq:np.float64(0.8776039613876492) - actor/pg_loss:np.float64(0.00012877536937594414) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000180150422920633) - actor/ppo_kl:np.float64(9.64217530006195e-05) - actor/pg_clipfrac_lower:np.float64(3.570612534531392e-06) - actor/grad_norm:np.float64(0.18000993877649307) - perf/mfu/actor:np.float64(0.08642174822282235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.50779724121094) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016780760139226913 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016780760139226913 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:713.5625 - response_length/max:4647.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:713.5625 - response_length_non_aborted/max:4647.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010390952229499817 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7623797114938498) - timing_s/agent_loop/generate_sequences/max:np.float64(27.237358892336488) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.646888445196964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.237358892336488) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:4647 - timing_s/gen:33.228857627138495 - timing_s/reward:0.07251898013055325 - timing_s/old_log_prob:2.6381760984659195 - timing_s/adv:0.12340777926146984 - timing_s/update_actor:10.632954278960824 - timing_s/step:46.78136018849909 - timing_s/stop_profile:0.0001156739890575409 - timing_per_token_ms/update_actor:0.04278510493707075 - timing_per_token_ms/gen:0.18190449344802978 - timing_per_token_ms/adv:0.0004965708162782466 - perf/total_num_tokens:248520 - perf/time_per_step:46.78136018849909 - perf/throughput:664.0465320979944 (TaskRunner pid=2049544) Training Progress: 93%|█████████▎| 93/100 [1:14:41<06:53, 59.12s/it] (TaskRunner pid=2049544) step:94 - global_seqlen/min:22853 - global_seqlen/max:49175 - global_seqlen/minmax_diff:26322 - global_seqlen/balanced_min:33215 - global_seqlen/balanced_max:33359 - global_seqlen/mean:33330.0 - actor/entropy:0.36058807373046875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8867406845092773 - training/rollout_probs_diff_mean:0.003440327476710081 - training/rollout_probs_diff_std:0.009632454253733158 - training/rollout_actor_probs_pearson_corr:0.9993698000907898 - rollout_corr/rollout_is_mean:0.9999211430549622 - rollout_corr/rollout_is_ratio_fraction_high:1.0051463505078573e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.523158713709563e-05 - rollout_corr/rollout_is_max:2.117000102996826 - rollout_corr/rollout_is_min:0.0013813151745125651 - rollout_corr/rollout_is_std:0.030682172626256943 - rollout_corr/rollout_is_eff_sample_size:0.9990594896328889 - rollout_corr/rollout_is_seq_mean:0.9999591112136841 - rollout_corr/rollout_is_seq_std:0.0020710721146315336 - rollout_corr/rollout_is_seq_max:1.011606216430664 - rollout_corr/rollout_is_seq_min:0.9853842854499817 - rollout_corr/rollout_is_seq_max_deviation:0.01461571455001831 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3442378360778093) - rollout_corr/training_log_ppl:np.float64(0.2689030548208393) - rollout_corr/kl:np.float64(0.0007270200237998736) - rollout_corr/k3_kl:np.float64(0.0005578451756775848) - rollout_corr/rollout_ppl:np.float64(1.3432344142347574) - rollout_corr/rollout_log_ppl:np.float64(0.26817603285599034) - rollout_corr/log_ppl_diff:np.float64(0.0007270219648489729) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013523521920433268) - rollout_corr/log_ppl_diff_max:np.float64(0.018366605043411255) - rollout_corr/log_ppl_diff_min:np.float64(-0.00493505597114563) - rollout_corr/ppl_ratio:np.float64(1.0007292504888028) - rollout_corr/chi2_token:np.float64(0.0007120152004063129) - rollout_corr/chi2_seq:np.float64(0.3402676989790052) - actor/pg_loss:np.float64(-4.822376649826765e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.676928118551587e-05) - actor/ppo_kl:np.float64(0.00014517965372551345) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16034816205501556) - perf/mfu/actor:np.float64(0.09308032359557787) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.54820251464844) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0465502105653286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0465502105653286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:777.25 - response_length/max:3614.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:777.25 - response_length_non_aborted/max:3614.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001496635377407074 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8505959454923868) - timing_s/agent_loop/generate_sequences/max:np.float64(23.250541422516108) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.11456365948834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.250541422516108) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:293 - timing_s/agent_loop/slowest/response_length:3614 - timing_s/gen:29.206484995782375 - timing_s/reward:0.07558401487767696 - timing_s/old_log_prob:2.6541784573346376 - timing_s/adv:0.12619186379015446 - timing_s/update_actor:10.563141915947199 - timing_s/step:42.71316540054977 - timing_s/stop_profile:0.00011851266026496887 - timing_per_token_ms/update_actor:0.03961574375917792 - timing_per_token_ms/gen:0.14678395884821474 - timing_per_token_ms/adv:0.0004732668158946687 - perf/total_num_tokens:266640 - perf/time_per_step:42.71316540054977 - perf/throughput:780.3214696790185 (TaskRunner pid=2049544) Training Progress: 94%|█████████▍| 94/100 [1:15:24<05:25, 54.21s/it] (TaskRunner pid=2049544) step:95 - global_seqlen/min:27066 - global_seqlen/max:58256 - global_seqlen/minmax_diff:31190 - global_seqlen/balanced_min:37178 - global_seqlen/balanced_max:37249 - global_seqlen/mean:37221.375 - actor/entropy:0.3698161244392395 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553989887237549 - training/rollout_probs_diff_mean:0.0034687381703406572 - training/rollout_probs_diff_std:0.009108307771384716 - training/rollout_actor_probs_pearson_corr:0.999431848526001 - rollout_corr/rollout_is_mean:0.9999311566352844 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3214753380452748e-05 - rollout_corr/rollout_is_max:1.8554537296295166 - rollout_corr/rollout_is_min:0.4261438846588135 - rollout_corr/rollout_is_std:0.030212340876460075 - rollout_corr/rollout_is_eff_sample_size:0.9990878089036473 - rollout_corr/rollout_is_seq_mean:0.9999582767486572 - rollout_corr/rollout_is_seq_std:0.0013638396048918366 - rollout_corr/rollout_is_seq_max:1.0052945613861084 - rollout_corr/rollout_is_seq_min:0.9958161115646362 - rollout_corr/rollout_is_seq_max_deviation:0.0052945613861083984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3786341231316328) - rollout_corr/training_log_ppl:np.float64(0.30179976880026516) - rollout_corr/kl:np.float64(0.0004993855567150263) - rollout_corr/k3_kl:np.float64(0.0005308137937447555) - rollout_corr/rollout_ppl:np.float64(1.377938257995993) - rollout_corr/rollout_log_ppl:np.float64(0.30130038285278715) - rollout_corr/log_ppl_diff:np.float64(0.0004993859474780038) - rollout_corr/log_ppl_abs_diff:np.float64(0.001156414728029631) - rollout_corr/log_ppl_diff_max:np.float64(0.0048898980021476746) - rollout_corr/log_ppl_diff_min:np.float64(-0.00559312105178833) - rollout_corr/ppl_ratio:np.float64(1.0005006100982428) - rollout_corr/chi2_token:np.float64(0.001144631765782833) - rollout_corr/chi2_seq:np.float64(0.9150484425481409) - actor/pg_loss:np.float64(-2.8662499971687794e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000206823247367538) - actor/ppo_kl:np.float64(-2.0183409755958337e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1875382363796234) - perf/mfu/actor:np.float64(0.10496417728509784) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.40451049804688) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.021443139761686325 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.021443139761686325 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:886.79296875 - response_length/max:3209.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:886.79296875 - response_length_non_aborted/max:3209.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.542517364025116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.471508091315627) - timing_s/agent_loop/generate_sequences/max:np.float64(22.023407490924) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.383664465822221) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.023407490924) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:3175 - timing_s/gen:28.187608854845166 - timing_s/reward:0.08735943771898746 - timing_s/old_log_prob:2.6067604515701532 - timing_s/adv:0.13234028778970242 - timing_s/update_actor:10.463114799931645 - timing_s/step:41.565891321748495 - timing_s/stop_profile:0.00011608004570007324 - timing_per_token_ms/update_actor:0.035138125606360746 - timing_per_token_ms/gen:0.12416409575782276 - timing_per_token_ms/adv:0.00044443645549668175 - perf/total_num_tokens:297771 - perf/time_per_step:41.565891321748495 - perf/throughput:895.4788124686426 (TaskRunner pid=2049544) Training Progress: 95%|█████████▌| 95/100 [1:16:06<04:12, 50.43s/it] (TaskRunner pid=2049544) step:96 - global_seqlen/min:20271 - global_seqlen/max:71208 - global_seqlen/minmax_diff:50937 - global_seqlen/balanced_min:37576 - global_seqlen/balanced_max:37647 - global_seqlen/mean:37613.5 - actor/entropy:0.334806889295578 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31711164116859436 - training/rollout_probs_diff_mean:0.003466662485152483 - training/rollout_probs_diff_std:0.009574675001204014 - training/rollout_actor_probs_pearson_corr:0.9992694854736328 - rollout_corr/rollout_is_mean:1.0000592470169067 - rollout_corr/rollout_is_ratio_fraction_high:8.720068763068411e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.720068763068411e-06 - rollout_corr/rollout_is_max:2.086963415145874 - rollout_corr/rollout_is_min:0.4454006552696228 - rollout_corr/rollout_is_std:0.030977971851825714 - rollout_corr/rollout_is_eff_sample_size:0.9990414042159874 - rollout_corr/rollout_is_seq_mean:1.0001949071884155 - rollout_corr/rollout_is_seq_std:0.0014157574623823166 - rollout_corr/rollout_is_seq_max:1.0063108205795288 - rollout_corr/rollout_is_seq_min:0.9960592985153198 - rollout_corr/rollout_is_seq_max_deviation:0.006310820579528809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3516691238619387) - rollout_corr/training_log_ppl:np.float64(0.27739123623177875) - rollout_corr/kl:np.float64(0.00038778805755357304) - rollout_corr/k3_kl:np.float64(0.0005165411663199393) - rollout_corr/rollout_ppl:np.float64(1.3510792162269354) - rollout_corr/rollout_log_ppl:np.float64(0.277003448762116) - rollout_corr/log_ppl_diff:np.float64(0.0003877874696627259) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011570610222406685) - rollout_corr/log_ppl_diff_max:np.float64(0.00403740257024765) - rollout_corr/log_ppl_diff_min:np.float64(-0.005538493394851685) - rollout_corr/ppl_ratio:np.float64(1.000388921238482) - rollout_corr/chi2_token:np.float64(0.0012935972772538662) - rollout_corr/chi2_seq:np.float64(2.2381823365576565) - actor/pg_loss:np.float64(5.385885015130043e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015849828230329877) - actor/ppo_kl:np.float64(8.225317838395085e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11362059134989977) - perf/mfu/actor:np.float64(0.1031450502424441) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.59148025512695) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007041455246508121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007041455246508121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:895.921875 - response_length/max:4074.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:895.921875 - response_length_non_aborted/max:4074.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.657893002033234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2360860873013735) - timing_s/agent_loop/generate_sequences/max:np.float64(26.039900217205286) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.459900658301194) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.039900217205286) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3982 - timing_s/gen:32.00374800339341 - timing_s/reward:0.0823327787220478 - timing_s/old_log_prob:2.7716220505535603 - timing_s/adv:0.1664755642414093 - timing_s/update_actor:10.801547262817621 - timing_s/step:45.91564348526299 - timing_s/stop_profile:0.00011668726801872253 - timing_per_token_ms/update_actor:0.035896510770127815 - timing_per_token_ms/gen:0.13953743526828777 - timing_per_token_ms/adv:0.0005532440621100446 - perf/total_num_tokens:300908 - perf/time_per_step:45.91564348526299 - perf/throughput:819.1870383363432 (TaskRunner pid=2049544) Training Progress: 96%|█████████▌| 96/100 [1:16:52<03:16, 49.09s/it] (TaskRunner pid=2049544) step:97 - global_seqlen/min:20006 - global_seqlen/max:63623 - global_seqlen/minmax_diff:43617 - global_seqlen/balanced_min:40703 - global_seqlen/balanced_max:40790 - global_seqlen/mean:40767.5 - actor/entropy:0.4114430248737335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3968365788459778 - training/rollout_probs_diff_mean:0.0036807965952903032 - training/rollout_probs_diff_std:0.009112636558711529 - training/rollout_actor_probs_pearson_corr:0.9994710087776184 - rollout_corr/rollout_is_mean:0.9999697208404541 - rollout_corr/rollout_is_ratio_fraction_high:3.885426394845126e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5541705579380505e-05 - rollout_corr/rollout_is_max:6.432518005371094 - rollout_corr/rollout_is_min:0.14259245991706848 - rollout_corr/rollout_is_std:0.030860379338264465 - rollout_corr/rollout_is_eff_sample_size:0.9990483051530745 - rollout_corr/rollout_is_seq_mean:1.000016450881958 - rollout_corr/rollout_is_seq_std:0.0015233448939397931 - rollout_corr/rollout_is_seq_max:1.0063254833221436 - rollout_corr/rollout_is_seq_min:0.994326114654541 - rollout_corr/rollout_is_seq_max_deviation:0.006325483322143555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3867830368690193) - rollout_corr/training_log_ppl:np.float64(0.30247966502793133) - rollout_corr/kl:np.float64(0.0005262127845906406) - rollout_corr/k3_kl:np.float64(0.0005436497510515892) - rollout_corr/rollout_ppl:np.float64(1.386001723818481) - rollout_corr/rollout_log_ppl:np.float64(0.30195345119864214) - rollout_corr/log_ppl_diff:np.float64(0.0005262138292891905) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011905098654096946) - rollout_corr/log_ppl_diff_max:np.float64(0.007206156849861145) - rollout_corr/log_ppl_diff_min:np.float64(-0.003447234630584717) - rollout_corr/ppl_ratio:np.float64(1.0005276133306324) - rollout_corr/chi2_token:np.float64(0.001151977339759469) - rollout_corr/chi2_seq:np.float64(0.950474871089682) - actor/pg_loss:np.float64(6.92273024469614e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019685469499108876) - actor/ppo_kl:np.float64(3.492400511362348e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16588949039578438) - perf/mfu/actor:np.float64(0.11018383492360453) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.53912734985352) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.043718818575143814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.043718818575143814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1005.359375 - response_length/max:4322.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1005.359375 - response_length_non_aborted/max:4322.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012983568012714386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.021046968176961) - timing_s/agent_loop/generate_sequences/max:np.float64(28.757525622844696) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.163475782763271) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.757525622844696) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:4322 - timing_s/gen:34.78881582431495 - timing_s/reward:0.08717712201178074 - timing_s/old_log_prob:2.7238584086298943 - timing_s/adv:0.1590470802038908 - timing_s/update_actor:10.954232659190893 - timing_s/step:48.80308444797993 - timing_s/stop_profile:0.00013154186308383942 - timing_per_token_ms/update_actor:0.0335875165854875 - timing_per_token_ms/gen:0.13516938837291917 - timing_per_token_ms/adv:0.000487665052443401 - perf/total_num_tokens:326140 - perf/time_per_step:48.80308444797993 - perf/throughput:835.3467913171513 (TaskRunner pid=2049544) Training Progress: 97%|█████████▋| 97/100 [1:17:40<02:27, 49.03s/it] (TaskRunner pid=2049544) step:98 - global_seqlen/min:23082 - global_seqlen/max:43864 - global_seqlen/minmax_diff:20782 - global_seqlen/balanced_min:32541 - global_seqlen/balanced_max:33196 - global_seqlen/mean:32691.375 - actor/entropy:0.3132493495941162 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6835976243019104 - training/rollout_probs_diff_mean:0.0034196285996586084 - training/rollout_probs_diff_std:0.009468399919569492 - training/rollout_actor_probs_pearson_corr:0.9993211627006531 - rollout_corr/rollout_is_mean:1.0001163482666016 - rollout_corr/rollout_is_ratio_fraction_high:1.5585144865326583e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5585144865326583e-05 - rollout_corr/rollout_is_max:5.03674840927124 - rollout_corr/rollout_is_min:0.0648747980594635 - rollout_corr/rollout_is_std:0.03038937970995903 - rollout_corr/rollout_is_eff_sample_size:0.9990775757045947 - rollout_corr/rollout_is_seq_mean:1.0001397132873535 - rollout_corr/rollout_is_seq_std:0.001354765729047358 - rollout_corr/rollout_is_seq_max:1.0069159269332886 - rollout_corr/rollout_is_seq_min:0.9959137439727783 - rollout_corr/rollout_is_seq_max_deviation:0.006915926933288574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2990983063355088) - rollout_corr/training_log_ppl:np.float64(0.24720221558527555) - rollout_corr/kl:np.float64(0.0005138034307572426) - rollout_corr/k3_kl:np.float64(0.0005089092589116717) - rollout_corr/rollout_ppl:np.float64(1.298400120344013) - rollout_corr/rollout_log_ppl:np.float64(0.24668841203674674) - rollout_corr/log_ppl_diff:np.float64(0.0005138035485288128) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011324260121909901) - rollout_corr/log_ppl_diff_max:np.float64(0.00813785195350647) - rollout_corr/log_ppl_diff_min:np.float64(-0.005736410617828369) - rollout_corr/ppl_ratio:np.float64(1.0005151100922376) - rollout_corr/chi2_token:np.float64(0.0009681761730462313) - rollout_corr/chi2_seq:np.float64(4.5980356687214226) - actor/pg_loss:np.float64(-9.167857933789492e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002514609163881687) - actor/ppo_kl:np.float64(0.00017433107315412144) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1871425360441208) - perf/mfu/actor:np.float64(0.09212177950831188) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56390380859375) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003707965661305934 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0003707965661305934 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:751.91796875 - response_length/max:4148.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.91796875 - response_length_non_aborted/max:4148.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010955892503261566 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1268691644072533) - timing_s/agent_loop/generate_sequences/max:np.float64(24.762351796031) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.324899325562001) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.762351796031) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:4148 - timing_s/gen:30.538867769762874 - timing_s/reward:0.07450433447957039 - timing_s/old_log_prob:2.632541459053755 - timing_s/adv:0.1884304340928793 - timing_s/update_actor:10.359910435974598 - timing_s/step:43.87974337860942 - timing_s/stop_profile:0.000130457803606987 - timing_per_token_ms/update_actor:0.03961255237801484 - timing_per_token_ms/gen:0.15865088637787156 - timing_per_token_ms/adv:0.0007204898619776597 - perf/total_num_tokens:261531 - perf/time_per_step:43.87974337860942 - perf/throughput:745.0220188830104 (TaskRunner pid=2049544) Training Progress: 98%|█████████▊| 98/100 [1:18:24<01:34, 47.50s/it] (TaskRunner pid=2049544) step:99 - global_seqlen/min:21119 - global_seqlen/max:73608 - global_seqlen/minmax_diff:52489 - global_seqlen/balanced_min:37066 - global_seqlen/balanced_max:37818 - global_seqlen/mean:37233.375 - actor/entropy:0.36280471086502075 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29852432012557983 - training/rollout_probs_diff_mean:0.0034003094770014286 - training/rollout_probs_diff_std:0.009052536450326443 - training/rollout_actor_probs_pearson_corr:0.9994275569915771 - rollout_corr/rollout_is_mean:0.9999655485153198 - rollout_corr/rollout_is_ratio_fraction_high:4.338112375990022e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7352449503960088e-05 - rollout_corr/rollout_is_max:2.0474398136138916 - rollout_corr/rollout_is_min:0.40364018082618713 - rollout_corr/rollout_is_std:0.029825152829289436 - rollout_corr/rollout_is_eff_sample_size:0.9991112508590329 - rollout_corr/rollout_is_seq_mean:0.9999865293502808 - rollout_corr/rollout_is_seq_std:0.0013986814301460981 - rollout_corr/rollout_is_seq_max:1.0060322284698486 - rollout_corr/rollout_is_seq_min:0.9942169785499573 - rollout_corr/rollout_is_seq_max_deviation:0.006032228469848633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2896207650192082) - rollout_corr/training_log_ppl:np.float64(0.2305254971361137) - rollout_corr/kl:np.float64(0.0005066159342224097) - rollout_corr/k3_kl:np.float64(0.0004654821964606981) - rollout_corr/rollout_ppl:np.float64(1.2889360883273184) - rollout_corr/rollout_log_ppl:np.float64(0.23001887879217975) - rollout_corr/log_ppl_diff:np.float64(0.0005066183439339511) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010047785108326934) - rollout_corr/log_ppl_diff_max:np.float64(0.0074303895235061646) - rollout_corr/log_ppl_diff_min:np.float64(-0.004489123821258545) - rollout_corr/ppl_ratio:np.float64(1.0005076131783426) - rollout_corr/chi2_token:np.float64(0.000825623283162713) - rollout_corr/chi2_seq:np.float64(0.5253964632283896) - actor/pg_loss:np.float64(7.684435695409775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010054797257907921) - actor/ppo_kl:np.float64(4.189955548028479e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08311776397749782) - perf/mfu/actor:np.float64(0.10069730946959708) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.47393417358398) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.86328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.86328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017708174884319305 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017708174884319305 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:900.44921875 - response_length/max:6057.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:900.44921875 - response_length_non_aborted/max:6057.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011433474719524384 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8707099817693233) - timing_s/agent_loop/generate_sequences/max:np.float64(35.631088424474) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.754026362330478) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(35.631088424474) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:6057 - timing_s/gen:41.34765339829028 - timing_s/reward:0.08056613802909851 - timing_s/old_log_prob:2.6803389824926853 - timing_s/adv:0.13033771142363548 - timing_s/update_actor:10.983067074790597 - timing_s/step:55.307873111218214 - timing_s/stop_profile:0.00011896714568138123 - timing_per_token_ms/update_actor:0.03687238624886475 - timing_per_token_ms/gen:0.17937077152588887 - timing_per_token_ms/adv:0.000437570161930108 - perf/total_num_tokens:297867 - perf/time_per_step:55.307873111218214 - perf/throughput:673.2020760430919 (TaskRunner pid=2049544) Training Progress: 99%|█████████▉| 99/100 [1:19:20<00:49, 49.85s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We use the formula for the electric field due to a point charge: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Solving for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2049544) step:100 - global_seqlen/min:17010 - global_seqlen/max:43433 - global_seqlen/minmax_diff:26423 - global_seqlen/balanced_min:31299 - global_seqlen/balanced_max:31491 - global_seqlen/mean:31419.0 - actor/entropy:0.37953582406044006 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.630595862865448 - training/rollout_probs_diff_mean:0.0036211644764989614 - training/rollout_probs_diff_std:0.00978316180408001 - training/rollout_actor_probs_pearson_corr:0.999374508857727 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:1.0963950444420334e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.644592521188315e-05 - rollout_corr/rollout_is_max:2.3022942543029785 - rollout_corr/rollout_is_min:0.3131349980831146 - rollout_corr/rollout_is_std:0.031683772802352905 - rollout_corr/rollout_is_eff_sample_size:0.998997264270168 - rollout_corr/rollout_is_seq_mean:1.0000271797180176 - rollout_corr/rollout_is_seq_std:0.0016033316496759653 - rollout_corr/rollout_is_seq_max:1.0072500705718994 - rollout_corr/rollout_is_seq_min:0.9959959983825684 - rollout_corr/rollout_is_seq_max_deviation:0.007250070571899414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3458213401027024) - rollout_corr/training_log_ppl:np.float64(0.26966726542741526) - rollout_corr/kl:np.float64(0.00037315004849824973) - rollout_corr/k3_kl:np.float64(0.0005444729254122649) - rollout_corr/rollout_ppl:np.float64(1.3452776027843356) - rollout_corr/rollout_log_ppl:np.float64(0.2692941162531497) - rollout_corr/log_ppl_diff:np.float64(0.0003731491742655635) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012809229083359241) - rollout_corr/log_ppl_diff_max:np.float64(0.0049625784158706665) - rollout_corr/log_ppl_diff_min:np.float64(-0.0106421560049057) - rollout_corr/ppl_ratio:np.float64(1.0003747479058802) - rollout_corr/chi2_token:np.float64(0.0014326744712889194) - rollout_corr/chi2_seq:np.float64(1.9170411999803036) - actor/pg_loss:np.float64(-7.347844075411558e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001629320678375734) - actor/ppo_kl:np.float64(-0.00012778303023264215) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12031856318935752) - perf/mfu/actor:np.float64(0.0881824181802986) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.57986450195312) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08949529688241312) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8910875) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.054003469751280585) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7157250000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05584489583302406) - val-aux/sciknoweval/score/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/score/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08949529688241312) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8910875) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.054003469751280585) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7157250000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05584489583302406) - val-core/sciknoweval/acc/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/acc/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08949529688241312) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8910875) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.054003469751280585) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7157250000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05584489583302406) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9859375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03383512807052093) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99745) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.014287439995541685) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9741250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.042452979144002284) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9857875) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03387729307844336) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998374999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.002749370152057837) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9560875) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04764508939236297) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9946999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021483511676811038) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.933025) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0436722700672832) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9988624999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009237722502874987) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9139250000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.029568343249240757) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0017543234207183663) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018172748386859894 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018172748386859894 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:712.5625 - response_length/max:3652.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:712.5625 - response_length_non_aborted/max:3652.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.623674511909485e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2800081353634596) - timing_s/agent_loop/generate_sequences/max:np.float64(22.433935675770044) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.539074072250514) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.433935675770044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3652 - timing_s/gen:28.36895477026701 - timing_s/reward:0.07503514550626278 - timing_s/old_log_prob:2.573247427120805 - timing_s/adv:0.12070851400494576 - timing_s/update_actor:10.477075356990099 - timing_s/step:41.70409296452999 - timing_s/testing:119.00530271604657 - timing_s/save_checkpoint:6.902973407879472 - timing_s/stop_profile:0.00019830837845802307 - timing_per_token_ms/update_actor:0.041682880410699334 - timing_per_token_ms/gen:0.15551790835380125 - timing_per_token_ms/adv:0.00048023693467704956 - perf/total_num_tokens:251352 - perf/time_per_step:41.70409296452999 - perf/throughput:753.3792912537954 (TaskRunner pid=2049544) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2049544) "np.float64(0.6828125), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2049544) "np.float64(0.17979486326514935), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2049544) "np.float64(0.7532), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2049544) "np.float64(0.1476854115376502), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2049544) "np.float64(0.6133875), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2049544) "np.float64(0.15413287282151064), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2049544) "np.float64(0.6832499999999999), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2049544) "np.float64(0.17928347494256552), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2049544) "np.float64(0.807675), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2049544) "np.float64(0.11309093981217563), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2049544) "np.float64(0.5515125000000001), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2049544) "np.float64(0.12241577016075818), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 85.21s/it] (TaskRunner pid=2049544) "np.float64(0.7018125), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2049544) "np.float64(0.13581544160753714), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2049544) "np.float64(0.8526375000000002), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2049544) "np.float64(0.0888708326383256), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2049544) "np.float64(0.4995375), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2049544) "np.float64(0.08946281937139845), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2049544) "np.float64(0.710175), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2049544) "np.float64(0.08949529688241312), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "np.float64(0.8910875), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2049544) 'np.float64(0.061136881006381635), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/mean@16': np.float64(0.6828125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/std@16': np.float64(0.17979486326514935), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7532), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.1476854115376502), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.6133875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.15413287282151064), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.6832499999999999), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.17928347494256552), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.807675), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.11309093981217563), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.5515125000000001), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.12241577016075818), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7018125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.13581544160753714), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8526375000000002), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.0888708326383256), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.4995375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.08946281937139845), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.710175), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.08949529688241312), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8910875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.061136881006381635), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/mean@16': np.float64(0.6828125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/std@16': np.float64(0.17979486326514935), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7532), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.1476854115376502), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.6133875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.15413287282151064), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.6832499999999999), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.17928347494256552), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.807675), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.11309093981217563), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.5515125000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.12241577016075818), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7018125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.13581544160753714), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8526375000000002), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.0888708326383256), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.4995375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.08946281937139845), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.710175), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.08949529688241312), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8910875), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.061136881006381635), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9859375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2049544) 'np.float64(0.03383512807052093), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.99745), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2049544) 'np.float64(0.014287439995541685), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2049544) 'np.float64(0.9741250000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2049544) 'np.float64(0.042452979144002284), ' (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9857875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2049544) 'np.float64(0.03387729307844336), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2049544) 'np.float64(0.9998374999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2049544) 'np.float64(0.002749370152057837), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.9560875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2049544) 'np.float64(0.04764508939236297), ' (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2049544) 'np.float64(0.9946999999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2049544) 'np.float64(0.021483511676811038), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.933025), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) 'np.float64(0.0436722700672832), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2049544) 'np.float64(0.9988624999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2049544) 'np.float64(0.009237722502874987), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) 'np.float64(0.9139250000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2049544) 'np.float64(0.029568343249240757), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2049544) 'np.float64(0.9998750000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2049544) "np.float64(0.0017543234207183663), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2049544) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2049544) 'np.float64(2.0)}') (TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 49.28s/it] (TaskRunner pid=2049544) (TaskRunner pid=2049544) wandb: updating run metadata (TaskRunner pid=2049544) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2049544) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2049544) wandb: uploading history steps 98-99, summary, console lines 919-1044 (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: Run history: (TaskRunner pid=2049544) wandb: actor/entropy ▁▁▂▁▂▁▁▁▂▃▃▂▂▂▃▂▃▃▃▃▄▄▃▄▅▄▆▄▇▆▅▄▆▆▆▅▇█▇▇ (TaskRunner pid=2049544) wandb: actor/grad_norm █▄▇▃█▆▃▇▆▁▄▄▃▃▃▂▁▂▁▂▄▃▂▂▂▂▄▂▃▃▂▃▂▃▂▁▃▂▂▁ (TaskRunner pid=2049544) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2049544) wandb: actor/lr ▁▂▃▄▆███████████████████████████████████ (TaskRunner pid=2049544) wandb: actor/pg_clipfrac ▂▄▅▃▆▇▄▃█▃▃▄▂▂▃▃▂▂▂▂▃▃▃▂▂▁▁▁▁▂▁▁▁▂▂▂▁▁▂▁ (TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▆▁▁▄▁▃▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁ (TaskRunner pid=2049544) wandb: actor/pg_loss ▆▂▅▅█▆▅▃▃▅▅▄▃▄▅▄▄▃▂▅▁▂▄▄▁▄▆▃▅▄▅▅▂▅▅▄▅▃▅▃ (TaskRunner pid=2049544) wandb: actor/ppo_kl ▁▂▂▃▄▅▃▁▄▃▄▂▄▁▂▂▃▃▂▂▇▄▂▃▂▂▄▄▃▁▄▃▁▂█▃▃▂▃▂ (TaskRunner pid=2049544) wandb: critic/advantages/max █████████▅█████████████▅███▁████████████ (TaskRunner pid=2049544) wandb: critic/advantages/mean ▅▇▆▅▂▄▃▅▅▅▃▂▃▆▃▇▆▂█▂▄▅▃▃▅▁▅▄▄▄▄▅▄▆▆▅▄▃▂▅ (TaskRunner pid=2049544) wandb: +204 ... (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: Run summary: (TaskRunner pid=2049544) wandb: actor/entropy 0.37954 (TaskRunner pid=2049544) wandb: actor/grad_norm 0.12032 (TaskRunner pid=2049544) wandb: actor/kl_loss 0 (TaskRunner pid=2049544) wandb: actor/lr 0.0 (TaskRunner pid=2049544) wandb: actor/pg_clipfrac 0.00016 (TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2049544) wandb: actor/pg_loss -7e-05 (TaskRunner pid=2049544) wandb: actor/ppo_kl -0.00013 (TaskRunner pid=2049544) wandb: critic/advantages/max 0.875 (TaskRunner pid=2049544) wandb: critic/advantages/mean -0.00182 (TaskRunner pid=2049544) wandb: +204 ... (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: 🚀 View run qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/o8ivivjg (TaskRunner pid=2049544) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2049544) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2049544) wandb: Find logs at: ./wandb/run-20260702_073403-o8ivivjg/logs main_ppo exit code: 0 [2026-07-02 08:56:17] Finished physics grpo [2026-07-02 08:56:17] Starting physics rlsd_tr Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B 2026-07-02 08:56:19,506 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 08:56:29,254 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2074004) TaskRunner hostname: mole-workspace-rw, PID: 2074004 (TaskRunner pid=2074004) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2074004) 'calculate_entropy': False, (TaskRunner pid=2074004) 'calculate_sum_pi_squared': False, (TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2074004) 'async_save': False, (TaskRunner pid=2074004) 'load_contents': ['model', (TaskRunner pid=2074004) 'optimizer', (TaskRunner pid=2074004) 'extra'], (TaskRunner pid=2074004) 'save_contents': ['model', (TaskRunner pid=2074004) 'optimizer', (TaskRunner pid=2074004) 'extra']}, (TaskRunner pid=2074004) 'clip_ratio': 0.2, (TaskRunner pid=2074004) 'clip_ratio_c': 3.0, (TaskRunner pid=2074004) 'clip_ratio_high': 0.28, (TaskRunner pid=2074004) 'clip_ratio_low': 0.2, (TaskRunner pid=2074004) 'data_loader_seed': 42, (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_coeff': 0, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'freeze_vision_tower': False, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': False, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'grad_clip': 1.0, (TaskRunner pid=2074004) 'kl_loss_coef': 0.001, (TaskRunner pid=2074004) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2074004) 'loss_scale_factor': None, (TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2074004) 'betas': [0.9, 0.999], (TaskRunner pid=2074004) 'clip_grad': 1.0, (TaskRunner pid=2074004) 'lr': 1e-06, (TaskRunner pid=2074004) 'lr_scheduler_type': 'constant', (TaskRunner pid=2074004) 'lr_warmup_steps': 10, (TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2074004) 'min_lr_ratio': 0.0, (TaskRunner pid=2074004) 'num_cycles': 0.5, (TaskRunner pid=2074004) 'optimizer': 'AdamW', (TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2074004) 'override_optimizer_config': None, (TaskRunner pid=2074004) 'total_training_steps': -1, (TaskRunner pid=2074004) 'warmup_style': None, (TaskRunner pid=2074004) 'weight_decay': 0.01}, (TaskRunner pid=2074004) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2074004) 'clip_cov_lb': 1.0, (TaskRunner pid=2074004) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2074004) 'clip_cov_ub': 5.0, (TaskRunner pid=2074004) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2074004) 'loss_mode': 'rlsd', (TaskRunner pid=2074004) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2074004) 'ppo_epochs': 1, (TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'ppo_micro_batch_size': None, (TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'ppo_mini_batch_size': 8, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2074004) 'mode': 'disabled', (TaskRunner pid=2074004) 'record_file': None, (TaskRunner pid=2074004) 'replay_file': None}, (TaskRunner pid=2074004) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2074004) 'alpha': 0.5, (TaskRunner pid=2074004) 'distillation_add_tail': True, (TaskRunner pid=2074004) 'distillation_topk': 100, (TaskRunner pid=2074004) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2074004) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2074004) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'loss_weight': 0.0, (TaskRunner pid=2074004) 'mask': 'all'}, (TaskRunner pid=2074004) 'feedback_template': '\n' (TaskRunner pid=2074004) 'The ' (TaskRunner pid=2074004) 'following ' (TaskRunner pid=2074004) 'is ' (TaskRunner pid=2074004) 'feedback ' (TaskRunner pid=2074004) 'from ' (TaskRunner pid=2074004) 'your ' (TaskRunner pid=2074004) 'unsuccessful ' (TaskRunner pid=2074004) 'earlier ' (TaskRunner pid=2074004) 'attempt:\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) '{feedback_raw}', (TaskRunner pid=2074004) 'full_logit_distillation': True, (TaskRunner pid=2074004) 'include_environment_feedback': True, (TaskRunner pid=2074004) 'is_clip': 2, (TaskRunner pid=2074004) 'max_reprompt_len': 10240, (TaskRunner pid=2074004) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2074004) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) 'Correctly ' (TaskRunner pid=2074004) 'solve ' (TaskRunner pid=2074004) 'the ' (TaskRunner pid=2074004) 'original ' (TaskRunner pid=2074004) 'question.', (TaskRunner pid=2074004) 'reprompt_truncation': 'right', (TaskRunner pid=2074004) 'solution_template': '\n' (TaskRunner pid=2074004) 'Correct ' (TaskRunner pid=2074004) 'solution:\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) '{successful_previous_attempt}', (TaskRunner pid=2074004) 'srpo_dynamic_weighting': False, (TaskRunner pid=2074004) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2074004) 'success_reward_threshold': 0.5, (TaskRunner pid=2074004) 'teacher_regularization': 'trust-region', (TaskRunner pid=2074004) 'teacher_update_rate': 0.1, (TaskRunner pid=2074004) 'token_reweight_decay_steps': 0, (TaskRunner pid=2074004) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2074004) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2074004) 'shuffle': False, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2074004) 'tau_neg': 1.05, (TaskRunner pid=2074004) 'tau_pos': 1.0, (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_kl_loss': False, (TaskRunner pid=2074004) 'use_prefix_grouper': False, (TaskRunner pid=2074004) 'use_remove_padding': True, (TaskRunner pid=2074004) 'use_torch_compile': True}, (TaskRunner pid=2074004) 'hybrid_engine': True, (TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2074004) 'custom_chat_template': None, (TaskRunner pid=2074004) 'enable_activation_offload': False, (TaskRunner pid=2074004) 'enable_gradient_checkpointing': True, (TaskRunner pid=2074004) 'exclude_modules': None, (TaskRunner pid=2074004) 'external_lib': None, (TaskRunner pid=2074004) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2074004) 'hf_config_path': None, (TaskRunner pid=2074004) 'lora_adapter_path': None, (TaskRunner pid=2074004) 'lora_alpha': 16, (TaskRunner pid=2074004) 'lora_rank': 0, (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'target_modules': 'all-linear', (TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2074004) 'num_shards': 4}, (TaskRunner pid=2074004) 'tokenizer_path': None, (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_liger': False, (TaskRunner pid=2074004) 'use_remove_padding': True, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'nccl_timeout': 600, (TaskRunner pid=2074004) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': True, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'log_prob_micro_batch_size': None, (TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2074004) 'mode': 'disabled', (TaskRunner pid=2074004) 'record_file': None, (TaskRunner pid=2074004) 'replay_file': None}, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_torch_compile': True}, (TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2074004) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2074004) 'agent_loop_config_path': None, (TaskRunner pid=2074004) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2074004) 'name': None, (TaskRunner pid=2074004) 'path': None}, (TaskRunner pid=2074004) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2074004) 'num_workers': 8}, (TaskRunner pid=2074004) 'calculate_log_probs': True, (TaskRunner pid=2074004) 'cudagraph_capture_sizes': None, (TaskRunner pid=2074004) 'data_parallel_size': 1, (TaskRunner pid=2074004) 'disable_log_stats': True, (TaskRunner pid=2074004) 'do_sample': True, (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'enable_chunked_prefill': True, (TaskRunner pid=2074004) 'enable_prefix_caching': True, (TaskRunner pid=2074004) 'enable_rollout_routing_replay': False, (TaskRunner pid=2074004) 'enforce_eager': False, (TaskRunner pid=2074004) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2074004) 'expert_parallel_size': 1, (TaskRunner pid=2074004) 'free_cache_engine': True, (TaskRunner pid=2074004) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2074004) 'ignore_eos': False, (TaskRunner pid=2074004) 'layered_summon': False, (TaskRunner pid=2074004) 'load_format': 'dummy', (TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'log_prob_micro_batch_size': None, (TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2074004) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2074004) 'max_model_len': 10240, (TaskRunner pid=2074004) 'max_num_batched_tokens': 10240, (TaskRunner pid=2074004) 'max_num_seqs': 1024, (TaskRunner pid=2074004) 'mode': 'async', (TaskRunner pid=2074004) 'multi_stage_wake_up': False, (TaskRunner pid=2074004) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'format': 'hermes', (TaskRunner pid=2074004) 'interaction_config_path': None, (TaskRunner pid=2074004) 'max_assistant_turns': None, (TaskRunner pid=2074004) 'max_parallel_calls': 1, (TaskRunner pid=2074004) 'max_tool_response_length': 256, (TaskRunner pid=2074004) 'max_user_turns': None, (TaskRunner pid=2074004) 'num_repeat_rollouts': None, (TaskRunner pid=2074004) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2074004) 'tool_config_path': None, (TaskRunner pid=2074004) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2074004) 'use_inference_chat_template': False}, (TaskRunner pid=2074004) 'n': 8, (TaskRunner pid=2074004) 'name': 'vllm', (TaskRunner pid=2074004) 'over_sample_rate': 0, (TaskRunner pid=2074004) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2074004) 'profiler': (TaskRunner pid=2074004) {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2074004) 'port': 9090, (TaskRunner pid=2074004) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2074004) 'prompt_length': 2048, (TaskRunner pid=2074004) 'quantization': None, (TaskRunner pid=2074004) 'quantization_config_file': None, (TaskRunner pid=2074004) 'response_length': 8192, (TaskRunner pid=2074004) 'scheduling_policy': 'fcfs', (TaskRunner pid=2074004) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2074004) 'skip_rollout': False, (TaskRunner pid=2074004) 'skip_tokenizer_init': True, (TaskRunner pid=2074004) 'temperature': 1.0, (TaskRunner pid=2074004) 'tensor_model_parallel_size': 2, (TaskRunner pid=2074004) 'top_k': -1, (TaskRunner pid=2074004) 'top_p': 1.0, (TaskRunner pid=2074004) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2074004) 'backend': None, (TaskRunner pid=2074004) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2074004) 'token2text': False}, (TaskRunner pid=2074004) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2074004) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2074004) 'do_sample': True, (TaskRunner pid=2074004) 'n': 16, (TaskRunner pid=2074004) 'temperature': 0.6, (TaskRunner pid=2074004) 'top_k': -1, (TaskRunner pid=2074004) 'top_p': 0.95}}}, (TaskRunner pid=2074004) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2074004) 'adv_estimator': 'grpo', (TaskRunner pid=2074004) 'gamma': 1.0, (TaskRunner pid=2074004) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2074004) 'horizon': 10000, (TaskRunner pid=2074004) 'kl_coef': 0.001, (TaskRunner pid=2074004) 'target_kl': 0.1, (TaskRunner pid=2074004) 'type': 'fixed'}, (TaskRunner pid=2074004) 'kl_penalty': 'kl', (TaskRunner pid=2074004) 'lam': 1.0, (TaskRunner pid=2074004) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2074004) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2074004) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2074004) 'loss_type': 'ppo_clip', (TaskRunner pid=2074004) 'rollout_is': 'token', (TaskRunner pid=2074004) 'rollout_is_batch_normalize': False, (TaskRunner pid=2074004) 'rollout_is_threshold': 2.0, (TaskRunner pid=2074004) 'rollout_rs': None, (TaskRunner pid=2074004) 'rollout_rs_threshold': None}, (TaskRunner pid=2074004) 'use_kl_in_reward': False, (TaskRunner pid=2074004) 'use_pf_ppo': False}, (TaskRunner pid=2074004) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2074004) 'async_save': False, (TaskRunner pid=2074004) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2074004) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2074004) 'cliprange_value': 0.5, (TaskRunner pid=2074004) 'data_loader_seed': 42, (TaskRunner pid=2074004) 'enable': None, (TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'forward_micro_batch_size': None, (TaskRunner pid=2074004) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'grad_clip': 1.0, (TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2074004) 'enable_activation_offload': False, (TaskRunner pid=2074004) 'enable_gradient_checkpointing': True, (TaskRunner pid=2074004) 'external_lib': None, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': False, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'lora_alpha': 16, (TaskRunner pid=2074004) 'lora_rank': 0, (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2074004) 'target_modules': 'all-linear', (TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2074004) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_remove_padding': False, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2074004) 'betas': [0.9, 0.999], (TaskRunner pid=2074004) 'clip_grad': 1.0, (TaskRunner pid=2074004) 'lr': 1e-05, (TaskRunner pid=2074004) 'lr_scheduler_type': 'constant', (TaskRunner pid=2074004) 'lr_warmup_steps': -1, (TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2074004) 'min_lr_ratio': 0.0, (TaskRunner pid=2074004) 'num_cycles': 0.5, (TaskRunner pid=2074004) 'optimizer': 'AdamW', (TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2074004) 'override_optimizer_config': None, (TaskRunner pid=2074004) 'total_training_steps': -1, (TaskRunner pid=2074004) 'warmup_style': None, (TaskRunner pid=2074004) 'weight_decay': 0.01}, (TaskRunner pid=2074004) 'ppo_epochs': 1, (TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'ppo_micro_batch_size': None, (TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'ppo_mini_batch_size': 8, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'shuffle': False, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False}, (TaskRunner pid=2074004) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2074004) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2074004) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2074004) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2074004) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2074004) 'dataloader_num_workers': 8, (TaskRunner pid=2074004) 'filter_overlong_prompts': True, (TaskRunner pid=2074004) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2074004) 'image_key': 'images', (TaskRunner pid=2074004) 'image_patch_size': 14, (TaskRunner pid=2074004) 'max_prompt_length': 2048, (TaskRunner pid=2074004) 'max_response_length': 8192, (TaskRunner pid=2074004) 'prompt_key': 'prompt', (TaskRunner pid=2074004) 'return_full_prompt': False, (TaskRunner pid=2074004) 'return_multi_modal_inputs': True, (TaskRunner pid=2074004) 'return_raw_chat': True, (TaskRunner pid=2074004) 'return_raw_input_ids': False, (TaskRunner pid=2074004) 'reward_fn_key': 'data_source', (TaskRunner pid=2074004) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2074004) 'seed': None, (TaskRunner pid=2074004) 'shuffle': True, (TaskRunner pid=2074004) 'tokenizer': None, (TaskRunner pid=2074004) 'tool_config_path': None, (TaskRunner pid=2074004) 'train_batch_size': 32, (TaskRunner pid=2074004) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2074004) 'train_max_samples': 3200, (TaskRunner pid=2074004) 'truncation': 'error', (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_shm': False, (TaskRunner pid=2074004) 'val_batch_size': None, (TaskRunner pid=2074004) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2074004) 'val_max_samples': -1, (TaskRunner pid=2074004) 'validation_shuffle': False, (TaskRunner pid=2074004) 'video_key': 'videos'}, (TaskRunner pid=2074004) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2074004) 'cuda-memory-usage': 'true', (TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2074004) 'capture-range-end': None, (TaskRunner pid=2074004) 'cuda-graph-trace': 'graph', (TaskRunner pid=2074004) 'cuda-memory-usage': 'true', (TaskRunner pid=2074004) 'kill': 'none', (TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2074004) 'torch_memory': {'context': 'all', (TaskRunner pid=2074004) 'kw_args': {}, (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'stacks': 'all', (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2074004) 'profile_continuous_steps': False, (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'steps': None, (TaskRunner pid=2074004) 'tool': None}, (TaskRunner pid=2074004) 'max_model_len': 10240, (TaskRunner pid=2074004) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B', (TaskRunner pid=2074004) 'include_dashboard': False, (TaskRunner pid=2074004) 'num_cpus': None, (TaskRunner pid=2074004) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2074004) 'USER': 'root'}}}, (TaskRunner pid=2074004) 'timeline_json_file': None}, (TaskRunner pid=2074004) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2074004) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2074004) 'name': 'custom_reward_manager', (TaskRunner pid=2074004) 'path': None}, (TaskRunner pid=2074004) 'name': 'naive', (TaskRunner pid=2074004) 'source': 'register'}, (TaskRunner pid=2074004) 'reward_model': {'enable': False, (TaskRunner pid=2074004) 'enable_resource_pool': False, (TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'launch_reward_fn_async': False, (TaskRunner pid=2074004) 'max_length': None, (TaskRunner pid=2074004) 'micro_batch_size': None, (TaskRunner pid=2074004) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'model': {'external_lib': None, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2074004) 'trust_remote_code': False, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_remove_padding': False, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'n_gpus_per_node': 8, (TaskRunner pid=2074004) 'nnodes': 0, (TaskRunner pid=2074004) 'num_workers': 1, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'reward_loop_class_name': None, (TaskRunner pid=2074004) 'reward_loop_module_path': None, (TaskRunner pid=2074004) 'reward_loop_source': 'register', (TaskRunner pid=2074004) 'reward_manager': 'naive', (TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2074004) 'cudagraph_capture_sizes': None, (TaskRunner pid=2074004) 'data_parallel_size': 1, (TaskRunner pid=2074004) 'disable_log_stats': True, (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'enable_chunked_prefill': True, (TaskRunner pid=2074004) 'enable_prefix_caching': True, (TaskRunner pid=2074004) 'enforce_eager': True, (TaskRunner pid=2074004) 'engine_kwargs': {}, (TaskRunner pid=2074004) 'expert_parallel_size': 1, (TaskRunner pid=2074004) 'free_cache_engine': True, (TaskRunner pid=2074004) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2074004) 'limit_images': None, (TaskRunner pid=2074004) 'load_format': 'auto', (TaskRunner pid=2074004) 'max_model_len': None, (TaskRunner pid=2074004) 'max_num_batched_tokens': 8192, (TaskRunner pid=2074004) 'max_num_seqs': 1024, (TaskRunner pid=2074004) 'name': '???', (TaskRunner pid=2074004) 'prompt_length': 2048, (TaskRunner pid=2074004) 'response_length': 2048, (TaskRunner pid=2074004) 'skip_tokenizer_init': False, (TaskRunner pid=2074004) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2074004) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2074004) 'memory_limit_mb': 1024, (TaskRunner pid=2074004) 'url': None}, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False, (TaskRunner pid=2074004) 'use_reward_loop': False}, (TaskRunner pid=2074004) 'trainer': {'balance_batch': True, (TaskRunner pid=2074004) 'critic_warmup': 0, (TaskRunner pid=2074004) 'default_hdfs_dir': None, (TaskRunner pid=2074004) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'del_local_ckpt_after_load': False, (TaskRunner pid=2074004) 'device': 'cuda', (TaskRunner pid=2074004) 'esi_redundant_time': 0, (TaskRunner pid=2074004) 'experiment_name': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2074004) 'log_val_generations': 0, (TaskRunner pid=2074004) 'logger': ['console', 'wandb'], (TaskRunner pid=2074004) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2074004) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2074004) 'n_gpus_per_node': 8, (TaskRunner pid=2074004) 'nnodes': 1, (TaskRunner pid=2074004) 'project_name': 'SDPO-root', (TaskRunner pid=2074004) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2074004) 'resume_from_path': None, (TaskRunner pid=2074004) 'resume_mode': 'auto', (TaskRunner pid=2074004) 'rollout_data_dir': None, (TaskRunner pid=2074004) 'save_freq': 10, (TaskRunner pid=2074004) 'test_freq': 10, (TaskRunner pid=2074004) 'total_epochs': 30, (TaskRunner pid=2074004) 'total_training_steps': 100, (TaskRunner pid=2074004) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2074004) 'val_before_train': False, (TaskRunner pid=2074004) 'val_only': False, (TaskRunner pid=2074004) 'validation_data_dir': None}, (TaskRunner pid=2074004) 'transfer_queue': {'enable': False}, (TaskRunner pid=2074004) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2074004) 'dir': '/users/root/SDPO', (TaskRunner pid=2074004) 'log_dir': '/users/root/output', (TaskRunner pid=2074004) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2074004) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2074004) use_critic=need_critic(config), (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2074004) Using dataset class: RLHFDataset (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2074004) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2074004) dataset len: 720 (TaskRunner pid=2074004) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2074004) WARNING:2026-07-02 08:56:38,173:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2074004) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 128.23 examples/s] (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2074004) self.use_critic = need_critic(self.config) (WorkerDict pid=2074388) [W702 08:56:47.781903761 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2074386) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2074387) [W702 08:56:47.990315263 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2074386) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2074385) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2074385) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2074385) Actor use_remove_padding=True (WorkerDict pid=2074385) Actor use_fused_kernels=False (WorkerDict pid=2074385) Actor use_prefix_grouper=False (WorkerDict pid=2074385) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2074385) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2074385) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2074387) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.99s/it] [repeated 12x across cluster] (WorkerDict pid=2074385) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster] (WorkerDict pid=2074387) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 4x across cluster] (WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2074385) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2074386) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2074391) warnings.warn( (WorkerDict pid=2074385) reference model: Qwen/Qwen3-4B (WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2074391) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2074385) Model config after override: Qwen3Config { (WorkerDict pid=2074385) "architectures": [ (WorkerDict pid=2074385) "Qwen3ForCausalLM" (WorkerDict pid=2074385) ], (WorkerDict pid=2074385) "attention_bias": false, (WorkerDict pid=2074385) "attention_dropout": 0.0, (WorkerDict pid=2074385) "dtype": "bfloat16", (WorkerDict pid=2074385) "eos_token_id": 151645, (WorkerDict pid=2074385) "head_dim": 128, (WorkerDict pid=2074385) "hidden_act": "silu", (WorkerDict pid=2074385) "hidden_size": 2560, (WorkerDict pid=2074385) "initializer_range": 0.02, (WorkerDict pid=2074385) "intermediate_size": 9728, (WorkerDict pid=2074385) "layer_types": [ (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention" (WorkerDict pid=2074385) ], (WorkerDict pid=2074385) "max_position_embeddings": 40960, (WorkerDict pid=2074385) "max_window_layers": 36, (WorkerDict pid=2074385) "model_type": "qwen3", (WorkerDict pid=2074385) "num_attention_heads": 32, (WorkerDict pid=2074385) "num_hidden_layers": 36, (WorkerDict pid=2074385) "num_key_value_heads": 8, (WorkerDict pid=2074385) "pad_token_id": 151643, (WorkerDict pid=2074385) "rms_norm_eps": 1e-06, (WorkerDict pid=2074385) "rope_scaling": null, (WorkerDict pid=2074385) "rope_theta": 1000000, (WorkerDict pid=2074385) "sliding_window": null, (WorkerDict pid=2074385) "tie_word_embeddings": true, (WorkerDict pid=2074385) "transformers_version": "4.57.1", (WorkerDict pid=2074385) "use_cache": true, (WorkerDict pid=2074385) "use_sliding_window": false, (WorkerDict pid=2074385) "vocab_size": 151936 (WorkerDict pid=2074385) } (WorkerDict pid=2074385) (WorkerDict pid=2074387) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2074385) Ref use_remove_padding=True (WorkerDict pid=2074385) Ref use_fused_kernels=False (WorkerDict pid=2074385) Ref use_prefix_grouper=False (WorkerDict pid=2074392) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2074392) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2074392) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:06<00:03, 3.02s/it] [repeated 14x across cluster] (WorkerDict pid=2074392) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.04s/it] [repeated 6x across cluster] (vLLMHttpServer pid=2075282) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2075282) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2075282) ['serve', (vLLMHttpServer pid=2075282) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2075282) '--dtype', (vLLMHttpServer pid=2075282) 'bfloat16', (vLLMHttpServer pid=2075282) '--load_format', (vLLMHttpServer pid=2075282) 'dummy', (vLLMHttpServer pid=2075282) '--trust_remote_code', (vLLMHttpServer pid=2075282) '--max_model_len', (vLLMHttpServer pid=2075282) '40960', (vLLMHttpServer pid=2075282) '--max_num_seqs', (vLLMHttpServer pid=2075282) '1024', (vLLMHttpServer pid=2075282) '--enable_chunked_prefill', (vLLMHttpServer pid=2075282) '--max_num_batched_tokens', (vLLMHttpServer pid=2075282) '10240', (vLLMHttpServer pid=2075282) '--enable_prefix_caching', (vLLMHttpServer pid=2075282) '--enable_sleep_mode', (vLLMHttpServer pid=2075282) '--logprobs_mode', (vLLMHttpServer pid=2075282) 'processed_logprobs', (vLLMHttpServer pid=2075282) '--disable_custom_all_reduce', (vLLMHttpServer pid=2075282) '--gpu_memory_utilization', (vLLMHttpServer pid=2075282) '0.8', (vLLMHttpServer pid=2075282) '--disable_log_stats', (vLLMHttpServer pid=2075282) '--tensor_parallel_size', (vLLMHttpServer pid=2075282) '2', (vLLMHttpServer pid=2075282) '--seed', (vLLMHttpServer pid=2075282) '0', (vLLMHttpServer pid=2075282) '--override_generation_config', (vLLMHttpServer pid=2075282) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2075282) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2075282) '--hf_overrides', (vLLMHttpServer pid=2075282) '{}', (vLLMHttpServer pid=2075282) '--scheduling_policy', (vLLMHttpServer pid=2075282) 'fcfs'] (vLLMHttpServer pid=2075282) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2075282) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2075282) WARNING 07-02 08:57:45 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2074385) WARNING 07-02 08:57:52 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2075287) WARNING 07-02 08:57:46 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2074389) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/aotautograd/aqwmczjqsm4pvwfimkpe3rswyj2xfwbwedbijxvwd7tfo5n2eqnk/.2074388.133380852872896.tmp' (vLLMHttpServer pid=2075287) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2075287) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2075287) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2075287) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2074385) 2026-07-02 08:58:08,146 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2074386) 2026-07-02 08:58:08,165 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Rearranging to solve for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the known values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest option is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2074004) step:10 - global_seqlen/min:14569 - global_seqlen/max:25213 - global_seqlen/minmax_diff:10644 - global_seqlen/balanced_min:21068 - global_seqlen/balanced_max:21078 - global_seqlen/mean:21074.625 - actor/entropy:0.13031111657619476 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5310845375061035 - training/rollout_probs_diff_mean:0.0036879268009215593 - training/rollout_probs_diff_std:0.013145294040441513 - training/rollout_actor_probs_pearson_corr:0.9974448680877686 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999222755432129 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00015043777239043266 - rollout_corr/rollout_is_max:1.8829189538955688 - rollout_corr/rollout_is_min:0.005527450703084469 - rollout_corr/rollout_is_std:0.03613611310720444 - rollout_corr/rollout_is_eff_sample_size:0.9986955276850646 - rollout_corr/rollout_is_seq_mean:0.9999170899391174 - rollout_corr/rollout_is_seq_std:0.002527824370190501 - rollout_corr/rollout_is_seq_max:1.0131255388259888 - rollout_corr/rollout_is_seq_min:0.9898325800895691 - rollout_corr/rollout_is_seq_max_deviation:0.01312553882598877 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1374189392663538) - rollout_corr/training_log_ppl:np.float64(0.12561230757273734) - rollout_corr/kl:np.float64(0.0010259306004876123) - rollout_corr/k3_kl:np.float64(0.001012646021507635) - rollout_corr/rollout_ppl:np.float64(1.1362624615430832) - rollout_corr/rollout_log_ppl:np.float64(0.1245863765507238) - rollout_corr/log_ppl_diff:np.float64(0.0010259310220135376) - rollout_corr/log_ppl_abs_diff:np.float64(0.00206260806589853) - rollout_corr/log_ppl_diff_max:np.float64(0.03187315911054611) - rollout_corr/log_ppl_diff_min:np.float64(-0.010174185037612915) - rollout_corr/ppl_ratio:np.float64(1.0010317743290216) - rollout_corr/chi2_token:np.float64(0.0018072936218231916) - rollout_corr/chi2_seq:np.float64(0.3196559448260814) - actor/pg_loss:np.float64(0.0017917423974722624) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004963221986145072) - actor/ppo_kl:np.float64(8.269873148769591e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(95332.59919399978) - self_distillation/token_reweight_w_std:np.float64(1506740.680497941) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996795877814293) - self_distillation/token_reweight_w_clip_frac:np.float64(0.033084604699979536) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.3074006289243698) - perf/mfu/actor:np.float64(0.03828182711170613) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.1017189025879) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.596875) - val-aux/sciknoweval/reward/std@16:np.float64(0.17025334585854696) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6700875) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14365860035136047) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.525375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14433762594467914) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5972999999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16952408520330536) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.72475) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1008625343689749) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4628500000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10408901059613167) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6178625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1414437668650514) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7605999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0665978685707884) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4221375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0550829574380763) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6353249999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10804651530040861) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7887749999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.044373467337921746) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40466250000000004) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.019017006085694965) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6468625) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07892990828797705) - val-aux/sciknoweval/score/mean@16:np.float64(0.596875) - val-aux/sciknoweval/score/std@16:np.float64(0.17025334585854696) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6700875) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14365860035136047) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.525375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14433762594467914) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5972999999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16952408520330536) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.72475) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1008625343689749) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4628500000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10408901059613167) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6178625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1414437668650514) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7605999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0665978685707884) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4221375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0550829574380763) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6353249999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10804651530040861) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7887749999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.044373467337921746) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40466250000000004) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.019017006085694965) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6468625) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07892990828797705) - val-core/sciknoweval/acc/mean@16:np.float64(0.596875) - val-aux/sciknoweval/acc/std@16:np.float64(0.17025334585854696) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6700875) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14365860035136047) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.525375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14433762594467914) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5972999999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16952408520330536) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.72475) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1008625343689749) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4628500000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10408901059613167) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6178625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1414437668650514) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7605999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0665978685707884) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4221375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0550829574380763) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6353249999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10804651530040861) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7887749999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.044373467337921746) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40466250000000004) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.019017006085694965) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6468625) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07892990828797705) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.02228859585888569) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0055870721336611015) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9884875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.02914679194594295) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9939625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.021812516180988194) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9776375000000002) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03766353637974691) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.997625) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.013472998070955988) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9620000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04262616078525806) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9992875) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005053486156825662) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9403500000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03977205801345875) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998875) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0015086398882979818) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032043245155364275 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032043245155364275 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:389.48828125 - response_length/max:1190.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:389.48828125 - response_length_non_aborted/max:1190.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015878118574619293 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1143263150006533) - timing_s/agent_loop/generate_sequences/max:np.float64(8.482053555548191) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.887139186161221) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.482053555548191) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:337 - timing_s/agent_loop/slowest/response_length:1190 - timing_s/gen:14.678978031501174 - timing_s/reward:0.05808144994080067 - timing_s/old_log_prob:2.4217102844268084 - timing_s/adv:0.5421234201639891 - timing_s/update_actor:15.863357232883573 - timing_s/step:33.64816468022764 - timing_s/testing:93.59819382615387 - timing_s/save_checkpoint:6.729555023834109 - timing_s/stop_profile:0.00014853663742542267 - timing_per_token_ms/update_actor:0.09409038851749184 - timing_per_token_ms/adv:0.003215498616013269 - timing_per_token_ms/gen:0.14721818523404281 - perf/total_num_tokens:168597 - perf/time_per_step:33.64816468022764 - perf/throughput:626.3231650308669 (TaskRunner pid=2074004) Training Progress: 10%|█ | 10/100 [08:42<1:43:00, 68.67s/it] (TaskRunner pid=2074004) step:11 - global_seqlen/min:15931 - global_seqlen/max:27826 - global_seqlen/minmax_diff:11895 - global_seqlen/balanced_min:22821 - global_seqlen/balanced_max:22841 - global_seqlen/mean:22835.0 - actor/entropy:0.12203283607959747 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40049630403518677 - training/rollout_probs_diff_mean:0.003223797306418419 - training/rollout_probs_diff_std:0.012214891612529755 - training/rollout_actor_probs_pearson_corr:0.9976435303688049 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9997804164886475 - rollout_corr/rollout_is_ratio_fraction_high:2.6688492653192952e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.3376985306385905e-05 - rollout_corr/rollout_is_max:2.5879318714141846 - rollout_corr/rollout_is_min:0.06322140991687775 - rollout_corr/rollout_is_std:0.033195044845342636 - rollout_corr/rollout_is_eff_sample_size:0.9988987665423175 - rollout_corr/rollout_is_seq_mean:0.9999120235443115 - rollout_corr/rollout_is_seq_std:0.002345130778849125 - rollout_corr/rollout_is_seq_max:1.0110429525375366 - rollout_corr/rollout_is_seq_min:0.9912058711051941 - rollout_corr/rollout_is_seq_max_deviation:0.011042952537536621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1257328824140131) - rollout_corr/training_log_ppl:np.float64(0.1140785860188771) - rollout_corr/kl:np.float64(0.0006342849212974677) - rollout_corr/k3_kl:np.float64(0.0007503947783931153) - rollout_corr/rollout_ppl:np.float64(1.1249900460243225) - rollout_corr/rollout_log_ppl:np.float64(0.11344430113604176) - rollout_corr/log_ppl_diff:np.float64(0.0006342848828353453) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019007355404028203) - rollout_corr/log_ppl_diff_max:np.float64(0.010488316416740417) - rollout_corr/log_ppl_diff_min:np.float64(-0.008857212960720062) - rollout_corr/ppl_ratio:np.float64(1.0006378127727658) - rollout_corr/chi2_token:np.float64(0.001916841370984912) - rollout_corr/chi2_seq:np.float64(0.37116734171286225) - actor/pg_loss:np.float64(0.0027762840036302805) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005819873397285846) - actor/ppo_kl:np.float64(-7.081770549888233e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(90827.28597010626) - self_distillation/token_reweight_w_std:np.float64(1824642.9953269707) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001720192376524) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04453263468167279) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.29148364439606667) - perf/mfu/actor:np.float64(0.04050194858164669) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.13693618774414) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004592644691001624 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004592644691001624 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.09375 - response_length/max:1273.0 - response_length/min:68.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.09375 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.3513795137405396e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6711478792130947) - timing_s/agent_loop/generate_sequences/max:np.float64(9.547774223610759) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.419399234910088) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.547774223610759) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1273 - timing_s/gen:15.524424999952316 - timing_s/reward:0.057088837027549744 - timing_s/old_log_prob:2.659148082137108 - timing_s/adv:0.5871537029743195 - timing_s/update_actor:16.28799468278885 - timing_s/step:35.222145142033696 - timing_s/stop_profile:0.0001278892159461975 - timing_per_token_ms/update_actor:0.0891613459754152 - timing_per_token_ms/adv:0.003214110482670897 - timing_per_token_ms/gen:0.13810783040310579 - perf/total_num_tokens:182680 - perf/time_per_step:35.222145142033696 - perf/throughput:648.3137216066087 (TaskRunner pid=2074004) Training Progress: 11%|█ | 11/100 [09:18<1:26:42, 58.45s/it] (TaskRunner pid=2074004) step:12 - global_seqlen/min:16305 - global_seqlen/max:30523 - global_seqlen/minmax_diff:14218 - global_seqlen/balanced_min:22309 - global_seqlen/balanced_max:28829 - global_seqlen/mean:23127.125 - actor/entropy:0.12043065577745438 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6972464323043823 - training/rollout_probs_diff_mean:0.0032291633542627096 - training/rollout_probs_diff_std:0.012314102612435818 - training/rollout_actor_probs_pearson_corr:0.9975891709327698 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999233484268188 - rollout_corr/rollout_is_ratio_fraction_high:6.790191400796175e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5463217752985656e-05 - rollout_corr/rollout_is_max:3.958061456680298 - rollout_corr/rollout_is_min:0.3131382167339325 - rollout_corr/rollout_is_std:0.033765628933906555 - rollout_corr/rollout_is_eff_sample_size:0.9988609428668661 - rollout_corr/rollout_is_seq_mean:0.9999712705612183 - rollout_corr/rollout_is_seq_std:0.0021552243269979954 - rollout_corr/rollout_is_seq_max:1.0135027170181274 - rollout_corr/rollout_is_seq_min:0.9933554530143738 - rollout_corr/rollout_is_seq_max_deviation:0.013502717018127441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1240018154494464) - rollout_corr/training_log_ppl:np.float64(0.11423524462952628) - rollout_corr/kl:np.float64(0.0006360744373044014) - rollout_corr/k3_kl:np.float64(0.0007582616624119964) - rollout_corr/rollout_ppl:np.float64(1.1232725959271193) - rollout_corr/rollout_log_ppl:np.float64(0.11359916946094017) - rollout_corr/log_ppl_diff:np.float64(0.0006360751685861032) - rollout_corr/log_ppl_abs_diff:np.float64(0.001725419224385405) - rollout_corr/log_ppl_diff_max:np.float64(0.009453028440475464) - rollout_corr/log_ppl_diff_min:np.float64(-0.006760995835065842) - rollout_corr/ppl_ratio:np.float64(1.0006386556196958) - rollout_corr/chi2_token:np.float64(0.0019189040176570415) - rollout_corr/chi2_seq:np.float64(1.096296020084992) - actor/pg_loss:np.float64(0.0028759860433638096) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007010218323557638) - actor/ppo_kl:np.float64(-5.775291747767142e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(54891.713903531665) - self_distillation/token_reweight_w_std:np.float64(995585.2390469867) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996340507641435) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05258868928649463) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.3390292413532734) - perf/mfu/actor:np.float64(0.03925409114627591) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.01633071899414) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022134963423013687 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022134963423013687 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:460.22265625 - response_length/max:8192.0 - response_length/min:78.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:460.22265625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.100325405597687e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.941336028277874) - timing_s/agent_loop/generate_sequences/max:np.float64(50.94393623247743) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.578305393028131) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.94393623247743) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.6255786344409 - timing_s/reward:0.06343746557831764 - timing_s/old_log_prob:2.6843459140509367 - timing_s/adv:0.540890708565712 - timing_s/update_actor:17.096599811688066 - timing_s/step:77.09979971125722 - timing_s/stop_profile:0.00011315196752548218 - timing_per_token_ms/update_actor:0.09240556171426445 - timing_per_token_ms/adv:0.002923464917092548 - timing_per_token_ms/gen:0.480623158240669 - perf/total_num_tokens:185017 - perf/time_per_step:77.09979971125722 - perf/throughput:299.96349000402455 (TaskRunner pid=2074004) Training Progress: 12%|█▏ | 12/100 [10:35<1:34:04, 64.14s/it] (TaskRunner pid=2074004) step:13 - global_seqlen/min:18538 - global_seqlen/max:25997 - global_seqlen/minmax_diff:7459 - global_seqlen/balanced_min:22090 - global_seqlen/balanced_max:22116 - global_seqlen/mean:22109.375 - actor/entropy:0.14604653418064117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7997847199440002 - training/rollout_probs_diff_mean:0.0036600856110453606 - training/rollout_probs_diff_std:0.012734896503388882 - training/rollout_actor_probs_pearson_corr:0.9978328943252563 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.703125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.703125 - rollout_corr/rollout_is_mean:0.9999634623527527 - rollout_corr/rollout_is_ratio_fraction_high:2.843844413291663e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.531533239874989e-05 - rollout_corr/rollout_is_max:2.8094775676727295 - rollout_corr/rollout_is_min:6.210129504324868e-05 - rollout_corr/rollout_is_std:0.03579304739832878 - rollout_corr/rollout_is_eff_sample_size:0.9987203780861295 - rollout_corr/rollout_is_seq_mean:0.9999745488166809 - rollout_corr/rollout_is_seq_std:0.002158660674467683 - rollout_corr/rollout_is_seq_max:1.0080207586288452 - rollout_corr/rollout_is_seq_min:0.9927275776863098 - rollout_corr/rollout_is_seq_max_deviation:0.008020758628845215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1676474194973707) - rollout_corr/training_log_ppl:np.float64(0.14873290639661718) - rollout_corr/kl:np.float64(0.0009227445474575546) - rollout_corr/k3_kl:np.float64(0.0009155499404727152) - rollout_corr/rollout_ppl:np.float64(1.1665417086333036) - rollout_corr/rollout_log_ppl:np.float64(0.14781016217602883) - rollout_corr/log_ppl_diff:np.float64(0.0009227442205883563) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018867802718887106) - rollout_corr/log_ppl_diff_max:np.float64(0.060229599475860596) - rollout_corr/log_ppl_diff_min:np.float64(-0.004808485507965088) - rollout_corr/ppl_ratio:np.float64(1.0009324494749308) - rollout_corr/chi2_token:np.float64(0.0014858830254524946) - rollout_corr/chi2_seq:np.float64(0.4379577168729156) - actor/pg_loss:np.float64(0.002721810364164412) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005266812750051031) - actor/ppo_kl:np.float64(1.293608784180833e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(196224.12132495665) - self_distillation/token_reweight_w_std:np.float64(2690730.0869071595) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001536855706945) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06831683132622857) - self_distillation/empty_target_batch:np.float64(0.296875) - actor/grad_norm:np.float64(0.4452543556690216) - perf/mfu/actor:np.float64(0.03950919290459659) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.9030246734619) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021506573539227247 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021506573539227247 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:412.07421875 - response_length/max:1343.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.07421875 - response_length_non_aborted/max:1343.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001416429877281189 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2486017309129238) - timing_s/agent_loop/generate_sequences/max:np.float64(9.4201366789639) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.31274351340835) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.4201366789639) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1343 - timing_s/gen:15.389248048886657 - timing_s/reward:0.059297624975442886 - timing_s/old_log_prob:2.4455661196261644 - timing_s/adv:0.5371290631592274 - timing_s/update_actor:16.235428664833307 - timing_s/step:34.75727551802993 - timing_s/stop_profile:3.674998879432678e-05 - timing_per_token_ms/update_actor:0.09179040941248513 - timing_per_token_ms/adv:0.003036772088532734 - timing_per_token_ms/gen:0.14588209467050892 - perf/total_num_tokens:176875 - perf/time_per_step:34.75727551802993 - perf/throughput:636.107826936292 (TaskRunner pid=2074004) Training Progress: 13%|█▎ | 13/100 [11:09<1:20:06, 55.24s/it] (TaskRunner pid=2074004) step:14 - global_seqlen/min:15836 - global_seqlen/max:27136 - global_seqlen/minmax_diff:11300 - global_seqlen/balanced_min:21152 - global_seqlen/balanced_max:21214 - global_seqlen/mean:21164.375 - actor/entropy:0.13442379236221313 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967105984687805 - training/rollout_probs_diff_mean:0.003770587034523487 - training/rollout_probs_diff_std:0.013384245336055756 - training/rollout_actor_probs_pearson_corr:0.9973688125610352 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0001509189605713 - rollout_corr/rollout_is_ratio_fraction_high:3.0175318897818215e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012070127559127286 - rollout_corr/rollout_is_max:3.7234184741973877 - rollout_corr/rollout_is_min:0.041150957345962524 - rollout_corr/rollout_is_std:0.03739259019494057 - rollout_corr/rollout_is_eff_sample_size:0.9986041031225663 - rollout_corr/rollout_is_seq_mean:1.000020146369934 - rollout_corr/rollout_is_seq_std:0.002401510952040553 - rollout_corr/rollout_is_seq_max:1.0093960762023926 - rollout_corr/rollout_is_seq_min:0.9911119341850281 - rollout_corr/rollout_is_seq_max_deviation:0.009396076202392578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1523799053393304) - rollout_corr/training_log_ppl:np.float64(0.13728270668070763) - rollout_corr/kl:np.float64(0.0008854879360526979) - rollout_corr/k3_kl:np.float64(0.0008427418540151166) - rollout_corr/rollout_ppl:np.float64(1.1513059278950095) - rollout_corr/rollout_log_ppl:np.float64(0.13639721802610438) - rollout_corr/log_ppl_diff:np.float64(0.0008854886546032503) - rollout_corr/log_ppl_abs_diff:np.float64(0.001983832713449374) - rollout_corr/log_ppl_diff_max:np.float64(0.011519268155097961) - rollout_corr/log_ppl_diff_min:np.float64(-0.00734923779964447) - rollout_corr/ppl_ratio:np.float64(1.0008891641627997) - rollout_corr/chi2_token:np.float64(0.001656870823353529) - rollout_corr/chi2_seq:np.float64(1.6609382745809853) - actor/pg_loss:np.float64(0.004066150169819593) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009369107503971463) - actor/ppo_kl:np.float64(0.00011594514723167748) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(220119.85480736103) - self_distillation/token_reweight_w_std:np.float64(3045461.0947162565) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001637001987547) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08081251988187432) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.4899965673685074) - perf/mfu/actor:np.float64(0.03798268804792023) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.0521469116211) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005462990142405033 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005462990142405033 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:388.35546875 - response_length/max:1228.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:388.35546875 - response_length_non_aborted/max:1228.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.705312550067902e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7120892163366079) - timing_s/agent_loop/generate_sequences/max:np.float64(8.587548367679119) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.029178587690694) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.587548367679119) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1228 - timing_s/gen:14.909624475985765 - timing_s/reward:0.05932668782770634 - timing_s/old_log_prob:2.404671125113964 - timing_s/adv:0.5401296932250261 - timing_s/update_actor:16.007855907082558 - timing_s/step:34.01097614504397 - timing_s/stop_profile:0.00011675432324409485 - timing_per_token_ms/update_actor:0.09454481828002574 - timing_per_token_ms/adv:0.003190087666332139 - timing_per_token_ms/gen:0.14996755626173836 - perf/total_num_tokens:169315 - perf/time_per_step:34.01097614504397 - perf/throughput:622.2807281314695 (TaskRunner pid=2074004) Training Progress: 14%|█▍ | 14/100 [11:43<1:10:00, 48.85s/it] (TaskRunner pid=2074004) step:15 - global_seqlen/min:13000 - global_seqlen/max:27017 - global_seqlen/minmax_diff:14017 - global_seqlen/balanced_min:21238 - global_seqlen/balanced_max:21268 - global_seqlen/mean:21253.875 - actor/entropy:0.11480943113565445 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772899866104126 - training/rollout_probs_diff_mean:0.0029744282364845276 - training/rollout_probs_diff_std:0.01138862781226635 - training/rollout_actor_probs_pearson_corr:0.9978798031806946 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:0.9999694228172302 - rollout_corr/rollout_is_ratio_fraction_high:2.9455948606482707e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.9455948606482707e-05 - rollout_corr/rollout_is_max:2.554089307785034 - rollout_corr/rollout_is_min:0.44323766231536865 - rollout_corr/rollout_is_std:0.032444074749946594 - rollout_corr/rollout_is_eff_sample_size:0.998948250895511 - rollout_corr/rollout_is_seq_mean:1.000088095664978 - rollout_corr/rollout_is_seq_std:0.001991908298805356 - rollout_corr/rollout_is_seq_max:1.0099899768829346 - rollout_corr/rollout_is_seq_min:0.9942845106124878 - rollout_corr/rollout_is_seq_max_deviation:0.00998997688293457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.126635740045458) - rollout_corr/training_log_ppl:np.float64(0.11509814600140089) - rollout_corr/kl:np.float64(0.0005166397006295398) - rollout_corr/k3_kl:np.float64(0.0006518633542142993) - rollout_corr/rollout_ppl:np.float64(1.126022334676236) - rollout_corr/rollout_log_ppl:np.float64(0.11458150647194998) - rollout_corr/log_ppl_diff:np.float64(0.000516639529450913) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016528939577256097) - rollout_corr/log_ppl_diff_max:np.float64(0.007337689399719238) - rollout_corr/log_ppl_diff_min:np.float64(-0.009172983467578888) - rollout_corr/ppl_ratio:np.float64(1.0005192304961383) - rollout_corr/chi2_token:np.float64(0.0016447578091174364) - rollout_corr/chi2_seq:np.float64(0.39781012921594083) - actor/pg_loss:np.float64(0.002455260371789336) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006204967883149948) - actor/ppo_kl:np.float64(4.5773536659687863e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(192128.32348678238) - self_distillation/token_reweight_w_std:np.float64(3233259.402355935) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002948227338493) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06099457883101422) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.3918348103761673) - perf/mfu/actor:np.float64(0.03843189897453754) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.9206657409668) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015439826529473066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015439826529473066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:397.83984375 - response_length/max:1327.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:397.83984375 - response_length_non_aborted/max:1327.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.262189269065857e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7463704198598862) - timing_s/agent_loop/generate_sequences/max:np.float64(8.988874156028032) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.964175615350541) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.988874156028032) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1327 - timing_s/gen:14.802029374986887 - timing_s/reward:0.06032639183104038 - timing_s/old_log_prob:2.3973333202302456 - timing_s/adv:0.5653208959847689 - timing_s/update_actor:15.675101241096854 - timing_s/step:33.58521797135472 - timing_s/stop_profile:3.232806921005249e-05 - timing_per_token_ms/update_actor:0.09218966683191214 - timing_per_token_ms/adv:0.0033248107461861004 - timing_per_token_ms/gen:0.14533593895732705 - perf/total_num_tokens:170031 - perf/time_per_step:33.58521797135472 - perf/throughput:632.8342134961789 (TaskRunner pid=2074004) Training Progress: 15%|█▌ | 15/100 [12:17<1:02:41, 44.25s/it] (TaskRunner pid=2074004) step:16 - global_seqlen/min:16611 - global_seqlen/max:28011 - global_seqlen/minmax_diff:11400 - global_seqlen/balanced_min:20244 - global_seqlen/balanced_max:26855 - global_seqlen/mean:21153.75 - actor/entropy:0.12396589666604996 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8040750622749329 - training/rollout_probs_diff_mean:0.0032445986289530993 - training/rollout_probs_diff_std:0.012475800700485706 - training/rollout_actor_probs_pearson_corr:0.9975688457489014 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:0.9998238682746887 - rollout_corr/rollout_is_ratio_fraction_high:3.8920348742976785e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.730087185744196e-05 - rollout_corr/rollout_is_max:2.290397882461548 - rollout_corr/rollout_is_min:3.1829338240640936e-06 - rollout_corr/rollout_is_std:0.03366485983133316 - rollout_corr/rollout_is_eff_sample_size:0.9988676034321022 - rollout_corr/rollout_is_seq_mean:0.9998279809951782 - rollout_corr/rollout_is_seq_std:0.002493756590411067 - rollout_corr/rollout_is_seq_max:1.0092169046401978 - rollout_corr/rollout_is_seq_min:0.9886294007301331 - rollout_corr/rollout_is_seq_max_deviation:0.011370599269866943 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1432740236632526) - rollout_corr/training_log_ppl:np.float64(0.12944419973246113) - rollout_corr/kl:np.float64(0.00073082422317583) - rollout_corr/k3_kl:np.float64(0.00073855356768604) - rollout_corr/rollout_ppl:np.float64(1.1423929436132312) - rollout_corr/rollout_log_ppl:np.float64(0.1287133757614356) - rollout_corr/log_ppl_diff:np.float64(0.0007308239710255293) - rollout_corr/log_ppl_abs_diff:np.float64(0.001978938538741204) - rollout_corr/log_ppl_diff_max:np.float64(0.010752767324447632) - rollout_corr/log_ppl_diff_min:np.float64(-0.0069452449679374695) - rollout_corr/ppl_ratio:np.float64(1.000734378118068) - rollout_corr/chi2_token:np.float64(0.001494420226663351) - rollout_corr/chi2_seq:np.float64(0.7517771443817765) - actor/pg_loss:np.float64(0.002636288059875369) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006587515040337166) - actor/ppo_kl:np.float64(-0.0003202963638955225) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(219386.07391313673) - self_distillation/token_reweight_w_std:np.float64(3319235.12259777) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0016510419081897) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05755699300789274) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.43999863415956497) - perf/mfu/actor:np.float64(0.03636736138421752) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.8370819091797) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04519382491707802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04519382491707802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:401.4609375 - response_length/max:8192.0 - response_length/min:97.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:401.4609375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.450495958328247e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.158904004842043) - timing_s/agent_loop/generate_sequences/max:np.float64(50.02395389974117) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.92313740587997) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.02395389974117) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.74822269193828 - timing_s/reward:0.05993708409368992 - timing_s/old_log_prob:2.6834859494119883 - timing_s/adv:0.5374118015170097 - timing_s/update_actor:16.872027203440666 - timing_s/step:75.98889152519405 - timing_s/stop_profile:0.00010964088141918182 - timing_per_token_ms/update_actor:0.09969879574213003 - timing_per_token_ms/adv:0.003175629625462446 - timing_per_token_ms/gen:0.5424350778595587 - perf/total_num_tokens:169230 - perf/time_per_step:75.98889152519405 - perf/throughput:278.37950489100757 (TaskRunner pid=2074004) Training Progress: 16%|█▌ | 16/100 [13:33<1:15:20, 53.82s/it] (TaskRunner pid=2074004) step:17 - global_seqlen/min:16383 - global_seqlen/max:22557 - global_seqlen/minmax_diff:6174 - global_seqlen/balanced_min:19789 - global_seqlen/balanced_max:19793 - global_seqlen/mean:19791.0 - actor/entropy:0.1373070627450943 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535119295120239 - training/rollout_probs_diff_mean:0.0037673222832381725 - training/rollout_probs_diff_std:0.013155417516827583 - training/rollout_actor_probs_pearson_corr:0.997542679309845 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:5.546680768020451e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.765353075228631e-05 - rollout_corr/rollout_is_max:2.7404403686523438 - rollout_corr/rollout_is_min:0.32417601346969604 - rollout_corr/rollout_is_std:0.037032198160886765 - rollout_corr/rollout_is_eff_sample_size:0.9986302566846117 - rollout_corr/rollout_is_seq_mean:1.0000929832458496 - rollout_corr/rollout_is_seq_std:0.0029761020559817553 - rollout_corr/rollout_is_seq_max:1.0141609907150269 - rollout_corr/rollout_is_seq_min:0.9883509874343872 - rollout_corr/rollout_is_seq_max_deviation:0.014160990715026855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1427205740474164) - rollout_corr/training_log_ppl:np.float64(0.13067101137858117) - rollout_corr/kl:np.float64(0.001165712662167806) - rollout_corr/k3_kl:np.float64(0.0011212053367444241) - rollout_corr/rollout_ppl:np.float64(1.1413688631728292) - rollout_corr/rollout_log_ppl:np.float64(0.12950529891531914) - rollout_corr/log_ppl_diff:np.float64(0.0011657124632620253) - rollout_corr/log_ppl_abs_diff:np.float64(0.002574236517830286) - rollout_corr/log_ppl_diff_max:np.float64(0.018865570425987244) - rollout_corr/log_ppl_diff_min:np.float64(-0.009756885468959808) - rollout_corr/ppl_ratio:np.float64(1.0011729013640434) - rollout_corr/chi2_token:np.float64(0.001890765968710184) - rollout_corr/chi2_seq:np.float64(1.113262691302225) - actor/pg_loss:np.float64(0.004973054630681872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013291486598063784) - actor/ppo_kl:np.float64(0.000477771908172997) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(430373.50576436426) - self_distillation/token_reweight_w_std:np.float64(5256699.274065455) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005029244348407) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08690081464010291) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.691158652305603) - perf/mfu/actor:np.float64(0.03633210791443406) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.84893798828125) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020605919416993856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020605919416993856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.125 - response_length/max:919.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.125 - response_length_non_aborted/max:919.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.005889296531677e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8571333661675453) - timing_s/agent_loop/generate_sequences/max:np.float64(7.130504677072167) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.622363231239433) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.130504677072167) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:919 - timing_s/gen:12.924050455912948 - timing_s/reward:0.059431292116642 - timing_s/old_log_prob:2.4258709643036127 - timing_s/adv:0.542781762778759 - timing_s/update_actor:15.836449747905135 - timing_s/step:31.876996461302042 - timing_s/stop_profile:0.0001156199723482132 - timing_per_token_ms/update_actor:0.10002305181588307 - timing_per_token_ms/adv:0.003428210820440851 - timing_per_token_ms/gen:0.14337116675444786 - perf/total_num_tokens:158328 - perf/time_per_step:31.876996461302042 - perf/throughput:620.8552309508152 (TaskRunner pid=2074004) Training Progress: 17%|█▋ | 17/100 [14:05<1:05:20, 47.23s/it] (TaskRunner pid=2074004) step:18 - global_seqlen/min:15614 - global_seqlen/max:27961 - global_seqlen/minmax_diff:12347 - global_seqlen/balanced_min:22588 - global_seqlen/balanced_max:22618 - global_seqlen/mean:22607.75 - actor/entropy:0.11930368840694427 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7529500722885132 - training/rollout_probs_diff_mean:0.0031947942916303873 - training/rollout_probs_diff_std:0.01253251638263464 - training/rollout_actor_probs_pearson_corr:0.9974825382232666 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999328255653381 - rollout_corr/rollout_is_ratio_fraction_high:3.530138565110974e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015885622997302562 - rollout_corr/rollout_is_max:2.3141205310821533 - rollout_corr/rollout_is_min:0.1902111917734146 - rollout_corr/rollout_is_std:0.03377268835902214 - rollout_corr/rollout_is_eff_sample_size:0.998860586053378 - rollout_corr/rollout_is_seq_mean:1.000009298324585 - rollout_corr/rollout_is_seq_std:0.0019673723727464676 - rollout_corr/rollout_is_seq_max:1.007437825202942 - rollout_corr/rollout_is_seq_min:0.9938141703605652 - rollout_corr/rollout_is_seq_max_deviation:0.0074378252029418945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.129554508253932) - rollout_corr/training_log_ppl:np.float64(0.11883334071171703) - rollout_corr/kl:np.float64(0.0006264003223446224) - rollout_corr/k3_kl:np.float64(0.0007624520011759728) - rollout_corr/rollout_ppl:np.float64(1.1288568419404328) - rollout_corr/rollout_log_ppl:np.float64(0.1182069395945291) - rollout_corr/log_ppl_diff:np.float64(0.0006264011171879247) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016409209492849186) - rollout_corr/log_ppl_diff_max:np.float64(0.008434589952230453) - rollout_corr/log_ppl_diff_min:np.float64(-0.006128266453742981) - rollout_corr/ppl_ratio:np.float64(1.0006289719603956) - rollout_corr/chi2_token:np.float64(0.0018175377044826746) - rollout_corr/chi2_seq:np.float64(0.6837903573177755) - actor/pg_loss:np.float64(0.004627972492016852) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010802133826928184) - actor/ppo_kl:np.float64(-1.8316878190205443e-05) - actor/pg_clipfrac_lower:np.float64(1.7377642052451847e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.390625) - self_distillation/token_reweight_w_mean:np.float64(205932.63544678479) - self_distillation/token_reweight_w_std:np.float64(3275553.812427452) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015711192972958) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07417638991319109) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.49010825902223587) - perf/mfu/actor:np.float64(0.04100974753741407) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.18482971191406) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013242431916296482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013242431916296482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:442.6171875 - response_length/max:1530.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:442.6171875 - response_length_non_aborted/max:1530.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014469213783740997 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7575890514999628) - timing_s/agent_loop/generate_sequences/max:np.float64(10.56420186534524) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3792500576455495) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.56420186534524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1530 - timing_s/gen:16.62187496200204 - timing_s/reward:0.0622586477547884 - timing_s/old_log_prob:2.5925257727503777 - timing_s/adv:0.5586317237466574 - timing_s/update_actor:15.944112984463573 - timing_s/step:35.868158884346485 - timing_s/stop_profile:0.00011105090379714966 - timing_per_token_ms/update_actor:0.08815623505470233 - timing_per_token_ms/adv:0.0030887180488253885 - timing_per_token_ms/gen:0.14669380427148565 - perf/total_num_tokens:180862 - perf/time_per_step:35.868158884346485 - perf/throughput:630.3013788049889 (TaskRunner pid=2074004) Training Progress: 18%|█▊ | 18/100 [14:41<59:54, 43.83s/it] (TaskRunner pid=2074004) step:19 - global_seqlen/min:16296 - global_seqlen/max:29019 - global_seqlen/minmax_diff:12723 - global_seqlen/balanced_min:21192 - global_seqlen/balanced_max:21199 - global_seqlen/mean:21195.375 - actor/entropy:0.10421572625637054 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5834612846374512 - training/rollout_probs_diff_mean:0.002985131461173296 - training/rollout_probs_diff_std:0.011980867013335228 - training/rollout_actor_probs_pearson_corr:0.9974684119224548 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.953125 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.953125 - rollout_corr/rollout_is_mean:1.0000929832458496 - rollout_corr/rollout_is_ratio_fraction_high:3.9236074371729046e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010789919906528667 - rollout_corr/rollout_is_max:3.10373592376709 - rollout_corr/rollout_is_min:0.06492556631565094 - rollout_corr/rollout_is_std:0.033075422048568726 - rollout_corr/rollout_is_eff_sample_size:0.9989074497400088 - rollout_corr/rollout_is_seq_mean:1.0000252723693848 - rollout_corr/rollout_is_seq_std:0.0022536139003932476 - rollout_corr/rollout_is_seq_max:1.0099234580993652 - rollout_corr/rollout_is_seq_min:0.9871556758880615 - rollout_corr/rollout_is_seq_max_deviation:0.012844324111938477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1145465495064855) - rollout_corr/training_log_ppl:np.float64(0.10453275293184561) - rollout_corr/kl:np.float64(0.0007910890604869714) - rollout_corr/k3_kl:np.float64(0.0006869278773535825) - rollout_corr/rollout_ppl:np.float64(1.11366256326437) - rollout_corr/rollout_log_ppl:np.float64(0.10374166368637816) - rollout_corr/log_ppl_diff:np.float64(0.0007910892454674467) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018341713002882898) - rollout_corr/log_ppl_diff_max:np.float64(0.020886391401290894) - rollout_corr/log_ppl_diff_min:np.float64(-0.008934102952480316) - rollout_corr/ppl_ratio:np.float64(1.0007949848659337) - rollout_corr/chi2_token:np.float64(0.0011295359581708908) - rollout_corr/chi2_seq:np.float64(0.5068342476151884) - actor/pg_loss:np.float64(0.004318527295254171) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010698111041165248) - actor/ppo_kl:np.float64(0.00017291589237800054) - actor/pg_clipfrac_lower:np.float64(1.0065717106044758e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(318408.1419896148) - self_distillation/token_reweight_w_std:np.float64(4400530.746190617) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.003058377886191) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06988180117332377) - self_distillation/empty_target_batch:np.float64(0.046875) - actor/grad_norm:np.float64(0.41591396927833557) - perf/mfu/actor:np.float64(0.03846046736330761) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.08876419067383) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0076731047593057156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0076731047593057156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.23046875 - response_length/max:1107.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.23046875 - response_length_non_aborted/max:1107.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.310990571975708e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9409778341650963) - timing_s/agent_loop/generate_sequences/max:np.float64(8.202672267332673) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.057653636649775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.202672267332673) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:282 - timing_s/agent_loop/slowest/response_length:1064 - timing_s/gen:14.027731182053685 - timing_s/reward:0.059779345989227295 - timing_s/old_log_prob:2.6110658906400204 - timing_s/adv:0.565576946362853 - timing_s/update_actor:15.789859525859356 - timing_s/step:33.141286708414555 - timing_s/stop_profile:0.00012139417231082916 - timing_per_token_ms/update_actor:0.09312090211814698 - timing_per_token_ms/adv:0.003335497404285446 - timing_per_token_ms/gen:0.13759827343672384 - perf/total_num_tokens:169563 - perf/time_per_step:33.141286708414555 - perf/throughput:639.5459291150124 (TaskRunner pid=2074004) Training Progress: 19%|█▉ | 19/100 [15:14<54:51, 40.64s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) We need to solve for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2074004) step:20 - global_seqlen/min:16557 - global_seqlen/max:25344 - global_seqlen/minmax_diff:8787 - global_seqlen/balanced_min:20112 - global_seqlen/balanced_max:20126 - global_seqlen/mean:20123.125 - actor/entropy:0.1016768291592598 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7593052387237549 - training/rollout_probs_diff_mean:0.0031040834728628397 - training/rollout_probs_diff_std:0.012476732954382896 - training/rollout_actor_probs_pearson_corr:0.9970778822898865 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9998236298561096 - rollout_corr/rollout_is_ratio_fraction_high:1.0585034942778293e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012702042295131832 - rollout_corr/rollout_is_max:3.1519858837127686 - rollout_corr/rollout_is_min:0.22677943110466003 - rollout_corr/rollout_is_std:0.0336710549890995 - rollout_corr/rollout_is_eff_sample_size:0.9988670087351662 - rollout_corr/rollout_is_seq_mean:0.9997695684432983 - rollout_corr/rollout_is_seq_std:0.00212579732760787 - rollout_corr/rollout_is_seq_max:1.0059138536453247 - rollout_corr/rollout_is_seq_min:0.9923778176307678 - rollout_corr/rollout_is_seq_max_deviation:0.007622182369232178 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1031243042089045) - rollout_corr/training_log_ppl:np.float64(0.09565349744298146) - rollout_corr/kl:np.float64(0.0009412075490131677) - rollout_corr/k3_kl:np.float64(0.0007612467196338457) - rollout_corr/rollout_ppl:np.float64(1.1020435886457562) - rollout_corr/rollout_log_ppl:np.float64(0.09471228851907654) - rollout_corr/log_ppl_diff:np.float64(0.0009412089239049237) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019292085235065315) - rollout_corr/log_ppl_diff_max:np.float64(0.011152833700180054) - rollout_corr/log_ppl_diff_min:np.float64(-0.005067870020866394) - rollout_corr/ppl_ratio:np.float64(1.000944792991504) - rollout_corr/chi2_token:np.float64(0.001142266672104597) - rollout_corr/chi2_seq:np.float64(0.5473281410522759) - actor/pg_loss:np.float64(0.0028149266727268696) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000719407254791804) - actor/ppo_kl:np.float64(9.32751950681876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(257358.8287920449) - self_distillation/token_reweight_w_std:np.float64(3547259.863764613) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000823934096843) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04804125138616655) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.4549396261572838) - perf/mfu/actor:np.float64(0.03647682227242746) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.2781219482422) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6) - val-aux/sciknoweval/reward/std@16:np.float64(0.1854085386899461) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6780125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15731648141387625) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5234624999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1564494677502724) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6009) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18569702583933162) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7384999999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11774162185268273) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4601125) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11507313107226662) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.615125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15177662910926504) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7845125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07917893866380923) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.41328750000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07234460911845528) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6239874999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.11172766129349468) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8170249999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04611981709792097) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38732500000000003) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0340747398319288) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6269750000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07494075368453937) - val-aux/sciknoweval/score/mean@16:np.float64(0.6) - val-aux/sciknoweval/score/std@16:np.float64(0.1854085386899461) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6780125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15731648141387625) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5234624999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1564494677502724) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6009) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18569702583933162) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7384999999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11774162185268273) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4601125) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11507313107226662) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.615125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.15177662910926504) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7845125000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07917893866380923) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.41328750000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07234460911845528) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6239874999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.11172766129349468) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8170249999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04611981709792097) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38732500000000003) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0340747398319288) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6269750000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07494075368453937) - val-core/sciknoweval/acc/mean@16:np.float64(0.6) - val-aux/sciknoweval/acc/std@16:np.float64(0.1854085386899461) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6780125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15731648141387625) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5234624999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1564494677502724) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6009) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18569702583933162) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7384999999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11774162185268273) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4601125) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11507313107226662) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.615125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.15177662910926504) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7845125000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07917893866380923) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.41328750000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07234460911845528) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6239874999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.11172766129349468) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8170249999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04611981709792097) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38732500000000003) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0340747398319288) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6269750000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07494075368453937) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.024206145913796356) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995875000000002) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006276698746323264) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9878875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03259636494557759) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9940750000000002) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.023588209728692643) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9772874999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04188475149905875) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9979250000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01420822710607692) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9603625000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0489078128065511) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996875) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0053028861809800276) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9365375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04813723029580622) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023961872793734074 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023961872793734074 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.03515625 - response_length/max:1235.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.03515625 - response_length_non_aborted/max:1235.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.98199325799942e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1489538233727217) - timing_s/agent_loop/generate_sequences/max:np.float64(8.91213802434504) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7332966142930673) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.91213802434504) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:1235 - timing_s/gen:14.779135162010789 - timing_s/reward:0.05472232215106487 - timing_s/old_log_prob:2.653603931888938 - timing_s/adv:0.5555432084947824 - timing_s/update_actor:15.782804861664772 - timing_s/step:33.897329518571496 - timing_s/testing:88.8644837513566 - timing_s/save_checkpoint:6.838149785995483 - timing_s/stop_profile:0.00017092563211917877 - timing_per_token_ms/update_actor:0.09803897792753842 - timing_per_token_ms/adv:0.0034509004472142277 - timing_per_token_ms/gen:0.15643766115197769 - perf/total_num_tokens:160985 - perf/time_per_step:33.897329518571496 - perf/throughput:593.649272252407 (TaskRunner pid=2074004) Training Progress: 20%|██ | 20/100 [17:24<1:29:48, 67.36s/it] (TaskRunner pid=2074004) step:21 - global_seqlen/min:18517 - global_seqlen/max:25867 - global_seqlen/minmax_diff:7350 - global_seqlen/balanced_min:21107 - global_seqlen/balanced_max:21412 - global_seqlen/mean:21162.875 - actor/entropy:0.09419602900743484 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948364734649658 - training/rollout_probs_diff_mean:0.002783035859465599 - training/rollout_probs_diff_std:0.011772817932069302 - training/rollout_actor_probs_pearson_corr:0.9972338080406189 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000512599945068 - rollout_corr/rollout_is_ratio_fraction_high:1.9861960026901215e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012910272926092148 - rollout_corr/rollout_is_max:2.5543158054351807 - rollout_corr/rollout_is_min:0.1329210251569748 - rollout_corr/rollout_is_std:0.03238707408308983 - rollout_corr/rollout_is_eff_sample_size:0.9989522955064933 - rollout_corr/rollout_is_seq_mean:1.0000442266464233 - rollout_corr/rollout_is_seq_std:0.0020267271902412176 - rollout_corr/rollout_is_seq_max:1.0177290439605713 - rollout_corr/rollout_is_seq_min:0.9942728877067566 - rollout_corr/rollout_is_seq_max_deviation:0.01772904396057129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0939804124645889) - rollout_corr/training_log_ppl:np.float64(0.08795880044999649) - rollout_corr/kl:np.float64(0.0004990964827804589) - rollout_corr/k3_kl:np.float64(0.0006357246575703357) - rollout_corr/rollout_ppl:np.float64(1.093429901637137) - rollout_corr/rollout_log_ppl:np.float64(0.08745970321979257) - rollout_corr/log_ppl_diff:np.float64(0.0004990972302039154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015424156808876432) - rollout_corr/log_ppl_diff_max:np.float64(0.005901060998439789) - rollout_corr/log_ppl_diff_min:np.float64(-0.0117645263671875) - rollout_corr/ppl_ratio:np.float64(1.000501276459545) - rollout_corr/chi2_token:np.float64(0.0017355745658278465) - rollout_corr/chi2_seq:np.float64(0.9698769934475422) - actor/pg_loss:np.float64(0.0027807088335976005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005495589271049539) - actor/ppo_kl:np.float64(-4.407575040854539e-05) - actor/pg_clipfrac_lower:np.float64(1.0252624633722007e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(141096.08209011448) - self_distillation/token_reweight_w_std:np.float64(1977869.731875727) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000101735349745) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04118481201294344) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.3234409838914871) - perf/mfu/actor:np.float64(0.03855656297649644) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.51013565063477) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002361090388149023 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002361090388149023 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:393.33984375 - response_length/max:1529.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:393.33984375 - response_length_non_aborted/max:1529.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015509501099586487 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4505607336759567) - timing_s/agent_loop/generate_sequences/max:np.float64(10.29461688362062) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.021393945564341) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.29461688362062) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1529 - timing_s/gen:15.94972218759358 - timing_s/reward:0.0574874933809042 - timing_s/old_log_prob:2.583099478855729 - timing_s/adv:0.6336547415703535 - timing_s/update_actor:15.871118316426873 - timing_s/step:35.191847467795014 - timing_s/stop_profile:0.00011814944446086884 - timing_per_token_ms/update_actor:0.09374386937282195 - timing_per_token_ms/adv:0.003742726009405347 - timing_per_token_ms/gen:0.15839636712442107 - perf/total_num_tokens:169303 - perf/time_per_step:35.191847467795014 - perf/throughput:601.3573177528319 (TaskRunner pid=2074004) Training Progress: 21%|██ | 21/100 [17:59<1:16:00, 57.72s/it] (TaskRunner pid=2074004) step:22 - global_seqlen/min:15901 - global_seqlen/max:25415 - global_seqlen/minmax_diff:9514 - global_seqlen/balanced_min:20195 - global_seqlen/balanced_max:20210 - global_seqlen/mean:20203.0 - actor/entropy:0.10593846440315247 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5313484072685242 - training/rollout_probs_diff_mean:0.003251798916608095 - training/rollout_probs_diff_std:0.012875033542513847 - training/rollout_actor_probs_pearson_corr:0.9971089959144592 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.00010347366333 - rollout_corr/rollout_is_ratio_fraction_high:4.197447924525477e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010493619629414752 - rollout_corr/rollout_is_max:2.9541661739349365 - rollout_corr/rollout_is_min:0.09866257756948471 - rollout_corr/rollout_is_std:0.03540460020303726 - rollout_corr/rollout_is_eff_sample_size:0.9987482024906889 - rollout_corr/rollout_is_seq_mean:1.0001643896102905 - rollout_corr/rollout_is_seq_std:0.0024298520293086767 - rollout_corr/rollout_is_seq_max:1.0121618509292603 - rollout_corr/rollout_is_seq_min:0.9910787343978882 - rollout_corr/rollout_is_seq_max_deviation:0.012161850929260254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1144308936782181) - rollout_corr/training_log_ppl:np.float64(0.10470682490995387) - rollout_corr/kl:np.float64(0.000816442931922623) - rollout_corr/k3_kl:np.float64(0.0008207170265279728) - rollout_corr/rollout_ppl:np.float64(1.1135047189891338) - rollout_corr/rollout_log_ppl:np.float64(0.10389038186986) - rollout_corr/log_ppl_diff:np.float64(0.0008164430400938727) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018727829810814) - rollout_corr/log_ppl_diff_max:np.float64(0.0158127099275589) - rollout_corr/log_ppl_diff_min:np.float64(-0.007389448583126068) - rollout_corr/ppl_ratio:np.float64(1.0008202025201172) - rollout_corr/chi2_token:np.float64(0.0015751214232295752) - rollout_corr/chi2_seq:np.float64(0.42517587705515325) - actor/pg_loss:np.float64(0.0036053532967343926) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010071134870486276) - actor/ppo_kl:np.float64(0.00023022429938990285) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(218233.54828879703) - self_distillation/token_reweight_w_std:np.float64(3210707.643729917) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001665785908699) - self_distillation/token_reweight_w_clip_frac:np.float64(0.058951567800249904) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.4596095457673073) - perf/mfu/actor:np.float64(0.036942325232477896) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(143.46227645874023) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006771008018404245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006771008018404245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:372.25 - response_length/max:1292.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:372.25 - response_length_non_aborted/max:1292.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001899413764476776 - timing_s/agent_loop/generate_sequences/min:np.float64(1.149434920400381) - timing_s/agent_loop/generate_sequences/max:np.float64(8.853524098172784) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8520722229222883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.853524098172784) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1292 - timing_s/gen:14.650895247235894 - timing_s/reward:0.04885207116603851 - timing_s/old_log_prob:2.5285760313272476 - timing_s/adv:0.5984222404658794 - timing_s/update_actor:15.83981061168015 - timing_s/step:33.70206049270928 - timing_s/stop_profile:0.00015066377818584442 - timing_per_token_ms/update_actor:0.09800407496213527 - timing_per_token_ms/adv:0.0037025580388177462 - timing_per_token_ms/gen:0.15374092561320407 - perf/total_num_tokens:161624 - perf/time_per_step:33.70206049270928 - perf/throughput:599.4588967155432 (TaskRunner pid=2074004) Training Progress: 22%|██▏ | 22/100 [18:33<1:05:41, 50.53s/it] (TaskRunner pid=2074004) step:23 - global_seqlen/min:16336 - global_seqlen/max:30208 - global_seqlen/minmax_diff:13872 - global_seqlen/balanced_min:23332 - global_seqlen/balanced_max:23529 - global_seqlen/mean:23365.625 - actor/entropy:0.13051186501979828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46211716532707214 - training/rollout_probs_diff_mean:0.003518641460686922 - training/rollout_probs_diff_std:0.012740753591060638 - training/rollout_actor_probs_pearson_corr:0.9975020289421082 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.44140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999224543571472 - rollout_corr/rollout_is_ratio_fraction_high:2.5563025701558217e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013633613707497716 - rollout_corr/rollout_is_max:3.358520746231079 - rollout_corr/rollout_is_min:0.2698378264904022 - rollout_corr/rollout_is_std:0.03545843064785004 - rollout_corr/rollout_is_eff_sample_size:0.9987440406221719 - rollout_corr/rollout_is_seq_mean:0.9998674988746643 - rollout_corr/rollout_is_seq_std:0.002211816841736436 - rollout_corr/rollout_is_seq_max:1.0098694562911987 - rollout_corr/rollout_is_seq_min:0.9933016896247864 - rollout_corr/rollout_is_seq_max_deviation:0.00986945629119873 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1375553528778255) - rollout_corr/training_log_ppl:np.float64(0.12625176343863131) - rollout_corr/kl:np.float64(0.0009684273467929216) - rollout_corr/k3_kl:np.float64(0.0008128368172322098) - rollout_corr/rollout_ppl:np.float64(1.1364499349147081) - rollout_corr/rollout_log_ppl:np.float64(0.12528333482623566) - rollout_corr/log_ppl_diff:np.float64(0.0009684286123956554) - rollout_corr/log_ppl_abs_diff:np.float64(0.001832397589168977) - rollout_corr/log_ppl_diff_max:np.float64(0.011887356638908386) - rollout_corr/log_ppl_diff_min:np.float64(-0.008548229932785034) - rollout_corr/ppl_ratio:np.float64(1.000971759436652) - rollout_corr/chi2_token:np.float64(0.0012992981355637312) - rollout_corr/chi2_seq:np.float64(0.5191449248231947) - actor/pg_loss:np.float64(0.005724000162445009) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016155378229996131) - actor/ppo_kl:np.float64(0.00011277476574633738) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44140625) - self_distillation/token_reweight_w_mean:np.float64(304747.9913651231) - self_distillation/token_reweight_w_std:np.float64(4644227.323464029) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019670443143696) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08922295937372837) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.5448018610477448) - perf/mfu/actor:np.float64(0.04195048253218733) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.12269973754883) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004490571562200785 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004490571562200785 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.42578125 - response_length/max:1832.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:458.42578125 - response_length_non_aborted/max:1832.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005329102277755737 - timing_s/agent_loop/generate_sequences/min:np.float64(1.116574076935649) - timing_s/agent_loop/generate_sequences/max:np.float64(11.822160206735134) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7243306759992265) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.822160206735134) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1832 - timing_s/gen:18.11809141561389 - timing_s/reward:0.05629480630159378 - timing_s/old_log_prob:2.732456060126424 - timing_s/adv:0.6987331956624985 - timing_s/update_actor:15.840048659592867 - timing_s/step:37.55766446888447 - timing_s/stop_profile:6.579048931598663e-05 - timing_per_token_ms/update_actor:0.08474012924752101 - timing_per_token_ms/adv:0.00373804036732646 - timing_per_token_ms/gen:0.154384411800011 - perf/total_num_tokens:186925 - perf/time_per_step:37.55766446888447 - perf/throughput:622.1266772154537 (TaskRunner pid=2074004) Training Progress: 23%|██▎ | 23/100 [19:11<59:59, 46.75s/it] (TaskRunner pid=2074004) step:24 - global_seqlen/min:17135 - global_seqlen/max:25434 - global_seqlen/minmax_diff:8299 - global_seqlen/balanced_min:20734 - global_seqlen/balanced_max:20748 - global_seqlen/mean:20742.75 - actor/entropy:0.10690584033727646 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519520163536072 - training/rollout_probs_diff_mean:0.0030989660881459713 - training/rollout_probs_diff_std:0.012050447054207325 - training/rollout_actor_probs_pearson_corr:0.9973185062408447 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0002549886703491 - rollout_corr/rollout_is_ratio_fraction_high:2.0413579477462918e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.165431790985167e-05 - rollout_corr/rollout_is_max:2.1403613090515137 - rollout_corr/rollout_is_min:0.21778209507465363 - rollout_corr/rollout_is_std:0.033476658165454865 - rollout_corr/rollout_is_eff_sample_size:0.9988811627143783 - rollout_corr/rollout_is_seq_mean:1.0003387928009033 - rollout_corr/rollout_is_seq_std:0.0020440113730728626 - rollout_corr/rollout_is_seq_max:1.0078518390655518 - rollout_corr/rollout_is_seq_min:0.9929835200309753 - rollout_corr/rollout_is_seq_max_deviation:0.007851839065551758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.109124246519059) - rollout_corr/training_log_ppl:np.float64(0.10045992501545697) - rollout_corr/kl:np.float64(0.00014452066502457228) - rollout_corr/k3_kl:np.float64(0.0006306728797227379) - rollout_corr/rollout_ppl:np.float64(1.1089712120592594) - rollout_corr/rollout_log_ppl:np.float64(0.10031540389900329) - rollout_corr/log_ppl_diff:np.float64(0.0001445211164536886) - rollout_corr/log_ppl_abs_diff:np.float64(0.001574805770360399) - rollout_corr/log_ppl_diff_max:np.float64(0.011304967105388641) - rollout_corr/log_ppl_diff_min:np.float64(-0.011325351893901825) - rollout_corr/ppl_ratio:np.float64(1.000147320330143) - rollout_corr/chi2_token:np.float64(0.0022659352980554104) - rollout_corr/chi2_seq:np.float64(0.813906122231856) - actor/pg_loss:np.float64(0.002153943874873221) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006025306415722298) - actor/ppo_kl:np.float64(-0.00010819188584676454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(135130.44865711615) - self_distillation/token_reweight_w_std:np.float64(2006584.000496325) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000246322248131) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04379224638978485) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.33335286378860474) - perf/mfu/actor:np.float64(0.037579949707391716) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.17943572998047) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005197807680815458 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005197807680815458 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:382.7109375 - response_length/max:1139.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.7109375 - response_length_non_aborted/max:1139.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.317960262298584e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8356650900095701) - timing_s/agent_loop/generate_sequences/max:np.float64(8.357717419043183) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8592212262228713) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.357717419043183) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1139 - timing_s/gen:14.09934650734067 - timing_s/reward:0.05855764448642731 - timing_s/old_log_prob:2.4800929073244333 - timing_s/adv:0.5536916963756084 - timing_s/update_actor:15.75769667699933 - timing_s/step:33.040523858740926 - timing_s/stop_profile:0.0001149345189332962 - timing_per_token_ms/update_actor:0.0949590620638496 - timing_per_token_ms/adv:0.003336657967094578 - timing_per_token_ms/gen:0.1439090626833718 - perf/total_num_tokens:165942 - perf/time_per_step:33.040523858740926 - perf/throughput:627.7972494831516 (TaskRunner pid=2074004) Training Progress: 24%|██▍ | 24/100 [19:44<54:01, 42.65s/it] (TaskRunner pid=2074004) step:25 - global_seqlen/min:15092 - global_seqlen/max:31297 - global_seqlen/minmax_diff:16205 - global_seqlen/balanced_min:21652 - global_seqlen/balanced_max:28332 - global_seqlen/mean:22490.0 - actor/entropy:0.10956007242202759 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6190921068191528 - training/rollout_probs_diff_mean:0.0033610411919653416 - training/rollout_probs_diff_std:0.013345292769372463 - training/rollout_actor_probs_pearson_corr:0.9969186782836914 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.000109314918518 - rollout_corr/rollout_is_ratio_fraction_high:3.6875877412967384e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.531381248147227e-05 - rollout_corr/rollout_is_max:2.4000637531280518 - rollout_corr/rollout_is_min:0.031893547624349594 - rollout_corr/rollout_is_std:0.03650045394897461 - rollout_corr/rollout_is_eff_sample_size:0.9986697273810201 - rollout_corr/rollout_is_seq_mean:1.0001757144927979 - rollout_corr/rollout_is_seq_std:0.002466355450451374 - rollout_corr/rollout_is_seq_max:1.009514331817627 - rollout_corr/rollout_is_seq_min:0.9879665374755859 - rollout_corr/rollout_is_seq_max_deviation:0.012033462524414062 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1296572694554925) - rollout_corr/training_log_ppl:np.float64(0.11874415549391415) - rollout_corr/kl:np.float64(0.0007365228179097016) - rollout_corr/k3_kl:np.float64(0.000829679726990662) - rollout_corr/rollout_ppl:np.float64(1.1288006519898772) - rollout_corr/rollout_log_ppl:np.float64(0.11800763171049766) - rollout_corr/log_ppl_diff:np.float64(0.0007365237834164873) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019228525488870218) - rollout_corr/log_ppl_diff_max:np.float64(0.01413746178150177) - rollout_corr/log_ppl_diff_min:np.float64(-0.0073609501123428345) - rollout_corr/ppl_ratio:np.float64(1.0007400074973702) - rollout_corr/chi2_token:np.float64(0.001852479763329029) - rollout_corr/chi2_seq:np.float64(0.793041022028774) - actor/pg_loss:np.float64(0.0035626995377242565) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000992866120668623) - actor/ppo_kl:np.float64(0.00013374969253865387) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(210967.00421022926) - self_distillation/token_reweight_w_std:np.float64(2976356.6710397173) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001523660030216) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0665886076312745) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4494774639606476) - perf/mfu/actor:np.float64(0.03848721918082872) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.24299240112305) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.052680645138025284 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.052680645138025284 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.71875 - response_length/max:8192.0 - response_length/min:75.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:423.71875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015040487051010132 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8068552948534489) - timing_s/agent_loop/generate_sequences/max:np.float64(50.265202172100544) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.223772806173656) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.265202172100544) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.000157160684466 - timing_s/reward:0.062263449653983116 - timing_s/old_log_prob:2.6894225608557463 - timing_s/adv:0.575139531865716 - timing_s/update_actor:16.96509013324976 - timing_s/step:76.38079810328782 - timing_s/stop_profile:0.00012892484664916992 - timing_per_token_ms/update_actor:0.09429240847737749 - timing_per_token_ms/adv:0.003196640350520876 - timing_per_token_ms/gen:0.5162637100881745 - perf/total_num_tokens:179920 - perf/time_per_step:76.38079810328782 - perf/throughput:294.44573189176873 (TaskRunner pid=2074004) Training Progress: 25%|██▌ | 25/100 [21:00<1:05:59, 52.79s/it] (TaskRunner pid=2074004) step:26 - global_seqlen/min:17856 - global_seqlen/max:22321 - global_seqlen/minmax_diff:4465 - global_seqlen/balanced_min:19619 - global_seqlen/balanced_max:19629 - global_seqlen/mean:19624.625 - actor/entropy:0.10051669180393219 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30771172046661377 - training/rollout_probs_diff_mean:0.0031875635031610727 - training/rollout_probs_diff_std:0.012896761298179626 - training/rollout_actor_probs_pearson_corr:0.9968799948692322 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:1.0000146627426147 - rollout_corr/rollout_is_ratio_fraction_high:2.208553814853076e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.93849171209149e-05 - rollout_corr/rollout_is_max:2.9205586910247803 - rollout_corr/rollout_is_min:0.3763677477836609 - rollout_corr/rollout_is_std:0.03457672521471977 - rollout_corr/rollout_is_eff_sample_size:0.9988058776680073 - rollout_corr/rollout_is_seq_mean:1.0000602006912231 - rollout_corr/rollout_is_seq_std:0.0020706949289888144 - rollout_corr/rollout_is_seq_max:1.0055534839630127 - rollout_corr/rollout_is_seq_min:0.991748571395874 - rollout_corr/rollout_is_seq_max_deviation:0.008251428604125977 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0977428681217134) - rollout_corr/training_log_ppl:np.float64(0.09058145447852439) - rollout_corr/kl:np.float64(0.0006791098424780984) - rollout_corr/k3_kl:np.float64(0.0007086409321068743) - rollout_corr/rollout_ppl:np.float64(1.0969782588072121) - rollout_corr/rollout_log_ppl:np.float64(0.0899023435304116) - rollout_corr/log_ppl_diff:np.float64(0.0006791109481127933) - rollout_corr/log_ppl_abs_diff:np.float64(0.001920343165693339) - rollout_corr/log_ppl_diff_max:np.float64(0.024022668600082397) - rollout_corr/log_ppl_diff_min:np.float64(-0.005489669740200043) - rollout_corr/ppl_ratio:np.float64(1.0006832026410848) - rollout_corr/chi2_token:np.float64(0.001474759541451931) - rollout_corr/chi2_seq:np.float64(0.6005796948447824) - actor/pg_loss:np.float64(0.003621779615059495) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007727315569354687) - actor/ppo_kl:np.float64(0.000100072484177538) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(201738.92587617878) - self_distillation/token_reweight_w_std:np.float64(2411880.3968902696) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00004991912283) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04766093760554213) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.45207298547029495) - perf/mfu/actor:np.float64(0.03555715209782605) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.1187286376953) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-5.383349707699381e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-5.383349707699381e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:353.73828125 - response_length/max:1273.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:353.73828125 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011607818305492401 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5807047858834267) - timing_s/agent_loop/generate_sequences/max:np.float64(8.703462390229106) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3116236440982902) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.703462390229106) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1273 - timing_s/gen:14.44053604081273 - timing_s/reward:0.058792874217033386 - timing_s/old_log_prob:2.4093342386186123 - timing_s/adv:0.5303562916815281 - timing_s/update_actor:15.99789859354496 - timing_s/step:33.523452650755644 - timing_s/stop_profile:2.9999762773513794e-05 - timing_per_token_ms/update_actor:0.10189939039309642 - timing_per_token_ms/adv:0.0033781301023683773 - timing_per_token_ms/gen:0.15946349857893627 - perf/total_num_tokens:156997 - perf/time_per_step:33.523452650755644 - perf/throughput:585.3998752589001 (TaskRunner pid=2074004) Training Progress: 26%|██▌ | 26/100 [21:34<57:59, 47.02s/it] (TaskRunner pid=2074004) step:27 - global_seqlen/min:19536 - global_seqlen/max:27649 - global_seqlen/minmax_diff:8113 - global_seqlen/balanced_min:24156 - global_seqlen/balanced_max:24592 - global_seqlen/mean:24227.0 - actor/entropy:0.08968006819486618 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35172876715660095 - training/rollout_probs_diff_mean:0.0026838507037609816 - training/rollout_probs_diff_std:0.01177211757749319 - training/rollout_actor_probs_pearson_corr:0.9971114993095398 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96484375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96484375 - rollout_corr/rollout_is_mean:1.000118613243103 - rollout_corr/rollout_is_ratio_fraction_high:3.249707515351474e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.68698815186508e-05 - rollout_corr/rollout_is_max:9.403421401977539 - rollout_corr/rollout_is_min:0.415858656167984 - rollout_corr/rollout_is_std:0.03169693797826767 - rollout_corr/rollout_is_eff_sample_size:0.9989963125083661 - rollout_corr/rollout_is_seq_mean:1.0000813007354736 - rollout_corr/rollout_is_seq_std:0.0021606753580272198 - rollout_corr/rollout_is_seq_max:1.014573335647583 - rollout_corr/rollout_is_seq_min:0.991974413394928 - rollout_corr/rollout_is_seq_max_deviation:0.014573335647583008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0907162227667868) - rollout_corr/training_log_ppl:np.float64(0.08525840124275419) - rollout_corr/kl:np.float64(0.0006036972056477907) - rollout_corr/k3_kl:np.float64(0.0006888124125836725) - rollout_corr/rollout_ppl:np.float64(1.0900488472543657) - rollout_corr/rollout_log_ppl:np.float64(0.08465470290684607) - rollout_corr/log_ppl_diff:np.float64(0.0006036983359081205) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015368564454547595) - rollout_corr/log_ppl_diff_max:np.float64(0.013522587716579437) - rollout_corr/log_ppl_diff_min:np.float64(-0.0061819590628147125) - rollout_corr/ppl_ratio:np.float64(1.0006064060144126) - rollout_corr/chi2_token:np.float64(0.0026917706709355116) - rollout_corr/chi2_seq:np.float64(5.136045180493966) - actor/pg_loss:np.float64(0.0036772675812244415) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007991504742221878) - actor/ppo_kl:np.float64(0.00012214449816383421) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(157488.76363183185) - self_distillation/token_reweight_w_std:np.float64(2501365.0777522195) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000293314224109) - self_distillation/token_reweight_w_clip_frac:np.float64(0.055393004906363785) - self_distillation/empty_target_batch:np.float64(0.03515625) - actor/grad_norm:np.float64(0.4007357805967331) - perf/mfu/actor:np.float64(0.043591956964482634) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.22241592407227) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009190579294227064 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009190579294227064 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.8125 - response_length/max:2125.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.8125 - response_length_non_aborted/max:2125.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.861744284629822e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1442433521151543) - timing_s/agent_loop/generate_sequences/max:np.float64(13.537522902712226) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.885370632960985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.537522902712226) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:2125 - timing_s/gen:19.567774968221784 - timing_s/reward:0.06262494809925556 - timing_s/old_log_prob:2.4061235673725605 - timing_s/adv:0.5981502141803503 - timing_s/update_actor:16.023667991161346 - timing_s/step:38.744677659124136 - timing_s/stop_profile:0.00011847354471683502 - timing_per_token_ms/update_actor:0.08267463981901053 - timing_per_token_ms/adv:0.0030861756211063604 - timing_per_token_ms/gen:0.15897386396904478 - perf/total_num_tokens:193816 - perf/time_per_step:38.744677659124136 - perf/throughput:625.2987884722971 (TaskRunner pid=2074004) Training Progress: 27%|██▋ | 27/100 [22:13<54:12, 44.55s/it] (TaskRunner pid=2074004) step:28 - global_seqlen/min:15689 - global_seqlen/max:28335 - global_seqlen/minmax_diff:12646 - global_seqlen/balanced_min:22049 - global_seqlen/balanced_max:22067 - global_seqlen/mean:22059.625 - actor/entropy:0.11999335139989853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3807188868522644 - training/rollout_probs_diff_mean:0.00325206876732409 - training/rollout_probs_diff_std:0.012448713183403015 - training/rollout_actor_probs_pearson_corr:0.9974731802940369 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001393556594849 - rollout_corr/rollout_is_ratio_fraction_high:5.649025843013078e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.590530392713845e-05 - rollout_corr/rollout_is_max:3.7063121795654297 - rollout_corr/rollout_is_min:0.3865959644317627 - rollout_corr/rollout_is_std:0.03557758033275604 - rollout_corr/rollout_is_eff_sample_size:0.998736192621627 - rollout_corr/rollout_is_seq_mean:1.0002413988113403 - rollout_corr/rollout_is_seq_std:0.002391271060332656 - rollout_corr/rollout_is_seq_max:1.0125643014907837 - rollout_corr/rollout_is_seq_min:0.9939977526664734 - rollout_corr/rollout_is_seq_max_deviation:0.012564301490783691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1245770892128348) - rollout_corr/training_log_ppl:np.float64(0.1141112526966026) - rollout_corr/kl:np.float64(0.0005681062639624201) - rollout_corr/k3_kl:np.float64(0.0007897823702087692) - rollout_corr/rollout_ppl:np.float64(1.1239187237806618) - rollout_corr/rollout_log_ppl:np.float64(0.11354314523669018) - rollout_corr/log_ppl_diff:np.float64(0.0005681074599124258) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016994898305711104) - rollout_corr/log_ppl_diff_max:np.float64(0.01184430718421936) - rollout_corr/log_ppl_diff_min:np.float64(-0.006551269441843033) - rollout_corr/ppl_ratio:np.float64(1.0005709801334888) - rollout_corr/chi2_token:np.float64(0.002181260846555233) - rollout_corr/chi2_seq:np.float64(0.662524172803387) - actor/pg_loss:np.float64(0.002658500336110592) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007556931734598038) - actor/ppo_kl:np.float64(8.9224213090211e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(108684.6212300947) - self_distillation/token_reweight_w_std:np.float64(1726174.4731562997) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000196180306375) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04746657198120374) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.3256382420659065) - perf/mfu/actor:np.float64(0.039955600042607634) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.3211898803711) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012464340776205063 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012464340776205063 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:414.89453125 - response_length/max:1399.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.89453125 - response_length_non_aborted/max:1399.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016648322343826294 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5349066201597452) - timing_s/agent_loop/generate_sequences/max:np.float64(9.329830139875412) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.712864897781401) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.329830139875412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1399 - timing_s/gen:15.155654072761536 - timing_s/reward:0.061105452477931976 - timing_s/old_log_prob:2.4498131908476353 - timing_s/adv:0.594944417476654 - timing_s/update_actor:15.982728907838464 - timing_s/step:34.33417861536145 - timing_s/stop_profile:0.00012039020657539368 - timing_per_token_ms/update_actor:0.09056550659767824 - timing_per_token_ms/adv:0.003371229211039705 - timing_per_token_ms/gen:0.14269114018775042 - perf/total_num_tokens:176477 - perf/time_per_step:34.33417861536145 - perf/throughput:642.4975313121457 (TaskRunner pid=2074004) Training Progress: 28%|██▊ | 28/100 [22:47<49:47, 41.50s/it] (TaskRunner pid=2074004) step:29 - global_seqlen/min:18998 - global_seqlen/max:31893 - global_seqlen/minmax_diff:12895 - global_seqlen/balanced_min:22424 - global_seqlen/balanced_max:28754 - global_seqlen/mean:23272.0 - actor/entropy:0.09004729986190796 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6166199445724487 - training/rollout_probs_diff_mean:0.0027836794033646584 - training/rollout_probs_diff_std:0.011670615524053574 - training/rollout_actor_probs_pearson_corr:0.9972294569015503 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999050498008728 - rollout_corr/rollout_is_ratio_fraction_high:1.6847496226546355e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.423747931374237e-05 - rollout_corr/rollout_is_max:2.463893175125122 - rollout_corr/rollout_is_min:0.2046324759721756 - rollout_corr/rollout_is_std:0.03296531364321709 - rollout_corr/rollout_is_eff_sample_size:0.9989144677772216 - rollout_corr/rollout_is_seq_mean:0.9998564124107361 - rollout_corr/rollout_is_seq_std:0.0022072712890803814 - rollout_corr/rollout_is_seq_max:1.006649136543274 - rollout_corr/rollout_is_seq_min:0.99166339635849 - rollout_corr/rollout_is_seq_max_deviation:0.00833660364151001 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1013299119658768) - rollout_corr/training_log_ppl:np.float64(0.09491147637072572) - rollout_corr/kl:np.float64(0.0012193648983345184) - rollout_corr/k3_kl:np.float64(0.0009733567474512483) - rollout_corr/rollout_ppl:np.float64(1.099963684566319) - rollout_corr/rollout_log_ppl:np.float64(0.09369211093780905) - rollout_corr/log_ppl_diff:np.float64(0.0012193654329166748) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020960377587471157) - rollout_corr/log_ppl_diff_max:np.float64(0.05848722159862518) - rollout_corr/log_ppl_diff_min:np.float64(-0.007673636078834534) - rollout_corr/ppl_ratio:np.float64(1.001229693647474) - rollout_corr/chi2_token:np.float64(0.0014654879923909903) - rollout_corr/chi2_seq:np.float64(0.6356173786334693) - actor/pg_loss:np.float64(0.003317820141091943) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006650509092196444) - actor/ppo_kl:np.float64(0.0003957474698959018) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(62520.58305266849) - self_distillation/token_reweight_w_std:np.float64(1281642.6764327008) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001606633886695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03943118227471132) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.3942992687225342) - perf/mfu/actor:np.float64(0.039518949200975054) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.29967880249023) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0459052175283432 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0459052175283432 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:463.71875 - response_length/max:8192.0 - response_length/min:80.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:463.71875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.063258767127991e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0605478826910257) - timing_s/agent_loop/generate_sequences/max:np.float64(49.57040260732174) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.396626845227729) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.57040260732174) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:375 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.50206731632352 - timing_s/reward:0.05697246454656124 - timing_s/old_log_prob:2.633130395784974 - timing_s/adv:0.6030823420733213 - timing_s/update_actor:17.1904456615448 - timing_s/step:76.02233720384538 - timing_s/stop_profile:0.00010943599045276642 - timing_per_token_ms/update_actor:0.09233438070183482 - timing_per_token_ms/adv:0.003239313026777465 - timing_per_token_ms/gen:0.46753544137343755 - perf/total_num_tokens:186176 - perf/time_per_step:76.02233720384538 - perf/throughput:306.1205542470858 (TaskRunner pid=2074004) Training Progress: 29%|██▉ | 29/100 [24:03<1:01:22, 51.87s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) We are given: (TaskRunner pid=2074004) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2074004) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2074004) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2074004) (TaskRunner pid=2074004) We need to solve for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest option is **B: 56 pC**. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2074004) step:30 - global_seqlen/min:16055 - global_seqlen/max:23382 - global_seqlen/minmax_diff:7327 - global_seqlen/balanced_min:20479 - global_seqlen/balanced_max:20784 - global_seqlen/mean:20529.125 - actor/entropy:0.11640558391809464 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8577249050140381 - training/rollout_probs_diff_mean:0.0033130382653325796 - training/rollout_probs_diff_std:0.013030188158154488 - training/rollout_actor_probs_pearson_corr:0.9971984624862671 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0003026723861694 - rollout_corr/rollout_is_ratio_fraction_high:6.133902934379876e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010223171557299793 - rollout_corr/rollout_is_max:9.797200202941895 - rollout_corr/rollout_is_min:0.02608746662735939 - rollout_corr/rollout_is_std:0.03469718620181084 - rollout_corr/rollout_is_eff_sample_size:0.9987981476185248 - rollout_corr/rollout_is_seq_mean:1.0002856254577637 - rollout_corr/rollout_is_seq_std:0.0022839626763015985 - rollout_corr/rollout_is_seq_max:1.0106697082519531 - rollout_corr/rollout_is_seq_min:0.992483377456665 - rollout_corr/rollout_is_seq_max_deviation:0.010669708251953125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.116787108592689) - rollout_corr/training_log_ppl:np.float64(0.10795232591772219) - rollout_corr/kl:np.float64(0.000616683089219805) - rollout_corr/k3_kl:np.float64(0.0007727639311809753) - rollout_corr/rollout_ppl:np.float64(1.1160784345120192) - rollout_corr/rollout_log_ppl:np.float64(0.10733564237307291) - rollout_corr/log_ppl_diff:np.float64(0.0006166835446492769) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018158721577492543) - rollout_corr/log_ppl_diff_max:np.float64(0.01455879956483841) - rollout_corr/log_ppl_diff_min:np.float64(-0.007083818316459656) - rollout_corr/ppl_ratio:np.float64(1.000619996106252) - rollout_corr/chi2_token:np.float64(0.0019336091354489326) - rollout_corr/chi2_seq:np.float64(1.0694307961966842) - actor/pg_loss:np.float64(0.002601322717964649) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044001030278195685) - actor/ppo_kl:np.float64(0.00017719261101882544) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(32765.776810475392) - self_distillation/token_reweight_w_std:np.float64(708157.8602350631) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9986124283168465) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04477454177686013) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.21477733738720417) - perf/mfu/actor:np.float64(0.03686629124421728) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.2654571533203) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.18777951588879277) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.699725) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14925101134863997) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5497249999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1686963837105089) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6241125000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1881880107415588) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7547625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10614923952325535) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.48316250000000005) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1385217733706531) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6387875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14554909217773004) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7954375) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07429913902055749) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4257375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10329608095410872) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.646375) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10428943394865786) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8249500000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04785468179260445) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38275000000000003) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06501528454070343) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6504875) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07057889100574483) - val-aux/sciknoweval/score/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/score/std@16:np.float64(0.18777951588879277) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.699725) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14925101134863997) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5497249999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1686963837105089) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6241125000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1881880107415588) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7547625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10614923952325535) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.48316250000000005) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1385217733706531) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6387875) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14554909217773004) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7954375) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07429913902055749) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4257375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10329608095410872) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.646375) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10428943394865786) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8249500000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04785468179260445) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38275000000000003) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06501528454070343) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6504875) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07057889100574483) - val-core/sciknoweval/acc/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/acc/std@16:np.float64(0.18777951588879277) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.699725) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14925101134863997) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5497249999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1686963837105089) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6241125000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1881880107415588) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7547625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10614923952325535) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.48316250000000005) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1385217733706531) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6387875) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14554909217773004) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7954375) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07429913902055749) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4257375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10329608095410872) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.646375) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10428943394865786) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8249500000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04785468179260445) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38275000000000003) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06501528454070343) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6504875) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07057889100574483) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99296875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.02339681404319957) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9992374999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.007777136517816352) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9865124999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03086320051797221) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9931000000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.022986682202438165) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.97585) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.038177039681755176) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9976375000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.013409115755367873) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9586375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04250866307068464) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9995375) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005306554090650112) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.93815) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03807095702890277) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.000683625445693766) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00048432277981191874 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00048432277981191874 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:382.09765625 - response_length/max:1653.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:382.09765625 - response_length_non_aborted/max:1653.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011514127254486084 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9901416879147291) - timing_s/agent_loop/generate_sequences/max:np.float64(10.789292139932513) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9146284311136696) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.789292139932513) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1653 - timing_s/gen:16.85866062529385 - timing_s/reward:0.059875741600990295 - timing_s/old_log_prob:2.412396924570203 - timing_s/adv:0.6014310400933027 - timing_s/update_actor:15.988812597468495 - timing_s/step:36.00774179585278 - timing_s/testing:91.78712977468967 - timing_s/save_checkpoint:6.680276766419411 - timing_s/stop_profile:0.00010959990322589874 - timing_per_token_ms/update_actor:0.09735444519352685 - timing_per_token_ms/adv:0.003662059635355274 - timing_per_token_ms/gen:0.17234898458646095 - perf/total_num_tokens:164233 - perf/time_per_step:36.00774179585278 - perf/throughput:570.1308656452446 (TaskRunner pid=2074004) Training Progress: 30%|███ | 30/100 [26:18<1:29:26, 76.67s/it] (TaskRunner pid=2074004) step:31 - global_seqlen/min:18767 - global_seqlen/max:37361 - global_seqlen/minmax_diff:18594 - global_seqlen/balanced_min:23793 - global_seqlen/balanced_max:23854 - global_seqlen/mean:23813.5 - actor/entropy:0.09906209260225296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35506775975227356 - training/rollout_probs_diff_mean:0.0027171920519322157 - training/rollout_probs_diff_std:0.01121894083917141 - training/rollout_actor_probs_pearson_corr:0.997601330280304 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0000742673873901 - rollout_corr/rollout_is_ratio_fraction_high:2.5133205781457946e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.188867751508951e-05 - rollout_corr/rollout_is_max:2.3356175422668457 - rollout_corr/rollout_is_min:0.32783398032188416 - rollout_corr/rollout_is_std:0.030455991625785828 - rollout_corr/rollout_is_eff_sample_size:0.9990735300790685 - rollout_corr/rollout_is_seq_mean:1.0001554489135742 - rollout_corr/rollout_is_seq_std:0.0015059742145240307 - rollout_corr/rollout_is_seq_max:1.0055322647094727 - rollout_corr/rollout_is_seq_min:0.9959016442298889 - rollout_corr/rollout_is_seq_max_deviation:0.005532264709472656 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.09525554953143) - rollout_corr/training_log_ppl:np.float64(0.08918771221215138) - rollout_corr/kl:np.float64(0.0004833923444760302) - rollout_corr/k3_kl:np.float64(0.0005884723202669306) - rollout_corr/rollout_ppl:np.float64(1.094714654609561) - rollout_corr/rollout_log_ppl:np.float64(0.08870431894683861) - rollout_corr/log_ppl_diff:np.float64(0.0004833932653127704) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014353545957419556) - rollout_corr/log_ppl_diff_max:np.float64(0.007750939577817917) - rollout_corr/log_ppl_diff_min:np.float64(-0.0094594806432724) - rollout_corr/ppl_ratio:np.float64(1.0004852761048824) - rollout_corr/chi2_token:np.float64(0.0013451201375573874) - rollout_corr/chi2_seq:np.float64(0.6003347004298121) - actor/pg_loss:np.float64(0.0021815065992996097) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000496987612450539) - actor/ppo_kl:np.float64(0.0001865752313179314) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(91561.24236924388) - self_distillation/token_reweight_w_std:np.float64(1730255.973181627) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005300368648022) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04233101321733557) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.2640515901148319) - perf/mfu/actor:np.float64(0.04318160738783638) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.296875) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013344685547053814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013344685547053814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:466.265625 - response_length/max:1752.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:466.265625 - response_length_non_aborted/max:1752.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.933486580848694e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.500905817374587) - timing_s/agent_loop/generate_sequences/max:np.float64(11.73058369755745) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.724863474373706) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.73058369755745) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1752 - timing_s/gen:17.499854369089007 - timing_s/reward:0.06082160025835037 - timing_s/old_log_prob:2.452167185023427 - timing_s/adv:0.642735943198204 - timing_s/update_actor:15.802205985412002 - timing_s/step:36.550422716885805 - timing_s/stop_profile:3.2784417271614075e-05 - timing_per_token_ms/update_actor:0.08294772915264452 - timing_per_token_ms/adv:0.003373800277144288 - timing_per_token_ms/gen:0.14660914822801688 - perf/total_num_tokens:190508 - perf/time_per_step:36.550422716885805 - perf/throughput:651.5246125730437 (TaskRunner pid=2074004) Training Progress: 31%|███ | 31/100 [26:54<1:14:20, 64.64s/it] (TaskRunner pid=2074004) step:32 - global_seqlen/min:14634 - global_seqlen/max:23574 - global_seqlen/minmax_diff:8940 - global_seqlen/balanced_min:18859 - global_seqlen/balanced_max:18877 - global_seqlen/mean:18872.25 - actor/entropy:0.10486175119876862 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43836408853530884 - training/rollout_probs_diff_mean:0.003058877307921648 - training/rollout_probs_diff_std:0.012399275787174702 - training/rollout_actor_probs_pearson_corr:0.9971876740455627 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999085068702698 - rollout_corr/rollout_is_ratio_fraction_high:2.3329055693466216e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001399743341607973 - rollout_corr/rollout_is_max:2.0753023624420166 - rollout_corr/rollout_is_min:0.2297990471124649 - rollout_corr/rollout_is_std:0.032623618841171265 - rollout_corr/rollout_is_eff_sample_size:0.998936593082401 - rollout_corr/rollout_is_seq_mean:0.9999740123748779 - rollout_corr/rollout_is_seq_std:0.0029045569244772196 - rollout_corr/rollout_is_seq_max:1.014037847518921 - rollout_corr/rollout_is_seq_min:0.9840235710144043 - rollout_corr/rollout_is_seq_max_deviation:0.015976428985595703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0999524341896176) - rollout_corr/training_log_ppl:np.float64(0.09279653291014256) - rollout_corr/kl:np.float64(0.0007732100730457603) - rollout_corr/k3_kl:np.float64(0.000771619481270136) - rollout_corr/rollout_ppl:np.float64(1.0990605875849724) - rollout_corr/rollout_log_ppl:np.float64(0.09202332204586128) - rollout_corr/log_ppl_diff:np.float64(0.0007732108642812818) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020941188995493576) - rollout_corr/log_ppl_diff_max:np.float64(0.02633790671825409) - rollout_corr/log_ppl_diff_min:np.float64(-0.011187225580215454) - rollout_corr/ppl_ratio:np.float64(1.0007794103585184) - rollout_corr/chi2_token:np.float64(0.0014043895062059164) - rollout_corr/chi2_seq:np.float64(0.37330453749746084) - actor/pg_loss:np.float64(0.0019935755990445614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005535308105208969) - actor/ppo_kl:np.float64(0.00010920122786206576) - actor/pg_clipfrac_lower:np.float64(7.207103408291005e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(222890.26285817288) - self_distillation/token_reweight_w_std:np.float64(2919852.600875127) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000309090828523) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03591290376789402) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3092605173587799) - perf/mfu/actor:np.float64(0.03432852611920223) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(144.16852188110352) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020937828812748194 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020937828812748194 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.8828125 - response_length/max:1186.0 - response_length/min:51.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.8828125 - response_length_non_aborted/max:1186.0 - response_length_non_aborted/min:51.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.379349648952484e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.595955204218626) - timing_s/agent_loop/generate_sequences/max:np.float64(8.028898818418384) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3284817362364265) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.028898818418384) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:1142 - timing_s/gen:13.935343297198415 - timing_s/reward:0.058442361652851105 - timing_s/old_log_prob:2.4146821219474077 - timing_s/adv:0.5538248158991337 - timing_s/update_actor:15.78230369463563 - timing_s/step:32.8320482596755 - timing_s/stop_profile:5.641952157020569e-05 - timing_per_token_ms/update_actor:0.10453379760386036 - timing_per_token_ms/adv:0.0036682484593724496 - timing_per_token_ms/gen:0.16254920444649965 - perf/total_num_tokens:150978 - perf/time_per_step:32.8320482596755 - perf/throughput:574.811837834041 (TaskRunner pid=2074004) Training Progress: 32%|███▏ | 32/100 [27:27<1:02:27, 55.11s/it] (TaskRunner pid=2074004) step:33 - global_seqlen/min:19167 - global_seqlen/max:32062 - global_seqlen/minmax_diff:12895 - global_seqlen/balanced_min:24910 - global_seqlen/balanced_max:31094 - global_seqlen/mean:25699.125 - actor/entropy:0.0963616594672203 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5835636854171753 - training/rollout_probs_diff_mean:0.002623476553708315 - training/rollout_probs_diff_std:0.011050577275454998 - training/rollout_actor_probs_pearson_corr:0.9976565837860107 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000323057174683 - rollout_corr/rollout_is_ratio_fraction_high:2.285731716256123e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.571463432512246e-05 - rollout_corr/rollout_is_max:3.547874927520752 - rollout_corr/rollout_is_min:0.27111193537712097 - rollout_corr/rollout_is_std:0.030399184674024582 - rollout_corr/rollout_is_eff_sample_size:0.9990767427790138 - rollout_corr/rollout_is_seq_mean:1.0000334978103638 - rollout_corr/rollout_is_seq_std:0.0019103900995105505 - rollout_corr/rollout_is_seq_max:1.0073792934417725 - rollout_corr/rollout_is_seq_min:0.9880338907241821 - rollout_corr/rollout_is_seq_max_deviation:0.011966109275817871 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1100693135522306) - rollout_corr/training_log_ppl:np.float64(0.10072795890664565) - rollout_corr/kl:np.float64(0.0004943630928924136) - rollout_corr/k3_kl:np.float64(0.0005992158173953044) - rollout_corr/rollout_ppl:np.float64(1.1094494266435504) - rollout_corr/rollout_log_ppl:np.float64(0.10023359479600913) - rollout_corr/log_ppl_diff:np.float64(0.0004943641106365249) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015578279126202688) - rollout_corr/log_ppl_diff_max:np.float64(0.022818326950073242) - rollout_corr/log_ppl_diff_min:np.float64(-0.005478024482727051) - rollout_corr/ppl_ratio:np.float64(1.0004977090284228) - rollout_corr/chi2_token:np.float64(0.001456292113289237) - rollout_corr/chi2_seq:np.float64(0.8751170034520328) - actor/pg_loss:np.float64(0.0023588838521391153) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005614726733256248) - actor/ppo_kl:np.float64(6.841486872133373e-06) - actor/pg_clipfrac_lower:np.float64(4.124868155486183e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(73720.56168391788) - self_distillation/token_reweight_w_std:np.float64(1263082.4041759328) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995736547280103) - self_distillation/token_reweight_w_clip_frac:np.float64(0.038523146067745984) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.31921055912971497) - perf/mfu/actor:np.float64(0.04383612915635905) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.20793533325195) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010148649103939533 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010148649103939533 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:512.69140625 - response_length/max:8192.0 - response_length/min:90.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:512.69140625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001917630434036255 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0906069297343493) - timing_s/agent_loop/generate_sequences/max:np.float64(50.3866535294801) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.971057376002136) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.3866535294801) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.46107683330774 - timing_s/reward:0.06383568607270718 - timing_s/old_log_prob:2.6003563329577446 - timing_s/adv:0.6247836276888847 - timing_s/update_actor:17.063110006973147 - timing_s/step:76.9013549759984 - timing_s/stop_profile:0.00010899640619754791 - timing_per_token_ms/update_actor:0.0829946058813926 - timing_per_token_ms/adv:0.003038934339636489 - timing_per_token_ms/gen:0.4301829105997588 - perf/total_num_tokens:205593 - perf/time_per_step:76.9013549759984 - perf/throughput:334.1829933688545 (TaskRunner pid=2074004) Training Progress: 33%|███▎ | 33/100 [28:44<1:08:51, 61.66s/it] (TaskRunner pid=2074004) step:34 - global_seqlen/min:17957 - global_seqlen/max:26538 - global_seqlen/minmax_diff:8581 - global_seqlen/balanced_min:21718 - global_seqlen/balanced_max:21733 - global_seqlen/mean:21729.125 - actor/entropy:0.09093478322029114 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351693630218506 - training/rollout_probs_diff_mean:0.002648385940119624 - training/rollout_probs_diff_std:0.01128794439136982 - training/rollout_actor_probs_pearson_corr:0.9973158240318298 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0001046657562256 - rollout_corr/rollout_is_ratio_fraction_high:3.717990330187604e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.29497618926689e-05 - rollout_corr/rollout_is_max:3.9687960147857666 - rollout_corr/rollout_is_min:0.3704526722431183 - rollout_corr/rollout_is_std:0.03138892725110054 - rollout_corr/rollout_is_eff_sample_size:0.9990158239877898 - rollout_corr/rollout_is_seq_mean:0.9999611377716064 - rollout_corr/rollout_is_seq_std:0.002279487904161215 - rollout_corr/rollout_is_seq_max:1.009277582168579 - rollout_corr/rollout_is_seq_min:0.9918825626373291 - rollout_corr/rollout_is_seq_max_deviation:0.009277582168579102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0867361915297806) - rollout_corr/training_log_ppl:np.float64(0.08140930149238557) - rollout_corr/kl:np.float64(0.0006834588668453989) - rollout_corr/k3_kl:np.float64(0.0006041397766978207) - rollout_corr/rollout_ppl:np.float64(1.0859681307338178) - rollout_corr/rollout_log_ppl:np.float64(0.08072584179171827) - rollout_corr/log_ppl_diff:np.float64(0.0006834597006672993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016792384558357298) - rollout_corr/log_ppl_diff_max:np.float64(0.011221528053283691) - rollout_corr/log_ppl_diff_min:np.float64(-0.004778027534484863) - rollout_corr/ppl_ratio:np.float64(1.000686518382281) - rollout_corr/chi2_token:np.float64(0.0010883505456149578) - rollout_corr/chi2_seq:np.float64(0.5784096252173185) - actor/pg_loss:np.float64(0.002307310584001243) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005154957439117425) - actor/ppo_kl:np.float64(9.805531538464862e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(82543.4987949559) - self_distillation/token_reweight_w_std:np.float64(1531421.5975041706) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002939573023468) - self_distillation/token_reweight_w_clip_frac:np.float64(0.032446051089209504) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.2603403516113758) - perf/mfu/actor:np.float64(0.0392491945722704) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.23517227172852) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008258585818111897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008258585818111897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.25390625 - response_length/max:1275.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.25390625 - response_length_non_aborted/max:1275.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014248304069042206 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9343549367040396) - timing_s/agent_loop/generate_sequences/max:np.float64(9.114851927384734) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.215552492758434) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.114851927384734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1275 - timing_s/gen:14.953994173556566 - timing_s/reward:0.05961340852081776 - timing_s/old_log_prob:2.531619004905224 - timing_s/adv:0.5816438850015402 - timing_s/update_actor:15.924356224015355 - timing_s/step:34.133271373808384 - timing_s/stop_profile:4.3351203203201294e-05 - timing_per_token_ms/update_actor:0.09160721050672402 - timing_per_token_ms/adv:0.003345992331729535 - timing_per_token_ms/gen:0.13899701792588712 - perf/total_num_tokens:173833 - perf/time_per_step:34.133271373808384 - perf/throughput:636.5966145475729 (TaskRunner pid=2074004) Training Progress: 34%|███▍ | 34/100 [29:18<58:45, 53.41s/it] (TaskRunner pid=2074004) step:35 - global_seqlen/min:17441 - global_seqlen/max:33644 - global_seqlen/minmax_diff:16203 - global_seqlen/balanced_min:21811 - global_seqlen/balanced_max:28467 - global_seqlen/mean:22649.75 - actor/entropy:0.10204526782035828 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.293226420879364 - training/rollout_probs_diff_mean:0.002843219321221113 - training/rollout_probs_diff_std:0.011386570520699024 - training/rollout_actor_probs_pearson_corr:0.9975972771644592 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.000219702720642 - rollout_corr/rollout_is_ratio_fraction_high:1.7400079741491936e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011310051922919229 - rollout_corr/rollout_is_max:2.0903584957122803 - rollout_corr/rollout_is_min:0.10869106650352478 - rollout_corr/rollout_is_std:0.03217843919992447 - rollout_corr/rollout_is_eff_sample_size:0.9989663329398729 - rollout_corr/rollout_is_seq_mean:1.0003238916397095 - rollout_corr/rollout_is_seq_std:0.002396445255726576 - rollout_corr/rollout_is_seq_max:1.0162620544433594 - rollout_corr/rollout_is_seq_min:0.9926619529724121 - rollout_corr/rollout_is_seq_max_deviation:0.016262054443359375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1133964317850769) - rollout_corr/training_log_ppl:np.float64(0.10447760274473694) - rollout_corr/kl:np.float64(0.000506502414840071) - rollout_corr/k3_kl:np.float64(0.0008595675468114905) - rollout_corr/rollout_ppl:np.float64(1.1128244805149734) - rollout_corr/rollout_log_ppl:np.float64(0.10397110023404821) - rollout_corr/log_ppl_diff:np.float64(0.0005065025106887333) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018531291279941797) - rollout_corr/log_ppl_diff_max:np.float64(0.026727454736828804) - rollout_corr/log_ppl_diff_min:np.float64(-0.01405394822359085) - rollout_corr/ppl_ratio:np.float64(1.000511220889166) - rollout_corr/chi2_token:np.float64(0.0022407646756619215) - rollout_corr/chi2_seq:np.float64(1.0442664730362594) - actor/pg_loss:np.float64(0.002358463010750711) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048152222007047385) - actor/ppo_kl:np.float64(0.0001821644597370664) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(110187.43466792861) - self_distillation/token_reweight_w_std:np.float64(1515968.7875927906) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997799128759652) - self_distillation/token_reweight_w_clip_frac:np.float64(0.034502132126362994) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.34709084033966064) - perf/mfu/actor:np.float64(0.0391226831248303) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.17388153076172) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.053757548332214355 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.053757548332214355 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:448.9921875 - response_length/max:8192.0 - response_length/min:68.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:448.9921875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.4147873520851135e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8165026474744081) - timing_s/agent_loop/generate_sequences/max:np.float64(50.60485251620412) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.408925204807019) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.60485251620412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.32023567520082 - timing_s/reward:0.0652729794383049 - timing_s/old_log_prob:2.546900112181902 - timing_s/adv:0.5270522274076939 - timing_s/update_actor:16.483777582645416 - timing_s/step:76.03851772285998 - timing_s/stop_profile:0.00011594034731388092 - timing_per_token_ms/update_actor:0.0909710790552071 - timing_per_token_ms/adv:0.0029087088566523574 - timing_per_token_ms/gen:0.489988304320447 - perf/total_num_tokens:181198 - perf/time_per_step:76.03851772285998 - perf/throughput:297.87206113817564 (TaskRunner pid=2074004) Training Progress: 35%|███▌ | 35/100 [30:34<1:05:13, 60.21s/it] (TaskRunner pid=2074004) step:36 - global_seqlen/min:18954 - global_seqlen/max:33911 - global_seqlen/minmax_diff:14957 - global_seqlen/balanced_min:26378 - global_seqlen/balanced_max:32662 - global_seqlen/mean:27187.75 - actor/entropy:0.1043354868888855 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4853993356227875 - training/rollout_probs_diff_mean:0.0027279683854430914 - training/rollout_probs_diff_std:0.011042559519410133 - training/rollout_actor_probs_pearson_corr:0.9978550672531128 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.000059723854065 - rollout_corr/rollout_is_ratio_fraction_high:2.6941470423480496e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.082440763246268e-05 - rollout_corr/rollout_is_max:7.419532775878906 - rollout_corr/rollout_is_min:0.13505788147449493 - rollout_corr/rollout_is_std:0.03132620081305504 - rollout_corr/rollout_is_eff_sample_size:0.9990196311945923 - rollout_corr/rollout_is_seq_mean:1.000086784362793 - rollout_corr/rollout_is_seq_std:0.0020494619384407997 - rollout_corr/rollout_is_seq_max:1.015174388885498 - rollout_corr/rollout_is_seq_min:0.99062579870224 - rollout_corr/rollout_is_seq_max_deviation:0.015174388885498047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1137358797714114) - rollout_corr/training_log_ppl:np.float64(0.10603692273616616) - rollout_corr/kl:np.float64(0.0006563753371509051) - rollout_corr/k3_kl:np.float64(0.0007457188598607445) - rollout_corr/rollout_ppl:np.float64(1.1129927351139486) - rollout_corr/rollout_log_ppl:np.float64(0.10538054626522353) - rollout_corr/log_ppl_diff:np.float64(0.0006563764709426323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015180988721112953) - rollout_corr/log_ppl_diff_max:np.float64(0.011238820850849152) - rollout_corr/log_ppl_diff_min:np.float64(-0.007395319640636444) - rollout_corr/ppl_ratio:np.float64(1.0006589936092496) - rollout_corr/chi2_token:np.float64(0.002004736103117466) - rollout_corr/chi2_seq:np.float64(1.2244051673915237) - actor/pg_loss:np.float64(0.0032588703325018287) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008896479857867234) - actor/ppo_kl:np.float64(8.400913770501006e-05) - actor/pg_clipfrac_lower:np.float64(5.440459517558338e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(142463.33054139256) - self_distillation/token_reweight_w_std:np.float64(2692305.3041795073) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015312230680138) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0585212862060871) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.31766650825738907) - perf/mfu/actor:np.float64(0.04633240200426601) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.1927490234375) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04938371479511261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04938371479511261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:579.9609375 - response_length/max:8192.0 - response_length/min:82.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:579.9609375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.186511695384979e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9956839792430401) - timing_s/agent_loop/generate_sequences/max:np.float64(51.712549494579434) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.89511316808057) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.712549494579434) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.78715647570789 - timing_s/reward:0.06627766788005829 - timing_s/old_log_prob:2.5614236313849688 - timing_s/adv:0.6225856561213732 - timing_s/update_actor:17.082536520436406 - timing_s/step:78.20577510073781 - timing_s/stop_profile:0.00011078454554080963 - timing_per_token_ms/update_actor:0.07853967559119643 - timing_per_token_ms/adv:0.0028624364655100787 - timing_per_token_ms/gen:0.3892177306911018 - perf/total_num_tokens:217502 - perf/time_per_step:78.20577510073781 - perf/throughput:347.6437637115562 (TaskRunner pid=2074004) Training Progress: 36%|███▌ | 36/100 [31:52<1:09:59, 65.62s/it] (TaskRunner pid=2074004) step:37 - global_seqlen/min:18791 - global_seqlen/max:27840 - global_seqlen/minmax_diff:9049 - global_seqlen/balanced_min:23579 - global_seqlen/balanced_max:23755 - global_seqlen/mean:23627.625 - actor/entropy:0.10102436691522598 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39139458537101746 - training/rollout_probs_diff_mean:0.002805094001814723 - training/rollout_probs_diff_std:0.011440390720963478 - training/rollout_actor_probs_pearson_corr:0.9976106882095337 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9998602271080017 - rollout_corr/rollout_is_ratio_fraction_high:1.6350957594113424e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.9052869144361466e-05 - rollout_corr/rollout_is_max:2.243018388748169 - rollout_corr/rollout_is_min:0.0117238974198699 - rollout_corr/rollout_is_std:0.03203921392560005 - rollout_corr/rollout_is_eff_sample_size:0.9989743035225582 - rollout_corr/rollout_is_seq_mean:0.9999796152114868 - rollout_corr/rollout_is_seq_std:0.002273126505315304 - rollout_corr/rollout_is_seq_max:1.0130821466445923 - rollout_corr/rollout_is_seq_min:0.9926750659942627 - rollout_corr/rollout_is_seq_max_deviation:0.013082146644592285 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0996112595312297) - rollout_corr/training_log_ppl:np.float64(0.09333795910788467) - rollout_corr/kl:np.float64(0.001013359834161065) - rollout_corr/k3_kl:np.float64(0.0008460905967808685) - rollout_corr/rollout_ppl:np.float64(1.0984738464467227) - rollout_corr/rollout_log_ppl:np.float64(0.09232459875784116) - rollout_corr/log_ppl_diff:np.float64(0.0010133603500435129) - rollout_corr/log_ppl_abs_diff:np.float64(0.00191976049245568) - rollout_corr/log_ppl_diff_max:np.float64(0.014345422387123108) - rollout_corr/log_ppl_diff_min:np.float64(-0.009604066610336304) - rollout_corr/ppl_ratio:np.float64(1.001017474103719) - rollout_corr/chi2_token:np.float64(0.001261905301362276) - rollout_corr/chi2_seq:np.float64(0.5114637641236186) - actor/pg_loss:np.float64(0.003147437470033765) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007599365783335088) - actor/ppo_kl:np.float64(0.0003716098390409428) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(192101.5823432547) - self_distillation/token_reweight_w_std:np.float64(2797782.555044028) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008915911894292) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04522743531560991) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.35525741800665855) - perf/mfu/actor:np.float64(0.04261905670880045) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.2181911468506) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008520892821252346 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008520892821252346 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.80078125 - response_length/max:2318.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.80078125 - response_length_non_aborted/max:2318.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.429400622844696e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.102793700993061) - timing_s/agent_loop/generate_sequences/max:np.float64(14.33990266919136) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.5264077913961955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.33990266919136) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2318 - timing_s/gen:20.240060921758413 - timing_s/reward:0.06487942114472389 - timing_s/old_log_prob:2.515492904931307 - timing_s/adv:0.5939651485532522 - timing_s/update_actor:16.080622201785445 - timing_s/step:39.57463042251766 - timing_s/stop_profile:0.00011385232210159302 - timing_per_token_ms/update_actor:0.08507320457401794 - timing_per_token_ms/adv:0.0031423235966017123 - timing_per_token_ms/gen:0.16547218229484384 - perf/total_num_tokens:189021 - perf/time_per_step:39.57463042251766 - perf/throughput:597.0396879955716 (TaskRunner pid=2074004) Training Progress: 37%|███▋ | 37/100 [32:32<1:00:42, 57.82s/it] (TaskRunner pid=2074004) step:38 - global_seqlen/min:17600 - global_seqlen/max:27502 - global_seqlen/minmax_diff:9902 - global_seqlen/balanced_min:21161 - global_seqlen/balanced_max:21168 - global_seqlen/mean:21164.5 - actor/entropy:0.09618508070707321 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6138811707496643 - training/rollout_probs_diff_mean:0.0026226078625768423 - training/rollout_probs_diff_std:0.011445230804383755 - training/rollout_actor_probs_pearson_corr:0.9974937438964844 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000364780426025 - rollout_corr/rollout_is_ratio_fraction_high:1.979805892915465e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.899029828375205e-05 - rollout_corr/rollout_is_max:2.0751965045928955 - rollout_corr/rollout_is_min:0.013780022040009499 - rollout_corr/rollout_is_std:0.03070547617971897 - rollout_corr/rollout_is_eff_sample_size:0.9990580618130113 - rollout_corr/rollout_is_seq_mean:1.0000827312469482 - rollout_corr/rollout_is_seq_std:0.0019129818538203835 - rollout_corr/rollout_is_seq_max:1.0122212171554565 - rollout_corr/rollout_is_seq_min:0.9918540716171265 - rollout_corr/rollout_is_seq_max_deviation:0.012221217155456543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0946312695741653) - rollout_corr/training_log_ppl:np.float64(0.08725073869572952) - rollout_corr/kl:np.float64(0.0006405033833267737) - rollout_corr/k3_kl:np.float64(0.0005329421040762838) - rollout_corr/rollout_ppl:np.float64(1.0939017231576145) - rollout_corr/rollout_log_ppl:np.float64(0.08661023495005793) - rollout_corr/log_ppl_diff:np.float64(0.0006405037456715945) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014858933245704975) - rollout_corr/log_ppl_diff_max:np.float64(0.014180347323417664) - rollout_corr/log_ppl_diff_min:np.float64(-0.004229940474033356) - rollout_corr/ppl_ratio:np.float64(1.0006429618224502) - rollout_corr/chi2_token:np.float64(0.0008146455511450768) - rollout_corr/chi2_seq:np.float64(0.17395253549329937) - actor/pg_loss:np.float64(0.002348349546082318) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007172689474828076) - actor/ppo_kl:np.float64(0.00023065414274014984) - actor/pg_clipfrac_lower:np.float64(1.249999968422344e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(223555.59363828856) - self_distillation/token_reweight_w_std:np.float64(3377666.5284842458) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001167273381725) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04519080827594735) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2977074235677719) - perf/mfu/actor:np.float64(0.03844596025288242) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.3445816040039) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034980697091668844 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034980697091668844 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:394.609375 - response_length/max:1200.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.609375 - response_length_non_aborted/max:1200.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.56140798330307e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0009879916906357) - timing_s/agent_loop/generate_sequences/max:np.float64(8.732898579910398) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.088401710039761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.732898579910398) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1200 - timing_s/gen:14.707298658788204 - timing_s/reward:0.05728561617434025 - timing_s/old_log_prob:2.3959624897688627 - timing_s/adv:0.5882252883166075 - timing_s/update_actor:15.931603338569403 - timing_s/step:33.75794658437371 - timing_s/stop_profile:0.0001205597072839737 - timing_per_token_ms/update_actor:0.09409390334386238 - timing_per_token_ms/adv:0.0034741270069964295 - timing_per_token_ms/gen:0.1455879890990715 - perf/total_num_tokens:169316 - perf/time_per_step:33.75794658437371 - perf/throughput:626.9486785015793 (TaskRunner pid=2074004) Training Progress: 38%|███▊ | 38/100 [33:06<52:18, 50.62s/it] (TaskRunner pid=2074004) step:39 - global_seqlen/min:14192 - global_seqlen/max:21395 - global_seqlen/minmax_diff:7203 - global_seqlen/balanced_min:18859 - global_seqlen/balanced_max:18868 - global_seqlen/mean:18864.75 - actor/entropy:0.08581894636154175 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8518780469894409 - training/rollout_probs_diff_mean:0.0027508889324963093 - training/rollout_probs_diff_std:0.012479580007493496 - training/rollout_actor_probs_pearson_corr:0.9966431260108948 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.82421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9999743700027466 - rollout_corr/rollout_is_ratio_fraction_high:2.358101301069837e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010611455945763737 - rollout_corr/rollout_is_max:2.9851624965667725 - rollout_corr/rollout_is_min:0.14805817604064941 - rollout_corr/rollout_is_std:0.03196844086050987 - rollout_corr/rollout_is_eff_sample_size:0.9989789431740315 - rollout_corr/rollout_is_seq_mean:1.0000545978546143 - rollout_corr/rollout_is_seq_std:0.0020085289143025875 - rollout_corr/rollout_is_seq_max:1.0090090036392212 - rollout_corr/rollout_is_seq_min:0.9939286708831787 - rollout_corr/rollout_is_seq_max_deviation:0.009009003639221191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.090072206221521) - rollout_corr/training_log_ppl:np.float64(0.08314310060450225) - rollout_corr/kl:np.float64(0.0007816835961591551) - rollout_corr/k3_kl:np.float64(0.0008022212134584095) - rollout_corr/rollout_ppl:np.float64(1.0891828034073114) - rollout_corr/rollout_log_ppl:np.float64(0.08236141606539604) - rollout_corr/log_ppl_diff:np.float64(0.0007816845391062088) - rollout_corr/log_ppl_abs_diff:np.float64(0.00177329323196318) - rollout_corr/log_ppl_diff_max:np.float64(0.045620523393154144) - rollout_corr/log_ppl_diff_min:np.float64(-0.008172988891601562) - rollout_corr/ppl_ratio:np.float64(1.0007883713115007) - rollout_corr/chi2_token:np.float64(0.0012875364627689123) - rollout_corr/chi2_seq:np.float64(0.3463961035013199) - actor/pg_loss:np.float64(0.0016197055811062455) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035407407904131105) - actor/ppo_kl:np.float64(0.00027624938138259836) - actor/pg_clipfrac_lower:np.float64(1.398360836901702e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_w_mean:np.float64(47769.31919008936) - self_distillation/token_reweight_w_std:np.float64(767346.1878132067) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993690950796008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02541799924802035) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.3479584529995918) - perf/mfu/actor:np.float64(0.0347302901051022) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.15287017822266) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007721307687461376 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007721307687461376 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:331.3046875 - response_length/max:879.0 - response_length/min:65.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.3046875 - response_length_non_aborted/max:879.0 - response_length_non_aborted/min:65.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011703930795192719 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9549711402505636) - timing_s/agent_loop/generate_sequences/max:np.float64(6.732800455763936) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4961698439146858) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.732800455763936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:317 - timing_s/agent_loop/slowest/response_length:879 - timing_s/gen:12.607269439846277 - timing_s/reward:0.05702884867787361 - timing_s/old_log_prob:2.344845812767744 - timing_s/adv:0.5173551607877016 - timing_s/update_actor:15.629758136346936 - timing_s/step:31.24427343159914 - timing_s/stop_profile:0.000109916552901268 - timing_per_token_ms/update_actor:0.10356457239260351 - timing_per_token_ms/adv:0.003428054710423552 - timing_per_token_ms/gen:0.14864608955887326 - perf/total_num_tokens:150918 - perf/time_per_step:31.24427343159914 - perf/throughput:603.7826432833924 (TaskRunner pid=2074004) Training Progress: 39%|███▉ | 39/100 [33:37<45:34, 44.83s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest option is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2074004) step:40 - global_seqlen/min:17338 - global_seqlen/max:29776 - global_seqlen/minmax_diff:12438 - global_seqlen/balanced_min:22374 - global_seqlen/balanced_max:22391 - global_seqlen/mean:22384.625 - actor/entropy:0.089930459856987 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3171451985836029 - training/rollout_probs_diff_mean:0.0025929557159543037 - training/rollout_probs_diff_std:0.011268679052591324 - training/rollout_actor_probs_pearson_corr:0.9974833726882935 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.95703125 - self_distillation/correct_sample_fraction:0.80078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.95703125 - rollout_corr/rollout_is_mean:1.000073790550232 - rollout_corr/rollout_is_ratio_fraction_high:1.7818165360949934e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.4545413402374834e-05 - rollout_corr/rollout_is_max:2.4635725021362305 - rollout_corr/rollout_is_min:0.2820615768432617 - rollout_corr/rollout_is_std:0.03070935793220997 - rollout_corr/rollout_is_eff_sample_size:0.9990580618130113 - rollout_corr/rollout_is_seq_mean:0.9999992847442627 - rollout_corr/rollout_is_seq_std:0.001931522972881794 - rollout_corr/rollout_is_seq_max:1.0066325664520264 - rollout_corr/rollout_is_seq_min:0.9905868768692017 - rollout_corr/rollout_is_seq_max_deviation:0.00941312313079834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0896218931302428) - rollout_corr/training_log_ppl:np.float64(0.08464303463551914) - rollout_corr/kl:np.float64(0.0005157805179152319) - rollout_corr/k3_kl:np.float64(0.0005807906663477524) - rollout_corr/rollout_ppl:np.float64(1.0890426081605256) - rollout_corr/rollout_log_ppl:np.float64(0.08412725336529547) - rollout_corr/log_ppl_diff:np.float64(0.0005157812702236697) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015912332892185077) - rollout_corr/log_ppl_diff_max:np.float64(0.009794257581233978) - rollout_corr/log_ppl_diff_min:np.float64(-0.0059789493680000305) - rollout_corr/ppl_ratio:np.float64(1.0005182197783142) - rollout_corr/chi2_token:np.float64(0.0013064609374850988) - rollout_corr/chi2_seq:np.float64(1.130385466851294) - actor/pg_loss:np.float64(0.002167836995795369) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005737871229030134) - actor/ppo_kl:np.float64(1.1175101938931675e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.95703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.95703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_w_mean:np.float64(140985.0619100898) - self_distillation/token_reweight_w_std:np.float64(2265311.5997434272) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001305928453803) - self_distillation/token_reweight_w_clip_frac:np.float64(0.036739757866598666) - self_distillation/empty_target_batch:np.float64(0.04296875) - actor/grad_norm:np.float64(0.2651498392224312) - perf/mfu/actor:np.float64(0.04031842311642975) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.09762573242188) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.625) - val-aux/sciknoweval/reward/std@16:np.float64(0.17935140746688946) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6972125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15230234946062082) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5531125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15011488729407602) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6247875) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17933428616120461) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.757625) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11678637676576642) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49601249999999997) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11323686397372593) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6350625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1464125828088421) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8041375000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08263344427572242) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4525625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0819621762464158) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6428) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1088052134913493) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8369375000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.051110544163678794) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4195875) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05406867605156187) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.648425) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.08121353362515363) - val-aux/sciknoweval/score/mean@16:np.float64(0.625) - val-aux/sciknoweval/score/std@16:np.float64(0.17935140746688946) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6972125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15230234946062082) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5531125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15011488729407602) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6247875) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17933428616120461) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.757625) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11678637676576642) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49601249999999997) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11323686397372593) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6350625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1464125828088421) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8041375000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08263344427572242) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4525625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0819621762464158) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6428) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1088052134913493) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8369375000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.051110544163678794) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4195875) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05406867605156187) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.648425) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.08121353362515363) - val-core/sciknoweval/acc/mean@16:np.float64(0.625) - val-aux/sciknoweval/acc/std@16:np.float64(0.17935140746688946) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6972125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15230234946062082) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5531125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15011488729407602) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6247875) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17933428616120461) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.757625) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11678637676576642) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49601249999999997) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11323686397372593) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6350625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1464125828088421) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8041375000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08263344427572242) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4525625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0819621762464158) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6428) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1088052134913493) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8369375000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.051110544163678794) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4195875) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05406867605156187) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.648425) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.08121353362515363) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9921875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.024141732305909894) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9989125000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.008527445661552757) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.985275) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03137280736233215) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9916750000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.024734393444018433) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00039508701573197775) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9734375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03851008868178078) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9960000000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01673449238498939) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.956475) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04152269363307004) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9987) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00904727412664771) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.936625) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03653224445364453) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999775) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002781121632849242) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000915408250875771 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000915408250875771 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:438.45703125 - response_length/max:1467.0 - response_length/min:64.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:438.45703125 - response_length_non_aborted/max:1467.0 - response_length_non_aborted/min:64.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014247745275497437 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8024486973881721) - timing_s/agent_loop/generate_sequences/max:np.float64(10.08868633955717) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.353498277778272) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.08868633955717) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1467 - timing_s/gen:16.001727301627398 - timing_s/reward:0.061644136905670166 - timing_s/old_log_prob:2.4333597384393215 - timing_s/adv:0.5891087763011456 - timing_s/update_actor:15.919175075367093 - timing_s/step:35.09279179573059 - timing_s/testing:93.67140794359148 - timing_s/save_checkpoint:6.656575288623571 - timing_s/stop_profile:0.00014066509902477264 - timing_per_token_ms/update_actor:0.08889569891927547 - timing_per_token_ms/adv:0.0032896953617781487 - timing_per_token_ms/gen:0.14256071363203168 - perf/total_num_tokens:179077 - perf/time_per_step:35.09279179573059 - perf/throughput:637.8695981299306 (TaskRunner pid=2074004) Training Progress: 40%|████ | 40/100 [35:53<1:12:01, 72.03s/it] (TaskRunner pid=2074004) step:41 - global_seqlen/min:20646 - global_seqlen/max:30800 - global_seqlen/minmax_diff:10154 - global_seqlen/balanced_min:23788 - global_seqlen/balanced_max:23817 - global_seqlen/mean:23809.0 - actor/entropy:0.08412408828735352 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083496570587158 - training/rollout_probs_diff_mean:0.002506777411326766 - training/rollout_probs_diff_std:0.010917266830801964 - training/rollout_actor_probs_pearson_corr:0.9974930882453918 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:1.6747051631682552e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.698820652673021e-05 - rollout_corr/rollout_is_max:2.611849784851074 - rollout_corr/rollout_is_min:0.2993384003639221 - rollout_corr/rollout_is_std:0.030081849545240402 - rollout_corr/rollout_is_eff_sample_size:0.999095900418749 - rollout_corr/rollout_is_seq_mean:0.9999955296516418 - rollout_corr/rollout_is_seq_std:0.00155645702034235 - rollout_corr/rollout_is_seq_max:1.0051690340042114 - rollout_corr/rollout_is_seq_min:0.9948093891143799 - rollout_corr/rollout_is_seq_max_deviation:0.005190610885620117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.084649627096951) - rollout_corr/training_log_ppl:np.float64(0.08000787740093074) - rollout_corr/kl:np.float64(0.00042501650833115523) - rollout_corr/k3_kl:np.float64(0.0005248165592366405) - rollout_corr/rollout_ppl:np.float64(1.084169534035027) - rollout_corr/rollout_log_ppl:np.float64(0.07958286030770978) - rollout_corr/log_ppl_diff:np.float64(0.0004250170932209585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013505659480870236) - rollout_corr/log_ppl_diff_max:np.float64(0.006395198404788971) - rollout_corr/log_ppl_diff_min:np.float64(-0.0090675950050354) - rollout_corr/ppl_ratio:np.float64(1.0004267231561244) - rollout_corr/chi2_token:np.float64(0.0012936093844473362) - rollout_corr/chi2_seq:np.float64(1.8906177775934339) - actor/pg_loss:np.float64(0.002206359291449189) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006284501350819482) - actor/ppo_kl:np.float64(-4.702512273624393e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(146167.46774941264) - self_distillation/token_reweight_w_std:np.float64(2535003.0120101087) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000315876677632) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04627777053974569) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.32163915038108826) - perf/mfu/actor:np.float64(0.04315536156757415) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.23479080200195) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009857733733952045 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009857733733952045 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:466.5 - response_length/max:1590.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:466.5 - response_length_non_aborted/max:1590.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.379800081253052e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2317185122519732) - timing_s/agent_loop/generate_sequences/max:np.float64(10.718674024567008) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.755297555726429) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.718674024567008) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1590 - timing_s/gen:16.80528495274484 - timing_s/reward:0.0609660129994154 - timing_s/old_log_prob:2.4625763557851315 - timing_s/adv:0.5879520736634731 - timing_s/update_actor:15.929620191454887 - timing_s/step:35.9443339202553 - timing_s/stop_profile:0.00010867416858673096 - timing_per_token_ms/update_actor:0.08363234591674833 - timing_per_token_ms/adv:0.003086816296691761 - timing_per_token_ms/gen:0.14071949484814478 - perf/total_num_tokens:190472 - perf/time_per_step:35.9443339202553 - perf/throughput:662.3853443166236 (TaskRunner pid=2074004) Training Progress: 41%|████ | 41/100 [36:29<1:00:11, 61.22s/it] (TaskRunner pid=2074004) step:42 - global_seqlen/min:20360 - global_seqlen/max:28679 - global_seqlen/minmax_diff:8319 - global_seqlen/balanced_min:24195 - global_seqlen/balanced_max:24221 - global_seqlen/mean:24215.75 - actor/entropy:0.09680385887622833 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6651126146316528 - training/rollout_probs_diff_mean:0.00268166302703321 - training/rollout_probs_diff_std:0.011481569148600101 - training/rollout_actor_probs_pearson_corr:0.9974598288536072 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999688863754272 - rollout_corr/rollout_is_ratio_fraction_high:2.390247755101882e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001195123913930729 - rollout_corr/rollout_is_max:3.0732650756835938 - rollout_corr/rollout_is_min:0.28940898180007935 - rollout_corr/rollout_is_std:0.030898984521627426 - rollout_corr/rollout_is_eff_sample_size:0.99904604449077 - rollout_corr/rollout_is_seq_mean:0.999945342540741 - rollout_corr/rollout_is_seq_std:0.001563522731885314 - rollout_corr/rollout_is_seq_max:1.0046355724334717 - rollout_corr/rollout_is_seq_min:0.9948148131370544 - rollout_corr/rollout_is_seq_max_deviation:0.005185186862945557 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0920318001881242) - rollout_corr/training_log_ppl:np.float64(0.0861618878916488) - rollout_corr/kl:np.float64(0.0007285167887651767) - rollout_corr/k3_kl:np.float64(0.0005457256108627462) - rollout_corr/rollout_ppl:np.float64(1.0912344371899962) - rollout_corr/rollout_log_ppl:np.float64(0.08543337112496374) - rollout_corr/log_ppl_diff:np.float64(0.0007285167666850612) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013972546003060415) - rollout_corr/log_ppl_diff_max:np.float64(0.014810048043727875) - rollout_corr/log_ppl_diff_min:np.float64(-0.0045769959688186646) - rollout_corr/ppl_ratio:np.float64(1.0007309333886951) - rollout_corr/chi2_token:np.float64(0.0006357615347951651) - rollout_corr/chi2_seq:np.float64(0.37331055640242994) - actor/pg_loss:np.float64(0.002819624845869839) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005845594248512498) - actor/ppo_kl:np.float64(0.00019738566773241928) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(118144.65402136883) - self_distillation/token_reweight_w_std:np.float64(2489355.6096611773) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007247163448483) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0432515628344845) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.2655489929020405) - perf/mfu/actor:np.float64(0.043623625719631715) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.27901077270508) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013405305799096823 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013405305799096823 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:490.2734375 - response_length/max:1590.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:490.2734375 - response_length_non_aborted/max:1590.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001036711037158966 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1733751446008682) - timing_s/agent_loop/generate_sequences/max:np.float64(11.362804541364312) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.0733919305566815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.362804541364312) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1590 - timing_s/gen:17.271388456225395 - timing_s/reward:0.06370862200856209 - timing_s/old_log_prob:2.4125118106603622 - timing_s/adv:0.5772267878055573 - timing_s/update_actor:16.07286419905722 - timing_s/step:36.48842146247625 - timing_s/stop_profile:0.00012125447392463684 - timing_per_token_ms/update_actor:0.08296699564878861 - timing_per_token_ms/adv:0.002979604120281001 - timing_per_token_ms/gen:0.13760966023604013 - perf/total_num_tokens:193726 - perf/time_per_step:36.48842146247625 - perf/throughput:663.6557304870765 (TaskRunner pid=2074004) Training Progress: 42%|████▏ | 42/100 [37:05<52:01, 53.81s/it] (TaskRunner pid=2074004) step:43 - global_seqlen/min:18165 - global_seqlen/max:29195 - global_seqlen/minmax_diff:11030 - global_seqlen/balanced_min:23675 - global_seqlen/balanced_max:23697 - global_seqlen/mean:23692.125 - actor/entropy:0.1023547351360321 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8032844066619873 - training/rollout_probs_diff_mean:0.0026671267114579678 - training/rollout_probs_diff_std:0.01125924102962017 - training/rollout_actor_probs_pearson_corr:0.9977118372917175 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:1.0000076293945312 - rollout_corr/rollout_is_ratio_fraction_high:1.6602332834736444e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012451749353203923 - rollout_corr/rollout_is_max:2.4638938903808594 - rollout_corr/rollout_is_min:0.01720614731311798 - rollout_corr/rollout_is_std:0.03037957102060318 - rollout_corr/rollout_is_eff_sample_size:0.9990778136835871 - rollout_corr/rollout_is_seq_mean:1.0000402927398682 - rollout_corr/rollout_is_seq_std:0.0017050451133400202 - rollout_corr/rollout_is_seq_max:1.0078315734863281 - rollout_corr/rollout_is_seq_min:0.9947986006736755 - rollout_corr/rollout_is_seq_max_deviation:0.007831573486328125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1006618356332183) - rollout_corr/training_log_ppl:np.float64(0.09350979735609144) - rollout_corr/kl:np.float64(0.000630945518594217) - rollout_corr/k3_kl:np.float64(0.0005851791151911812) - rollout_corr/rollout_ppl:np.float64(1.0999378017149866) - rollout_corr/rollout_log_ppl:np.float64(0.09287885023513809) - rollout_corr/log_ppl_diff:np.float64(0.0006309471209533513) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015661002835258842) - rollout_corr/log_ppl_diff_max:np.float64(0.007478758692741394) - rollout_corr/log_ppl_diff_min:np.float64(-0.007118046283721924) - rollout_corr/ppl_ratio:np.float64(1.000633135670796) - rollout_corr/chi2_token:np.float64(0.0010510697029531002) - rollout_corr/chi2_seq:np.float64(0.48250076407566667) - actor/pg_loss:np.float64(0.0034622400999069214) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007893792012509948) - actor/ppo_kl:np.float64(0.00017401028439456567) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(130333.681850672) - self_distillation/token_reweight_w_std:np.float64(2574542.6272122907) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005807324778289) - self_distillation/token_reweight_w_clip_frac:np.float64(0.056435555168718565) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.34318234771490097) - perf/mfu/actor:np.float64(0.04306781395749705) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.32390975952148) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010272692888975143 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010272692888975143 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.56640625 - response_length/max:1498.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.56640625 - response_length_non_aborted/max:1498.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.386613965034485e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1191709153354168) - timing_s/agent_loop/generate_sequences/max:np.float64(10.657457020133734) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.579246542722103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.657457020133734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1498 - timing_s/gen:16.503861967474222 - timing_s/reward:0.05490053817629814 - timing_s/old_log_prob:2.4478375520557165 - timing_s/adv:0.5797670539468527 - timing_s/update_actor:15.963536273688078 - timing_s/step:35.651620699092746 - timing_s/stop_profile:3.4825876355171204e-05 - timing_per_token_ms/update_actor:0.08422385219607822 - timing_per_token_ms/adv:0.003058859504723894 - timing_per_token_ms/gen:0.1370013030131094 - perf/total_num_tokens:189537 - perf/time_per_step:35.651620699092746 - perf/throughput:664.545525152042 (TaskRunner pid=2074004) Training Progress: 43%|████▎ | 43/100 [37:41<45:57, 48.37s/it] (TaskRunner pid=2074004) step:44 - global_seqlen/min:18372 - global_seqlen/max:30473 - global_seqlen/minmax_diff:12101 - global_seqlen/balanced_min:24351 - global_seqlen/balanced_max:24371 - global_seqlen/mean:24365.0 - actor/entropy:0.0982256531715393 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797517716884613 - training/rollout_probs_diff_mean:0.0026915573980659246 - training/rollout_probs_diff_std:0.011107339523732662 - training/rollout_actor_probs_pearson_corr:0.9976074695587158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0001195669174194 - rollout_corr/rollout_is_ratio_fraction_high:1.613319545867853e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0332990465685725e-05 - rollout_corr/rollout_is_max:3.7719850540161133 - rollout_corr/rollout_is_min:0.08076421916484833 - rollout_corr/rollout_is_std:0.030701594427227974 - rollout_corr/rollout_is_eff_sample_size:0.9990586567374643 - rollout_corr/rollout_is_seq_mean:1.000076413154602 - rollout_corr/rollout_is_seq_std:0.0018174588913097978 - rollout_corr/rollout_is_seq_max:1.0099468231201172 - rollout_corr/rollout_is_seq_min:0.9935795664787292 - rollout_corr/rollout_is_seq_max_deviation:0.009946823120117188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0983206499367952) - rollout_corr/training_log_ppl:np.float64(0.091351964041678) - rollout_corr/kl:np.float64(0.0005401147962658115) - rollout_corr/k3_kl:np.float64(0.0005354781641990769) - rollout_corr/rollout_ppl:np.float64(1.0977034606039524) - rollout_corr/rollout_log_ppl:np.float64(0.09081184863316594) - rollout_corr/log_ppl_diff:np.float64(0.0005401154085120652) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013354642323974986) - rollout_corr/log_ppl_diff_max:np.float64(0.010618716478347778) - rollout_corr/log_ppl_diff_min:np.float64(-0.006194006651639938) - rollout_corr/ppl_ratio:np.float64(1.000541973626241) - rollout_corr/chi2_token:np.float64(0.0011658316943794489) - rollout_corr/chi2_seq:np.float64(0.8551819822750986) - actor/pg_loss:np.float64(0.00199379853438586) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002786078871395148) - actor/ppo_kl:np.float64(0.00011415650995871829) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(42518.53975814115) - self_distillation/token_reweight_w_std:np.float64(780929.215723624) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994724853895605) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03163288858195301) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.21826045215129852) - perf/mfu/actor:np.float64(0.04429235206402637) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.40694427490234) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01199503056704998 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01199503056704998 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:484.25 - response_length/max:1568.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:484.25 - response_length_non_aborted/max:1568.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6304274797439575e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7051511611789465) - timing_s/agent_loop/generate_sequences/max:np.float64(11.325649071484804) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.876759573868185) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.325649071484804) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1568 - timing_s/gen:17.214315151795745 - timing_s/reward:0.06438344530761242 - timing_s/old_log_prob:2.4776456244289875 - timing_s/adv:0.5917312819510698 - timing_s/update_actor:15.933018373325467 - timing_s/step:36.368578501045704 - timing_s/stop_profile:4.634261131286621e-05 - timing_per_token_ms/update_actor:0.08174132143097408 - timing_per_token_ms/adv:0.003035764836605119 - timing_per_token_ms/gen:0.13886095727765024 - perf/total_num_tokens:194920 - perf/time_per_step:36.368578501045704 - perf/throughput:669.946448396366 (TaskRunner pid=2074004) Training Progress: 44%|████▍ | 44/100 [38:17<41:47, 44.78s/it] (TaskRunner pid=2074004) step:45 - global_seqlen/min:18847 - global_seqlen/max:30065 - global_seqlen/minmax_diff:11218 - global_seqlen/balanced_min:23771 - global_seqlen/balanced_max:23792 - global_seqlen/mean:23779.375 - actor/entropy:0.11290297657251358 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7185672521591187 - training/rollout_probs_diff_mean:0.0027959917206317186 - training/rollout_probs_diff_std:0.011578291654586792 - training/rollout_actor_probs_pearson_corr:0.9977681040763855 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000535249710083 - rollout_corr/rollout_is_ratio_fraction_high:8.408802386838943e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0001177232334157452 - rollout_corr/rollout_is_max:2.5759212970733643 - rollout_corr/rollout_is_min:0.01617681048810482 - rollout_corr/rollout_is_std:0.031018352136015892 - rollout_corr/rollout_is_eff_sample_size:0.9990390246046145 - rollout_corr/rollout_is_seq_mean:1.0000488758087158 - rollout_corr/rollout_is_seq_std:0.001919785514473915 - rollout_corr/rollout_is_seq_max:1.009157657623291 - rollout_corr/rollout_is_seq_min:0.9913671612739563 - rollout_corr/rollout_is_seq_max_deviation:0.009157657623291016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1144616291858256) - rollout_corr/training_log_ppl:np.float64(0.10538233329680224) - rollout_corr/kl:np.float64(0.0005276777127427579) - rollout_corr/k3_kl:np.float64(0.000598768252578985) - rollout_corr/rollout_ppl:np.float64(1.1138749476522207) - rollout_corr/rollout_log_ppl:np.float64(0.1048546542024269) - rollout_corr/log_ppl_diff:np.float64(0.000527679094375344) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015542648143309634) - rollout_corr/log_ppl_diff_max:np.float64(0.010761316865682602) - rollout_corr/log_ppl_diff_min:np.float64(-0.007235199213027954) - rollout_corr/ppl_ratio:np.float64(1.000530268996954) - rollout_corr/chi2_token:np.float64(0.001289344858378172) - rollout_corr/chi2_seq:np.float64(0.600404967321083) - actor/pg_loss:np.float64(0.003451700205914676) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003801528589519876) - actor/ppo_kl:np.float64(-1.3993248870747266e-07) - actor/pg_clipfrac_lower:np.float64(1.1601573987718439e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(70150.05770411994) - self_distillation/token_reweight_w_std:np.float64(1347604.018264451) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003975154832006) - self_distillation/token_reweight_w_clip_frac:np.float64(0.047006876673549414) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.3081394359469414) - perf/mfu/actor:np.float64(0.04282063737580456) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.50539779663086) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011149020865559578 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011149020865559578 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:464.54296875 - response_length/max:1638.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:464.54296875 - response_length_non_aborted/max:1638.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003325045108795166 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6967908591032028) - timing_s/agent_loop/generate_sequences/max:np.float64(11.396734926849604) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.698892642998544) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.396734926849604) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:375 - timing_s/agent_loop/slowest/response_length:1638 - timing_s/gen:17.557129818946123 - timing_s/reward:0.05820976942777634 - timing_s/old_log_prob:2.601362394168973 - timing_s/adv:0.6350415386259556 - timing_s/update_actor:16.227487675845623 - timing_s/step:37.15261420793831 - timing_s/stop_profile:0.00015691295266151428 - timing_per_token_ms/update_actor:0.08530232436641849 - timing_per_token_ms/adv:0.003338195067290223 - timing_per_token_ms/gen:0.1476344342048731 - perf/total_num_tokens:190235 - perf/time_per_step:37.15261420793831 - perf/throughput:640.0458085374546 (TaskRunner pid=2074004) Training Progress: 45%|████▌ | 45/100 [38:55<39:05, 42.64s/it] (TaskRunner pid=2074004) step:46 - global_seqlen/min:14903 - global_seqlen/max:28076 - global_seqlen/minmax_diff:13173 - global_seqlen/balanced_min:20663 - global_seqlen/balanced_max:20685 - global_seqlen/mean:20676.625 - actor/entropy:0.09572755545377731 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6351487040519714 - training/rollout_probs_diff_mean:0.0026079132221639156 - training/rollout_probs_diff_std:0.01122567243874073 - training/rollout_actor_probs_pearson_corr:0.9975275993347168 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0001404285430908 - rollout_corr/rollout_is_ratio_fraction_high:3.958632441936061e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.9379482991062105e-05 - rollout_corr/rollout_is_max:5.243876934051514 - rollout_corr/rollout_is_min:0.22313041985034943 - rollout_corr/rollout_is_std:0.030579039826989174 - rollout_corr/rollout_is_eff_sample_size:0.9990660338595477 - rollout_corr/rollout_is_seq_mean:1.0001130104064941 - rollout_corr/rollout_is_seq_std:0.002018552739173174 - rollout_corr/rollout_is_seq_max:1.0071282386779785 - rollout_corr/rollout_is_seq_min:0.9935094714164734 - rollout_corr/rollout_is_seq_max_deviation:0.007128238677978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0899349767714739) - rollout_corr/training_log_ppl:np.float64(0.08403534088211018) - rollout_corr/kl:np.float64(0.0005288002676930859) - rollout_corr/k3_kl:np.float64(0.0005856861293978) - rollout_corr/rollout_ppl:np.float64(1.089340966194868) - rollout_corr/rollout_log_ppl:np.float64(0.08350654012610903) - rollout_corr/log_ppl_diff:np.float64(0.0005288007560011465) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017128633153333794) - rollout_corr/log_ppl_diff_max:np.float64(0.009660359472036362) - rollout_corr/log_ppl_diff_min:np.float64(-0.006222076714038849) - rollout_corr/ppl_ratio:np.float64(1.000531758647412) - rollout_corr/chi2_token:np.float64(0.0013514722231775522) - rollout_corr/chi2_seq:np.float64(0.6378451467026025) - actor/pg_loss:np.float64(0.001488825073465705) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003111429728051007) - actor/ppo_kl:np.float64(0.00012299883569388825) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(29391.911929843947) - self_distillation/token_reweight_w_std:np.float64(617711.6287392898) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997125186491758) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02729684312362224) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.15495813265442848) - perf/mfu/actor:np.float64(0.03730698959695489) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.52793884277344) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007471918594092131 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007471918594092131 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:394.70703125 - response_length/max:1505.0 - response_length/min:67.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.70703125 - response_length_non_aborted/max:1505.0 - response_length_non_aborted/min:67.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011683255434036255 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8251722864806652) - timing_s/agent_loop/generate_sequences/max:np.float64(10.226146340370178) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7836934844599455) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.226146340370178) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1505 - timing_s/gen:16.00960760936141 - timing_s/reward:0.0563168041408062 - timing_s/old_log_prob:2.468111554160714 - timing_s/adv:0.5863060262054205 - timing_s/update_actor:15.916377088055015 - timing_s/step:35.136502992361784 - timing_s/stop_profile:0.00013731792569160461 - timing_per_token_ms/update_actor:0.09622204474893155 - timing_per_token_ms/adv:0.0035444978702122596 - timing_per_token_ms/gen:0.1584403741833976 - perf/total_num_tokens:165413 - perf/time_per_step:35.136502992361784 - perf/throughput:588.4656479472311 (TaskRunner pid=2074004) Training Progress: 46%|████▌ | 46/100 [39:30<36:21, 40.40s/it] (TaskRunner pid=2074004) step:47 - global_seqlen/min:18667 - global_seqlen/max:30924 - global_seqlen/minmax_diff:12257 - global_seqlen/balanced_min:24432 - global_seqlen/balanced_max:24607 - global_seqlen/mean:24464.125 - actor/entropy:0.093343086540699 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28589993715286255 - training/rollout_probs_diff_mean:0.0024511476512998343 - training/rollout_probs_diff_std:0.010416332632303238 - training/rollout_actor_probs_pearson_corr:0.997839629650116 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.78125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0000395774841309 - rollout_corr/rollout_is_ratio_fraction_high:1.5632695067324676e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.471443000715226e-05 - rollout_corr/rollout_is_max:3.1934804916381836 - rollout_corr/rollout_is_min:0.2297939509153366 - rollout_corr/rollout_is_std:0.028684185817837715 - rollout_corr/rollout_is_eff_sample_size:0.9991780129114268 - rollout_corr/rollout_is_seq_mean:0.9999590516090393 - rollout_corr/rollout_is_seq_std:0.0018708055140450597 - rollout_corr/rollout_is_seq_max:1.0055705308914185 - rollout_corr/rollout_is_seq_min:0.9900708198547363 - rollout_corr/rollout_is_seq_max_deviation:0.009929180145263672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.095050911884755) - rollout_corr/training_log_ppl:np.float64(0.08821879087918205) - rollout_corr/kl:np.float64(0.00046318759317642844) - rollout_corr/k3_kl:np.float64(0.0005760851533125333) - rollout_corr/rollout_ppl:np.float64(1.0945486696437001) - rollout_corr/rollout_log_ppl:np.float64(0.0877556023770012) - rollout_corr/log_ppl_diff:np.float64(0.0004631885021808557) - rollout_corr/log_ppl_abs_diff:np.float64(0.001563526107929647) - rollout_corr/log_ppl_diff_max:np.float64(0.01143985241651535) - rollout_corr/log_ppl_diff_min:np.float64(-0.00824648141860962) - rollout_corr/ppl_ratio:np.float64(1.0004658417310566) - rollout_corr/chi2_token:np.float64(0.001373328734189272) - rollout_corr/chi2_seq:np.float64(0.9047737107612193) - actor/pg_loss:np.float64(0.0018651984864845872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004548156155124161) - actor/ppo_kl:np.float64(-4.627550882929654e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_w_mean:np.float64(51240.47999361623) - self_distillation/token_reweight_w_std:np.float64(1059937.9959096338) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991865828633308) - self_distillation/token_reweight_w_clip_frac:np.float64(0.035300534596899524) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.20760727860033512) - perf/mfu/actor:np.float64(0.04301014037057002) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.6991195678711) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010366430506110191 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010366430506110191 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:499.75390625 - response_length/max:2319.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:499.75390625 - response_length_non_aborted/max:2319.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.209476411342621e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8284339234232903) - timing_s/agent_loop/generate_sequences/max:np.float64(14.41846733354032) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.81055764901248) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.41846733354032) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:2319 - timing_s/gen:20.333935126662254 - timing_s/reward:0.06512773968279362 - timing_s/old_log_prob:2.491063568741083 - timing_s/adv:0.754039952531457 - timing_s/update_actor:16.0744774825871 - timing_s/step:39.806080570444465 - timing_s/stop_profile:0.00010729581117630005 - timing_per_token_ms/update_actor:0.08213290625858834 - timing_per_token_ms/adv:0.0038527841918087045 - timing_per_token_ms/gen:0.1589370950285082 - perf/total_num_tokens:195713 - perf/time_per_step:39.806080570444465 - perf/throughput:614.5826127419417 (TaskRunner pid=2074004) Training Progress: 47%|████▋ | 47/100 [40:10<35:32, 40.24s/it] (TaskRunner pid=2074004) step:48 - global_seqlen/min:14563 - global_seqlen/max:26256 - global_seqlen/minmax_diff:11693 - global_seqlen/balanced_min:21802 - global_seqlen/balanced_max:21832 - global_seqlen/mean:21823.375 - actor/entropy:0.08697269856929779 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8820101022720337 - training/rollout_probs_diff_mean:0.0027279374189674854 - training/rollout_probs_diff_std:0.01222990918904543 - training/rollout_actor_probs_pearson_corr:0.9968819618225098 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.0002572536468506 - rollout_corr/rollout_is_ratio_fraction_high:4.7129349695751444e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013196218060329556 - rollout_corr/rollout_is_max:2.5243570804595947 - rollout_corr/rollout_is_min:0.09723177552223206 - rollout_corr/rollout_is_std:0.03248079866170883 - rollout_corr/rollout_is_eff_sample_size:0.9989465854769791 - rollout_corr/rollout_is_seq_mean:1.0003472566604614 - rollout_corr/rollout_is_seq_std:0.002232569269835949 - rollout_corr/rollout_is_seq_max:1.0138044357299805 - rollout_corr/rollout_is_seq_min:0.9910656213760376 - rollout_corr/rollout_is_seq_max_deviation:0.013804435729980469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0930587616749108) - rollout_corr/training_log_ppl:np.float64(0.08755744152949774) - rollout_corr/kl:np.float64(0.0005523046467796888) - rollout_corr/k3_kl:np.float64(0.00072532679760684) - rollout_corr/rollout_ppl:np.float64(1.0924336458556354) - rollout_corr/rollout_log_ppl:np.float64(0.0870051367310225) - rollout_corr/log_ppl_diff:np.float64(0.0005523047984752338) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017461000934417825) - rollout_corr/log_ppl_diff_max:np.float64(0.012880250811576843) - rollout_corr/log_ppl_diff_min:np.float64(-0.008728347718715668) - rollout_corr/ppl_ratio:np.float64(1.0005556710530072) - rollout_corr/chi2_token:np.float64(0.0017671133391559124) - rollout_corr/chi2_seq:np.float64(0.6951510151848197) - actor/pg_loss:np.float64(0.002507986268028617) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006494858009773452) - actor/ppo_kl:np.float64(0.0003249875776454303) - actor/pg_clipfrac_lower:np.float64(4.164445726928534e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(161986.24940627953) - self_distillation/token_reweight_w_std:np.float64(2150402.851084696) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997133247088641) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04499576549278572) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.378052793443203) - perf/mfu/actor:np.float64(0.04008844180647516) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.73404788970947) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010018521919846535 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010018521919846535 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:414.41796875 - response_length/max:1405.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.41796875 - response_length_non_aborted/max:1405.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017635710537433624 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7887936737388372) - timing_s/agent_loop/generate_sequences/max:np.float64(9.995722759515047) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.310525228465849) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.995722759515047) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1405 - timing_s/gen:16.018609369173646 - timing_s/reward:0.06094110943377018 - timing_s/old_log_prob:2.350804327055812 - timing_s/adv:0.5859208330512047 - timing_s/update_actor:15.523810938000679 - timing_s/step:34.628727208822966 - timing_s/stop_profile:0.00015043653547763824 - timing_per_token_ms/update_actor:0.08891733598721943 - timing_per_token_ms/adv:0.0033560392987519384 - timing_per_token_ms/gen:0.15098933339466727 - perf/total_num_tokens:174587 - perf/time_per_step:34.628727208822966 - perf/throughput:630.2101393561956 (TaskRunner pid=2074004) Training Progress: 48%|████▊ | 48/100 [40:45<33:25, 38.57s/it] (TaskRunner pid=2074004) step:49 - global_seqlen/min:15971 - global_seqlen/max:23148 - global_seqlen/minmax_diff:7177 - global_seqlen/balanced_min:19974 - global_seqlen/balanced_max:20001 - global_seqlen/mean:19989.25 - actor/entropy:0.08722288906574249 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4621162414550781 - training/rollout_probs_diff_mean:0.0027316466439515352 - training/rollout_probs_diff_std:0.012118976563215256 - training/rollout_actor_probs_pearson_corr:0.9968992471694946 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.76171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:1.0001474618911743 - rollout_corr/rollout_is_ratio_fraction_high:4.222527059027925e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.500686428509653e-05 - rollout_corr/rollout_is_max:2.718278646469116 - rollout_corr/rollout_is_min:0.33812370896339417 - rollout_corr/rollout_is_std:0.03343567997217178 - rollout_corr/rollout_is_eff_sample_size:0.9988835415737872 - rollout_corr/rollout_is_seq_mean:1.0000269412994385 - rollout_corr/rollout_is_seq_std:0.0024658842012286186 - rollout_corr/rollout_is_seq_max:1.0119198560714722 - rollout_corr/rollout_is_seq_min:0.9833274483680725 - rollout_corr/rollout_is_seq_max_deviation:0.01667255163192749 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0898574301972985) - rollout_corr/training_log_ppl:np.float64(0.084673610603204) - rollout_corr/kl:np.float64(0.0007291726103133556) - rollout_corr/k3_kl:np.float64(0.0008918259389361083) - rollout_corr/rollout_ppl:np.float64(1.0890288641676307) - rollout_corr/rollout_log_ppl:np.float64(0.08394443713768851) - rollout_corr/log_ppl_diff:np.float64(0.0007291734655154869) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020636038680095226) - rollout_corr/log_ppl_diff_max:np.float64(0.021112143993377686) - rollout_corr/log_ppl_diff_min:np.float64(-0.007896922528743744) - rollout_corr/ppl_ratio:np.float64(1.0007340346928686) - rollout_corr/chi2_token:np.float64(0.002316090976819396) - rollout_corr/chi2_seq:np.float64(1.0663579376414418) - actor/pg_loss:np.float64(0.0028226502472534776) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006949842643280135) - actor/ppo_kl:np.float64(0.00011967911359889172) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_w_mean:np.float64(197467.6685416242) - self_distillation/token_reweight_w_std:np.float64(2992394.127208988) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000012104865164) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04289876340772025) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.39689869433641434) - perf/mfu/actor:np.float64(0.03667143020489837) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.6230697631836) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0072983745485544205 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0072983745485544205 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:370.0390625 - response_length/max:1377.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:370.0390625 - response_length_non_aborted/max:1377.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011089816689491272 - timing_s/agent_loop/generate_sequences/min:np.float64(0.894102580845356) - timing_s/agent_loop/generate_sequences/max:np.float64(9.03606103733182) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6704066668535233) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.03606103733182) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1377 - timing_s/gen:15.202601235359907 - timing_s/reward:0.05952438898384571 - timing_s/old_log_prob:2.3819591104984283 - timing_s/adv:0.6096546649932861 - timing_s/update_actor:15.689758809283376 - timing_s/step:34.03324189968407 - timing_s/stop_profile:0.0001423526555299759 - timing_per_token_ms/update_actor:0.09811372868719047 - timing_per_token_ms/adv:0.0038123908162717842 - timing_per_token_ms/gen:0.16048349240325036 - perf/total_num_tokens:159914 - perf/time_per_step:34.03324189968407 - perf/throughput:587.34486884676 (TaskRunner pid=2074004) Training Progress: 49%|████▉ | 49/100 [41:19<31:38, 37.22s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2074004) step:50 - global_seqlen/min:18143 - global_seqlen/max:30858 - global_seqlen/minmax_diff:12715 - global_seqlen/balanced_min:24044 - global_seqlen/balanced_max:26361 - global_seqlen/mean:24342.0 - actor/entropy:0.10675854980945587 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083586573600769 - training/rollout_probs_diff_mean:0.0026588926557451487 - training/rollout_probs_diff_std:0.011075353249907494 - training/rollout_actor_probs_pearson_corr:0.9978457689285278 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9999751448631287 - rollout_corr/rollout_is_ratio_fraction_high:2.371916525589768e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.115749758668244e-05 - rollout_corr/rollout_is_max:3.3773059844970703 - rollout_corr/rollout_is_min:0.2692754864692688 - rollout_corr/rollout_is_std:0.030405066907405853 - rollout_corr/rollout_is_eff_sample_size:0.9990761478330206 - rollout_corr/rollout_is_seq_mean:0.9999284744262695 - rollout_corr/rollout_is_seq_std:0.002185159595683217 - rollout_corr/rollout_is_seq_max:1.0124692916870117 - rollout_corr/rollout_is_seq_min:0.9912188649177551 - rollout_corr/rollout_is_seq_max_deviation:0.012469291687011719 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1083718882873654) - rollout_corr/training_log_ppl:np.float64(0.10039229939866345) - rollout_corr/kl:np.float64(0.0004886855121668532) - rollout_corr/k3_kl:np.float64(0.0005853448297905572) - rollout_corr/rollout_ppl:np.float64(1.1078288545832038) - rollout_corr/rollout_log_ppl:np.float64(0.09990361310701701) - rollout_corr/log_ppl_diff:np.float64(0.0004886862916464452) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016839853378769476) - rollout_corr/log_ppl_diff_max:np.float64(0.010547831654548645) - rollout_corr/log_ppl_diff_min:np.float64(-0.011270195245742798) - rollout_corr/ppl_ratio:np.float64(1.000491572311148) - rollout_corr/chi2_token:np.float64(0.0013913614675402641) - rollout_corr/chi2_seq:np.float64(1.0534377871081233) - actor/pg_loss:np.float64(0.0014425351982936263) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040900465501181316) - actor/ppo_kl:np.float64(-0.00011984414344468064) - actor/pg_clipfrac_lower:np.float64(3.814697265625e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(121532.27847940498) - self_distillation/token_reweight_w_std:np.float64(2120513.0500425613) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004689684137702) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03253905474412022) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.20551946759223938) - perf/mfu/actor:np.float64(0.04243176899328058) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.7705078125) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63984375) - val-aux/sciknoweval/reward/std@16:np.float64(0.15617237535672698) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.70185) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.12792221226476658) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5791125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13673480051872064) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6415) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1561483190784494) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7504125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09557563742116812) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5252625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10832359511761105) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6525000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12038866118646771) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7888999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06876976879223133) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.47989999999999994) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0796137581343784) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6589125000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07946446968361667) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8169375000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04228047841467854) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.448425) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.047402393710538074) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6619124999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05008887435519682) - val-aux/sciknoweval/score/mean@16:np.float64(0.63984375) - val-aux/sciknoweval/score/std@16:np.float64(0.15617237535672698) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.70185) - val-aux/sciknoweval/score/best@2/std:np.float64(0.12792221226476658) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5791125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13673480051872064) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6415) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1561483190784494) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7504125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09557563742116812) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5252625) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10832359511761105) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6525000000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12038866118646771) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7888999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06876976879223133) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.47989999999999994) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0796137581343784) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6589125000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07946446968361667) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8169375000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04228047841467854) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.448425) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.047402393710538074) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6619124999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05008887435519682) - val-core/sciknoweval/acc/mean@16:np.float64(0.63984375) - val-aux/sciknoweval/acc/std@16:np.float64(0.15617237535672698) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.70185) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.12792221226476658) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5791125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13673480051872064) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6415) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1561483190784494) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7504125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09557563742116812) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5252625) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10832359511761105) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6525000000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12038866118646771) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7888999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06876976879223133) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.47989999999999994) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0796137581343784) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6589125000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07946446968361667) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8169375000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04228047841467854) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.448425) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.047402393710538074) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6619124999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05008887435519682) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9921875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.026059282360820557) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9991375000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.008110749235580297) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.985425) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.03423627902458588) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.992475) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.025689990721416346) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.000683625445693766) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9733875) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04275106884274511) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9975375) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.014926669851977312) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.954525) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.047782282724323125) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996125000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.005601383627055409) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9313375) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.04368822856243866) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00039508701573197775) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0210300050675869 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0210300050675869 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:494.0625 - response_length/max:4102.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:494.0625 - response_length_non_aborted/max:4102.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.833462536334991e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6969410367310047) - timing_s/agent_loop/generate_sequences/max:np.float64(22.719568610191345) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.79131419971236) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.719568610191345) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:295 - timing_s/agent_loop/slowest/response_length:4102 - timing_s/gen:28.734515454620123 - timing_s/reward:0.06417730078101158 - timing_s/old_log_prob:2.4680988006293774 - timing_s/adv:0.8539764694869518 - timing_s/update_actor:16.26316487416625 - timing_s/step:48.47273708879948 - timing_s/testing:90.87557928822935 - timing_s/save_checkpoint:6.687707267701626 - timing_s/stop_profile:0.0001324862241744995 - timing_per_token_ms/update_actor:0.08351391049506127 - timing_per_token_ms/adv:0.004385303536515856 - timing_per_token_ms/gen:0.22718623857226536 - perf/total_num_tokens:194736 - perf/time_per_step:48.47273708879948 - perf/throughput:502.17919312884584 (TaskRunner pid=2074004) Training Progress: 50%|█████ | 50/100 [43:45<58:14, 69.89s/it] (TaskRunner pid=2074004) step:51 - global_seqlen/min:16973 - global_seqlen/max:21967 - global_seqlen/minmax_diff:4994 - global_seqlen/balanced_min:19550 - global_seqlen/balanced_max:19572 - global_seqlen/mean:19567.375 - actor/entropy:0.11901509016752243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083246231079102 - training/rollout_probs_diff_mean:0.003160488326102495 - training/rollout_probs_diff_std:0.012160852551460266 - training/rollout_actor_probs_pearson_corr:0.9976589679718018 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0001776218414307 - rollout_corr/rollout_is_ratio_fraction_high:5.6728575145825744e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.807428871979937e-05 - rollout_corr/rollout_is_max:2.820579767227173 - rollout_corr/rollout_is_min:0.30462947487831116 - rollout_corr/rollout_is_std:0.03463872894644737 - rollout_corr/rollout_is_eff_sample_size:0.9988020720902324 - rollout_corr/rollout_is_seq_mean:1.000294804573059 - rollout_corr/rollout_is_seq_std:0.0022349499631673098 - rollout_corr/rollout_is_seq_max:1.0152208805084229 - rollout_corr/rollout_is_seq_min:0.995274543762207 - rollout_corr/rollout_is_seq_max_deviation:0.015220880508422852 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1107426239177585) - rollout_corr/training_log_ppl:np.float64(0.10169691975897877) - rollout_corr/kl:np.float64(0.0001784627847740694) - rollout_corr/k3_kl:np.float64(0.0006646001246366495) - rollout_corr/rollout_ppl:np.float64(1.1105284076184034) - rollout_corr/rollout_log_ppl:np.float64(0.10151845622021938) - rollout_corr/log_ppl_diff:np.float64(0.00017846353875938803) - rollout_corr/log_ppl_abs_diff:np.float64(0.001676651110756211) - rollout_corr/log_ppl_diff_max:np.float64(0.006288684904575348) - rollout_corr/log_ppl_diff_min:np.float64(-0.010029513388872147) - rollout_corr/ppl_ratio:np.float64(1.0001808677334338) - rollout_corr/chi2_token:np.float64(0.002416089177131653) - rollout_corr/chi2_seq:np.float64(2.1752094991970807) - actor/pg_loss:np.float64(0.0023010852746665478) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047835509440119495) - actor/ppo_kl:np.float64(-0.0001440780888302129) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(79808.69256585464) - self_distillation/token_reweight_w_std:np.float64(1330398.761181703) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998520778026432) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04437096892797854) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.2404463179409504) - perf/mfu/actor:np.float64(0.03516023093043365) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.7251968383789) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004383700899779797 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004383700899779797 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.29296875 - response_length/max:1233.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.29296875 - response_length_non_aborted/max:1233.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.046704947948456e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8948994744569063) - timing_s/agent_loop/generate_sequences/max:np.float64(8.36026605591178) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4736403662464) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.36026605591178) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:310 - timing_s/agent_loop/slowest/response_length:1233 - timing_s/gen:14.1562640350312 - timing_s/reward:0.051813943311572075 - timing_s/old_log_prob:2.4304460305720568 - timing_s/adv:0.6007976606488228 - timing_s/update_actor:15.809248071163893 - timing_s/step:33.15378465689719 - timing_s/stop_profile:0.00011606328189373016 - timing_per_token_ms/update_actor:0.10099239212697086 - timing_per_token_ms/adv:0.0038380062517891565 - timing_per_token_ms/gen:0.1606129413203145 - perf/total_num_tokens:156539 - perf/time_per_step:33.15378465689719 - perf/throughput:590.2003407001462 (TaskRunner pid=2074004) Training Progress: 51%|█████ | 51/100 [44:18<48:05, 58.88s/it] (TaskRunner pid=2074004) step:52 - global_seqlen/min:16957 - global_seqlen/max:26308 - global_seqlen/minmax_diff:9351 - global_seqlen/balanced_min:22416 - global_seqlen/balanced_max:22423 - global_seqlen/mean:22420.25 - actor/entropy:0.11742626130580902 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6386399269104004 - training/rollout_probs_diff_mean:0.0029321101028472185 - training/rollout_probs_diff_std:0.011663330718874931 - training/rollout_actor_probs_pearson_corr:0.9977625608444214 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:0.9999903440475464 - rollout_corr/rollout_is_ratio_fraction_high:3.6525008908938617e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.305001781787723e-05 - rollout_corr/rollout_is_max:3.6345746517181396 - rollout_corr/rollout_is_min:0.252546101808548 - rollout_corr/rollout_is_std:0.03316900134086609 - rollout_corr/rollout_is_eff_sample_size:0.9989007886433641 - rollout_corr/rollout_is_seq_mean:1.0000351667404175 - rollout_corr/rollout_is_seq_std:0.002520139329135418 - rollout_corr/rollout_is_seq_max:1.0145207643508911 - rollout_corr/rollout_is_seq_min:0.9895867109298706 - rollout_corr/rollout_is_seq_max_deviation:0.014520764350891113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1248778235167265) - rollout_corr/training_log_ppl:np.float64(0.11452079268201487) - rollout_corr/kl:np.float64(0.0006545804253192955) - rollout_corr/k3_kl:np.float64(0.0007203538166162815) - rollout_corr/rollout_ppl:np.float64(1.1241182647645473) - rollout_corr/rollout_log_ppl:np.float64(0.11386621197380009) - rollout_corr/log_ppl_diff:np.float64(0.0006545807082147803) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016909198930079583) - rollout_corr/log_ppl_diff_max:np.float64(0.011592693626880646) - rollout_corr/log_ppl_diff_min:np.float64(-0.010950211435556412) - rollout_corr/ppl_ratio:np.float64(1.0006579770706594) - rollout_corr/chi2_token:np.float64(0.0015927678905427456) - rollout_corr/chi2_seq:np.float64(0.5052446881309152) - actor/pg_loss:np.float64(0.0011261987965554) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031590935577696655) - actor/ppo_kl:np.float64(4.731501897392576e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(120173.51662478969) - self_distillation/token_reweight_w_std:np.float64(1645237.523866353) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993055597878993) - self_distillation/token_reweight_w_clip_frac:np.float64(0.026213745499262586) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.3443529047071934) - perf/mfu/actor:np.float64(0.04109087151546889) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.80995559692383) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003660490969195962 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003660490969195962 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:427.7890625 - response_length/max:1240.0 - response_length/min:50.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:427.7890625 - response_length_non_aborted/max:1240.0 - response_length_non_aborted/min:50.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.612925350666046e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6170574445277452) - timing_s/agent_loop/generate_sequences/max:np.float64(9.085840474814177) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.157990079183946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.085840474814177) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1240 - timing_s/gen:14.695505177602172 - timing_s/reward:0.06002468429505825 - timing_s/old_log_prob:2.512907398864627 - timing_s/adv:0.5510312523692846 - timing_s/update_actor:15.702309988439083 - timing_s/step:33.61664153076708 - timing_s/stop_profile:0.00011726468801498413 - timing_per_token_ms/update_actor:0.08754535513898754 - timing_per_token_ms/adv:0.0030721738850441266 - timing_per_token_ms/gen:0.13418837023213628 - perf/total_num_tokens:179362 - perf/time_per_step:33.61664153076708 - perf/throughput:666.9390212428042 (TaskRunner pid=2074004) Training Progress: 52%|█████▏ | 52/100 [44:52<41:03, 51.32s/it] (TaskRunner pid=2074004) step:53 - global_seqlen/min:15926 - global_seqlen/max:32544 - global_seqlen/minmax_diff:16618 - global_seqlen/balanced_min:21720 - global_seqlen/balanced_max:28240 - global_seqlen/mean:22614.625 - actor/entropy:0.09381777793169022 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6157671809196472 - training/rollout_probs_diff_mean:0.0025205647107213736 - training/rollout_probs_diff_std:0.011054250411689281 - training/rollout_actor_probs_pearson_corr:0.9975994229316711 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.73828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000879764556885 - rollout_corr/rollout_is_ratio_fraction_high:4.539635483524762e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.26341677363962e-05 - rollout_corr/rollout_is_max:2.6310083866119385 - rollout_corr/rollout_is_min:0.3212862014770508 - rollout_corr/rollout_is_std:0.029877066612243652 - rollout_corr/rollout_is_eff_sample_size:0.999108513924274 - rollout_corr/rollout_is_seq_mean:1.0001354217529297 - rollout_corr/rollout_is_seq_std:0.0017414779867976904 - rollout_corr/rollout_is_seq_max:1.0066789388656616 - rollout_corr/rollout_is_seq_min:0.9954082369804382 - rollout_corr/rollout_is_seq_max_deviation:0.006678938865661621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.098151988349855) - rollout_corr/training_log_ppl:np.float64(0.09104150990060589) - rollout_corr/kl:np.float64(0.00047328945871338846) - rollout_corr/k3_kl:np.float64(0.0005634092783721734) - rollout_corr/rollout_ppl:np.float64(1.0976255191490054) - rollout_corr/rollout_log_ppl:np.float64(0.09056821986359864) - rollout_corr/log_ppl_diff:np.float64(0.0004732900370072457) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015623152985426714) - rollout_corr/log_ppl_diff_max:np.float64(0.005912806838750839) - rollout_corr/log_ppl_diff_min:np.float64(-0.005394577980041504) - rollout_corr/ppl_ratio:np.float64(1.0004753414541483) - rollout_corr/chi2_token:np.float64(0.0013314660172909498) - rollout_corr/chi2_seq:np.float64(0.4222771213389933) - actor/pg_loss:np.float64(0.002000119653530419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005868266684956325) - actor/ppo_kl:np.float64(0.00012776876268461024) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_w_mean:np.float64(53422.7099313552) - self_distillation/token_reweight_w_std:np.float64(849382.4181836921) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993473722133785) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037509667774429545) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.31460506841540337) - perf/mfu/actor:np.float64(0.03860267849970589) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.8161849975586) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.057434335350990295 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.057434335350990295 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:430.23828125 - response_length/max:8192.0 - response_length/min:95.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:430.23828125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011334381997585297 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0902089104056358) - timing_s/agent_loop/generate_sequences/max:np.float64(50.08467713370919) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.101931656085071) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.08467713370919) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:375 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.888266975060105 - timing_s/reward:0.061832696199417114 - timing_s/old_log_prob:2.569713741540909 - timing_s/adv:0.5755481589585543 - timing_s/update_actor:16.975983068346977 - timing_s/step:76.15641017071903 - timing_s/stop_profile:2.802535891532898e-05 - timing_per_token_ms/update_actor:0.09383299009129588 - timing_per_token_ms/adv:0.003181282902980672 - timing_per_token_ms/gen:0.5074247280763757 - perf/total_num_tokens:180917 - perf/time_per_step:76.15641017071903 - perf/throughput:296.94972424914766 (TaskRunner pid=2074004) Training Progress: 53%|█████▎ | 53/100 [46:08<46:02, 58.79s/it] (TaskRunner pid=2074004) step:54 - global_seqlen/min:16107 - global_seqlen/max:24016 - global_seqlen/minmax_diff:7909 - global_seqlen/balanced_min:19289 - global_seqlen/balanced_max:19501 - global_seqlen/mean:19320.75 - actor/entropy:0.09417933225631714 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39961040019989014 - training/rollout_probs_diff_mean:0.0026771980337798595 - training/rollout_probs_diff_std:0.011605466715991497 - training/rollout_actor_probs_pearson_corr:0.9973964095115662 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0001647472381592 - rollout_corr/rollout_is_ratio_fraction_high:1.1300201549602207e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.040161239681765e-05 - rollout_corr/rollout_is_max:2.7947683334350586 - rollout_corr/rollout_is_min:0.2384055107831955 - rollout_corr/rollout_is_std:0.03145531937479973 - rollout_corr/rollout_is_eff_sample_size:0.9990118978361671 - rollout_corr/rollout_is_seq_mean:1.0001072883605957 - rollout_corr/rollout_is_seq_std:0.0022169763687998056 - rollout_corr/rollout_is_seq_max:1.009650707244873 - rollout_corr/rollout_is_seq_min:0.9886189699172974 - rollout_corr/rollout_is_seq_max_deviation:0.011381030082702637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0925864586606622) - rollout_corr/training_log_ppl:np.float64(0.08678890572628006) - rollout_corr/kl:np.float64(0.0009032392683874235) - rollout_corr/k3_kl:np.float64(0.0007826630679801383) - rollout_corr/rollout_ppl:np.float64(1.0915671489201486) - rollout_corr/rollout_log_ppl:np.float64(0.08588566524122143) - rollout_corr/log_ppl_diff:np.float64(0.0009032404850586317) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018698280400712974) - rollout_corr/log_ppl_diff_max:np.float64(0.03783611208200455) - rollout_corr/log_ppl_diff_min:np.float64(-0.005891826003789902) - rollout_corr/ppl_ratio:np.float64(1.0009094115812331) - rollout_corr/chi2_token:np.float64(0.0010525539983063936) - rollout_corr/chi2_seq:np.float64(0.43362018815241754) - actor/pg_loss:np.float64(0.002331349765881896) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007941815301819588) - actor/ppo_kl:np.float64(0.0004760969166213158) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(237838.98983726162) - self_distillation/token_reweight_w_std:np.float64(3087923.700471521) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999815991614014) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04877196221787017) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.3219018206000328) - perf/mfu/actor:np.float64(0.03530995118577071) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.7523651123047) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000819264561869204 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000819264561869204 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.6796875 - response_length/max:1721.0 - response_length/min:64.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.6796875 - response_length_non_aborted/max:1721.0 - response_length_non_aborted/min:64.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.285946488380432e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.751097746193409) - timing_s/agent_loop/generate_sequences/max:np.float64(10.377681985497475) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.351018370616657) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.377681985497475) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1721 - timing_s/gen:16.176042480394244 - timing_s/reward:0.05808785557746887 - timing_s/old_log_prob:2.318193769082427 - timing_s/adv:0.6144152693450451 - timing_s/update_actor:15.822785630822182 - timing_s/step:35.07668521627784 - timing_s/stop_profile:0.00010946020483970642 - timing_per_token_ms/update_actor:0.10236912148093488 - timing_per_token_ms/adv:0.00397509975897057 - timing_per_token_ms/gen:0.18279253373555546 - perf/total_num_tokens:154566 - perf/time_per_step:35.07668521627784 - perf/throughput:550.8145904001764 (TaskRunner pid=2074004) Training Progress: 54%|█████▍ | 54/100 [46:43<39:37, 51.69s/it] (TaskRunner pid=2074004) step:55 - global_seqlen/min:14652 - global_seqlen/max:26201 - global_seqlen/minmax_diff:11549 - global_seqlen/balanced_min:20421 - global_seqlen/balanced_max:21183 - global_seqlen/mean:20601.125 - actor/entropy:0.085003562271595 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7348287105560303 - training/rollout_probs_diff_mean:0.0025023941416293383 - training/rollout_probs_diff_std:0.01154287438839674 - training/rollout_actor_probs_pearson_corr:0.9971373081207275 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:1.0001779794692993 - rollout_corr/rollout_is_ratio_fraction_high:1.0315978215658106e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013410771498456597 - rollout_corr/rollout_is_max:2.935729742050171 - rollout_corr/rollout_is_min:0.038828011602163315 - rollout_corr/rollout_is_std:0.030352089554071426 - rollout_corr/rollout_is_eff_sample_size:0.999079955499621 - rollout_corr/rollout_is_seq_mean:1.0003106594085693 - rollout_corr/rollout_is_seq_std:0.002173425629734993 - rollout_corr/rollout_is_seq_max:1.0108247995376587 - rollout_corr/rollout_is_seq_min:0.9864917993545532 - rollout_corr/rollout_is_seq_max_deviation:0.013508200645446777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.085447907447815) - rollout_corr/training_log_ppl:np.float64(0.0803521656380326) - rollout_corr/kl:np.float64(0.0008227331833889906) - rollout_corr/k3_kl:np.float64(0.0011211537594633114) - rollout_corr/rollout_ppl:np.float64(1.0845554950647056) - rollout_corr/rollout_log_ppl:np.float64(0.0795294317031221) - rollout_corr/log_ppl_diff:np.float64(0.0008227339349105023) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020583992882166058) - rollout_corr/log_ppl_diff_max:np.float64(0.04965774714946747) - rollout_corr/log_ppl_diff_min:np.float64(-0.009343862533569336) - rollout_corr/ppl_ratio:np.float64(1.0008315925952047) - rollout_corr/chi2_token:np.float64(0.002741375006735325) - rollout_corr/chi2_seq:np.float64(0.736898449016735) - actor/pg_loss:np.float64(0.001779418089427054) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009461770737289044) - actor/ppo_kl:np.float64(0.0004943441474765109) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(220652.25802971283) - self_distillation/token_reweight_w_std:np.float64(2705884.476536175) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000388199230656) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0328305515286047) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.42648517340421677) - perf/mfu/actor:np.float64(0.037390062668015575) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.62273788452148) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006393327843397856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006393327843397856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:378.66015625 - response_length/max:2305.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:378.66015625 - response_length_non_aborted/max:2305.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012177042663097382 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9361808206886053) - timing_s/agent_loop/generate_sequences/max:np.float64(13.556594530120492) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.681363452458754) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.556594530120492) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:318 - timing_s/agent_loop/slowest/response_length:2305 - timing_s/gen:19.531876169145107 - timing_s/reward:0.05353007838129997 - timing_s/old_log_prob:2.345628321170807 - timing_s/adv:0.5701816361397505 - timing_s/update_actor:15.82276027649641 - timing_s/step:38.36092114262283 - timing_s/stop_profile:0.00011600367724895477 - timing_per_token_ms/update_actor:0.0960066517999406 - timing_per_token_ms/adv:0.003459651087863833 - timing_per_token_ms/gen:0.20149041304295684 - perf/total_num_tokens:164809 - perf/time_per_step:38.36092114262283 - perf/throughput:537.0341583667052 (TaskRunner pid=2074004) Training Progress: 55%|█████▌ | 55/100 [47:22<35:46, 47.70s/it] (TaskRunner pid=2074004) step:56 - global_seqlen/min:16673 - global_seqlen/max:30667 - global_seqlen/minmax_diff:13994 - global_seqlen/balanced_min:21519 - global_seqlen/balanced_max:21670 - global_seqlen/mean:21542.25 - actor/entropy:0.0999220460653305 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9910814762115479 - training/rollout_probs_diff_mean:0.0026262723840773106 - training/rollout_probs_diff_std:0.011972086504101753 - training/rollout_actor_probs_pearson_corr:0.9973817467689514 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.92578125 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.92578125 - rollout_corr/rollout_is_mean:1.0003807544708252 - rollout_corr/rollout_is_ratio_fraction_high:4.651249400922097e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.372248703381047e-05 - rollout_corr/rollout_is_max:44.50603103637695 - rollout_corr/rollout_is_min:0.0011786288814619184 - rollout_corr/rollout_is_std:0.030854584649205208 - rollout_corr/rollout_is_eff_sample_size:0.9990490190485594 - rollout_corr/rollout_is_seq_mean:1.0000252723693848 - rollout_corr/rollout_is_seq_std:0.004327365662902594 - rollout_corr/rollout_is_seq_max:1.0554637908935547 - rollout_corr/rollout_is_seq_min:0.9875951409339905 - rollout_corr/rollout_is_seq_max_deviation:0.05546379089355469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0925669269636273) - rollout_corr/training_log_ppl:np.float64(0.08702500175058958) - rollout_corr/kl:np.float64(0.0007599289466115877) - rollout_corr/k3_kl:np.float64(0.000812841809917586) - rollout_corr/rollout_ppl:np.float64(1.0917559866793454) - rollout_corr/rollout_log_ppl:np.float64(0.08626507179360487) - rollout_corr/log_ppl_diff:np.float64(0.0007599299569847062) - rollout_corr/log_ppl_abs_diff:np.float64(0.001818598095269408) - rollout_corr/log_ppl_diff_max:np.float64(0.024008244276046753) - rollout_corr/log_ppl_diff_min:np.float64(-0.014017723500728607) - rollout_corr/ppl_ratio:np.float64(1.0007652402855456) - rollout_corr/chi2_token:np.float64(0.007633969187736511) - rollout_corr/chi2_seq:np.float64(13.315856878180057) - actor/pg_loss:np.float64(0.0017298469319939613) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006172826681449806) - actor/ppo_kl:np.float64(-5.590297072188832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.92578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.92578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(123974.65548743168) - self_distillation/token_reweight_w_std:np.float64(2170329.61304679) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005807334091514) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04290506925462978) - self_distillation/empty_target_batch:np.float64(0.07421875) - actor/grad_norm:np.float64(0.27438871562480927) - perf/mfu/actor:np.float64(0.03876809494412483) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.63893127441406) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024676039814949036 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024676039814949036 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:419.9140625 - response_length/max:2024.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:419.9140625 - response_length_non_aborted/max:2024.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012747198343276978 - timing_s/agent_loop/generate_sequences/min:np.float64(0.48513596691191196) - timing_s/agent_loop/generate_sequences/max:np.float64(13.075700936838984) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.0640050550719025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.075700936838984) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:2024 - timing_s/gen:18.986181549727917 - timing_s/reward:0.06117998994886875 - timing_s/old_log_prob:2.4063704144209623 - timing_s/adv:0.5998788680881262 - timing_s/update_actor:15.829487767070532 - timing_s/step:37.97097833454609 - timing_s/stop_profile:0.00010865926742553711 - timing_per_token_ms/update_actor:0.09185140692749441 - timing_per_token_ms/adv:0.003480827606727049 - timing_per_token_ms/gen:0.17661892825659936 - perf/total_num_tokens:172338 - perf/time_per_step:37.97097833454609 - perf/throughput:567.3346051344905 (TaskRunner pid=2074004) Training Progress: 56%|█████▌ | 56/100 [48:00<32:51, 44.80s/it] (TaskRunner pid=2074004) step:57 - global_seqlen/min:19855 - global_seqlen/max:29475 - global_seqlen/minmax_diff:9620 - global_seqlen/balanced_min:23604 - global_seqlen/balanced_max:23634 - global_seqlen/mean:23625.125 - actor/entropy:0.09015602618455887 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.456105500459671 - training/rollout_probs_diff_mean:0.0024542445316910744 - training/rollout_probs_diff_std:0.011214017868041992 - training/rollout_actor_probs_pearson_corr:0.9974842667579651 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:1.0000454187393188 - rollout_corr/rollout_is_ratio_fraction_high:5.9702000726247206e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010234629007754847 - rollout_corr/rollout_is_max:2.9357409477233887 - rollout_corr/rollout_is_min:0.22341881692409515 - rollout_corr/rollout_is_std:0.030461862683296204 - rollout_corr/rollout_is_eff_sample_size:0.9990729351369017 - rollout_corr/rollout_is_seq_mean:1.0001904964447021 - rollout_corr/rollout_is_seq_std:0.002443463308736682 - rollout_corr/rollout_is_seq_max:1.0193251371383667 - rollout_corr/rollout_is_seq_min:0.9929532408714294 - rollout_corr/rollout_is_seq_max_deviation:0.0193251371383667 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0897927153855562) - rollout_corr/training_log_ppl:np.float64(0.0843119082164776) - rollout_corr/kl:np.float64(0.00043658774763111197) - rollout_corr/k3_kl:np.float64(0.0005583141538294534) - rollout_corr/rollout_ppl:np.float64(1.0893031116575003) - rollout_corr/rollout_log_ppl:np.float64(0.0838753203752276) - rollout_corr/log_ppl_diff:np.float64(0.0004365878412500024) - rollout_corr/log_ppl_abs_diff:np.float64(0.001471972107538022) - rollout_corr/log_ppl_diff_max:np.float64(0.00870583951473236) - rollout_corr/log_ppl_diff_min:np.float64(-0.012766733765602112) - rollout_corr/ppl_ratio:np.float64(1.000438829883933) - rollout_corr/chi2_token:np.float64(0.0013955945614725351) - rollout_corr/chi2_seq:np.float64(1.2546640543732792) - actor/pg_loss:np.float64(0.0011045325081795454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035431018341114395) - actor/ppo_kl:np.float64(0.00010037352073410943) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(72716.30810955423) - self_distillation/token_reweight_w_std:np.float64(1347980.3629608657) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001339728478342) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02943878156656865) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.24138232693076134) - perf/mfu/actor:np.float64(0.0430505553899597) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.70356369018555) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008093886077404022 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008093886077404022 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.00390625 - response_length/max:1631.0 - response_length/min:45.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:458.00390625 - response_length_non_aborted/max:1631.0 - response_length_non_aborted/min:45.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013142824172973633 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5333198085427284) - timing_s/agent_loop/generate_sequences/max:np.float64(11.510037498548627) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.700965259689838) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.510037498548627) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:1631 - timing_s/gen:17.477134620770812 - timing_s/reward:0.0652174036949873 - timing_s/old_log_prob:2.3839294649660587 - timing_s/adv:0.5826396737247705 - timing_s/update_actor:15.731351995840669 - timing_s/step:36.32835609279573 - timing_s/stop_profile:0.00012176670134067535 - timing_per_token_ms/update_actor:0.08323422625192813 - timing_per_token_ms/adv:0.003082733285669232 - timing_per_token_ms/gen:0.1490599887484824 - perf/total_num_tokens:189001 - perf/time_per_step:36.32835609279573 - perf/throughput:650.3218846361477 (TaskRunner pid=2074004) Training Progress: 57%|█████▋ | 57/100 [48:36<30:17, 42.27s/it] (TaskRunner pid=2074004) step:58 - global_seqlen/min:14549 - global_seqlen/max:27391 - global_seqlen/minmax_diff:12842 - global_seqlen/balanced_min:22974 - global_seqlen/balanced_max:22989 - global_seqlen/mean:22981.5 - actor/entropy:0.10685306042432785 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6651362180709839 - training/rollout_probs_diff_mean:0.0026612752117216587 - training/rollout_probs_diff_std:0.011326903477311134 - training/rollout_actor_probs_pearson_corr:0.9977207779884338 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9998153448104858 - rollout_corr/rollout_is_ratio_fraction_high:8.753195288591087e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00012254473404027522 - rollout_corr/rollout_is_max:2.095289468765259 - rollout_corr/rollout_is_min:0.04907945916056633 - rollout_corr/rollout_is_std:0.030399184674024582 - rollout_corr/rollout_is_eff_sample_size:0.9990763858113328 - rollout_corr/rollout_is_seq_mean:0.9997419118881226 - rollout_corr/rollout_is_seq_std:0.0024453294463455677 - rollout_corr/rollout_is_seq_max:1.0176780223846436 - rollout_corr/rollout_is_seq_min:0.9827814102172852 - rollout_corr/rollout_is_seq_max_deviation:0.017678022384643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1037812139838934) - rollout_corr/training_log_ppl:np.float64(0.09657903140760027) - rollout_corr/kl:np.float64(0.0008447893188705535) - rollout_corr/k3_kl:np.float64(0.0005789172635033424) - rollout_corr/rollout_ppl:np.float64(1.1028364589437842) - rollout_corr/rollout_log_ppl:np.float64(0.09573424071641057) - rollout_corr/log_ppl_diff:np.float64(0.0008447906911897007) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017328100257145707) - rollout_corr/log_ppl_diff_max:np.float64(0.02989429235458374) - rollout_corr/log_ppl_diff_min:np.float64(-0.01144128292798996) - rollout_corr/ppl_ratio:np.float64(1.0008499294053763) - rollout_corr/chi2_token:np.float64(0.0006312790792435408) - rollout_corr/chi2_seq:np.float64(0.1915365003515035) - actor/pg_loss:np.float64(0.001689612865447998) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005289059029109922) - actor/ppo_kl:np.float64(3.6509948291296723e-06) - actor/pg_clipfrac_lower:np.float64(5.7276392908534035e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(124004.99291193858) - self_distillation/token_reweight_w_std:np.float64(2293706.94823437) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002958420664072) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03820525103947148) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.2612778563052416) - perf/mfu/actor:np.float64(0.04187616741345731) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.75995635986328) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.020541559904813766 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.020541559904813766 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:446.265625 - response_length/max:1848.0 - response_length/min:58.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:446.265625 - response_length_non_aborted/max:1848.0 - response_length_non_aborted/min:58.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.460374712944031e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.705725435167551) - timing_s/agent_loop/generate_sequences/max:np.float64(11.959222553297877) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.448366878685192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.959222553297877) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1848 - timing_s/gen:17.897763485088944 - timing_s/reward:0.06247272342443466 - timing_s/old_log_prob:2.423390854150057 - timing_s/adv:0.5865025147795677 - timing_s/update_actor:16.00029367208481 - timing_s/step:37.05911082029343 - timing_s/stop_profile:0.00011603906750679016 - timing_per_token_ms/update_actor:0.08702811866112312 - timing_per_token_ms/adv:0.0031900796008722653 - timing_per_token_ms/gen:0.1566626123480353 - perf/total_num_tokens:183852 - perf/time_per_step:37.05911082029343 - perf/throughput:620.130906848834 (TaskRunner pid=2074004) Training Progress: 58%|█████▊ | 58/100 [49:13<28:30, 40.72s/it] (TaskRunner pid=2074004) step:59 - global_seqlen/min:16270 - global_seqlen/max:25747 - global_seqlen/minmax_diff:9477 - global_seqlen/balanced_min:21707 - global_seqlen/balanced_max:21791 - global_seqlen/mean:21721.5 - actor/entropy:0.10231564939022064 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46255624294281006 - training/rollout_probs_diff_mean:0.0026242148596793413 - training/rollout_probs_diff_std:0.011124146170914173 - training/rollout_actor_probs_pearson_corr:0.9976997971534729 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.67578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.0001174211502075 - rollout_corr/rollout_is_ratio_fraction_high:3.8038724596844986e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.8038724596844986e-05 - rollout_corr/rollout_is_max:2.163364887237549 - rollout_corr/rollout_is_min:2.3478437469748314e-06 - rollout_corr/rollout_is_std:0.03120991960167885 - rollout_corr/rollout_is_eff_sample_size:0.9990270077403499 - rollout_corr/rollout_is_seq_mean:1.000049114227295 - rollout_corr/rollout_is_seq_std:0.002008486306294799 - rollout_corr/rollout_is_seq_max:1.0093592405319214 - rollout_corr/rollout_is_seq_min:0.9932237863540649 - rollout_corr/rollout_is_seq_max_deviation:0.009359240531921387 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1050791395828128) - rollout_corr/training_log_ppl:np.float64(0.09638163917770726) - rollout_corr/kl:np.float64(0.0006338543649420103) - rollout_corr/k3_kl:np.float64(0.0009249671898068357) - rollout_corr/rollout_ppl:np.float64(1.1043821601197124) - rollout_corr/rollout_log_ppl:np.float64(0.09574778548085305) - rollout_corr/log_ppl_diff:np.float64(0.000633853696854203) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018177191577706253) - rollout_corr/log_ppl_diff_max:np.float64(0.02475271373987198) - rollout_corr/log_ppl_diff_min:np.float64(-0.016850709915161133) - rollout_corr/ppl_ratio:np.float64(1.0006383063737303) - rollout_corr/chi2_token:np.float64(0.002892733784392476) - rollout_corr/chi2_seq:np.float64(1.0235684728249907) - actor/pg_loss:np.float64(0.0013639027019962668) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011857084368784854) - actor/ppo_kl:np.float64(5.230466129590994e-05) - actor/pg_clipfrac_lower:np.float64(1.7208149074576795e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_w_mean:np.float64(187076.23620933085) - self_distillation/token_reweight_w_std:np.float64(2489852.4811536297) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00058325775899) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037732115903054364) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.295432485640049) - perf/mfu/actor:np.float64(0.03939332486847546) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.67478942871094) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005545332562178373 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005545332562178373 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:410.765625 - response_length/max:1910.0 - response_length/min:68.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:410.765625 - response_length_non_aborted/max:1910.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.881092071533203e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5705593023449183) - timing_s/agent_loop/generate_sequences/max:np.float64(11.528048625215888) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9566591325128684) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.528048625215888) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1910 - timing_s/gen:17.374146005138755 - timing_s/reward:0.0605713315308094 - timing_s/old_log_prob:2.3631235398352146 - timing_s/adv:0.5710447505116463 - timing_s/update_actor:15.808569202199578 - timing_s/step:36.26629160158336 - timing_s/stop_profile:0.00011047534644603729 - timing_per_token_ms/update_actor:0.0909730520578665 - timing_per_token_ms/adv:0.0032861724012593872 - timing_per_token_ms/gen:0.16522258363896264 - perf/total_num_tokens:173772 - perf/time_per_step:36.26629160158336 - perf/throughput:598.9446133238407 (TaskRunner pid=2074004) Training Progress: 59%|█████▉ | 59/100 [49:49<26:55, 39.40s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substitute the known values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2074004) step:60 - global_seqlen/min:14585 - global_seqlen/max:28896 - global_seqlen/minmax_diff:14311 - global_seqlen/balanced_min:22508 - global_seqlen/balanced_max:22534 - global_seqlen/mean:22528.125 - actor/entropy:0.10657462477684021 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5907468795776367 - training/rollout_probs_diff_mean:0.0025613645557314157 - training/rollout_probs_diff_std:0.011115039698779583 - training/rollout_actor_probs_pearson_corr:0.9978196620941162 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001739263534546 - rollout_corr/rollout_is_ratio_fraction_high:3.601494609029032e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.904109901981428e-05 - rollout_corr/rollout_is_max:3.41389799118042 - rollout_corr/rollout_is_min:0.04764045029878616 - rollout_corr/rollout_is_std:0.030336376279592514 - rollout_corr/rollout_is_eff_sample_size:0.9990810264110818 - rollout_corr/rollout_is_seq_mean:1.0001338720321655 - rollout_corr/rollout_is_seq_std:0.0019507677061483264 - rollout_corr/rollout_is_seq_max:1.0104360580444336 - rollout_corr/rollout_is_seq_min:0.9874913692474365 - rollout_corr/rollout_is_seq_max_deviation:0.012508630752563477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.101980961393565) - rollout_corr/training_log_ppl:np.float64(0.09394121470904793) - rollout_corr/kl:np.float64(0.0005248432001501158) - rollout_corr/k3_kl:np.float64(0.0005477877917310359) - rollout_corr/rollout_ppl:np.float64(1.1014048550277948) - rollout_corr/rollout_log_ppl:np.float64(0.09341637074976461) - rollout_corr/log_ppl_diff:np.float64(0.0005248439592833165) - rollout_corr/log_ppl_abs_diff:np.float64(0.001438066206901567) - rollout_corr/log_ppl_diff_max:np.float64(0.013666749000549316) - rollout_corr/log_ppl_diff_min:np.float64(-0.009086042642593384) - rollout_corr/ppl_ratio:np.float64(1.0005271381232888) - rollout_corr/chi2_token:np.float64(0.0011212392710149288) - rollout_corr/chi2_seq:np.float64(0.3878027028404176) - actor/pg_loss:np.float64(0.0020158932311460376) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007685895379836438) - actor/ppo_kl:np.float64(0.0001438019309926153) - actor/pg_clipfrac_lower:np.float64(8.778089977568015e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(88144.51772208884) - self_distillation/token_reweight_w_std:np.float64(1184668.7561488296) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000381489051506) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04390312758914661) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.3155992925167084) - perf/mfu/actor:np.float64(0.04083755030532129) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.75739669799805) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62265625) - val-aux/sciknoweval/reward/std@16:np.float64(0.1723676683272905) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.69395) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14379069680642126) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5512875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1475520499816467) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6222125000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1724222276112314) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7506250000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10453211597440552) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4925750000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11041911356297991) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6338750000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13849568719279187) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7915000000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0681230254543447) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4484499999999999) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07370449353224175) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6356125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10212696045932126) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.817) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03823086848173044) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41914999999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04379924098110083) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6377375000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0775045006079097) - val-aux/sciknoweval/score/mean@16:np.float64(0.62265625) - val-aux/sciknoweval/score/std@16:np.float64(0.1723676683272905) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.69395) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14379069680642126) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5512875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1475520499816467) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6222125000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1724222276112314) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7506250000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10453211597440552) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4925750000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11041911356297991) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6338750000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13849568719279187) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7915000000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0681230254543447) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4484499999999999) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07370449353224175) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6356125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10212696045932126) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.817) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03823086848173044) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41914999999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04379924098110083) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6377375000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0775045006079097) - val-core/sciknoweval/acc/mean@16:np.float64(0.62265625) - val-aux/sciknoweval/acc/std@16:np.float64(0.1723676683272905) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.69395) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14379069680642126) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5512875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1475520499816467) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6222125000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1724222276112314) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7506250000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10453211597440552) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4925750000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11041911356297991) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6338750000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13849568719279187) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7915000000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0681230254543447) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4484499999999999) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07370449353224175) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6356125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10212696045932126) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.817) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03823086848173044) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41914999999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04379924098110083) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6377375000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0775045006079097) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.007159754662762969) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.002242736757610738) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9955625000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.009417988829529579) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9975125) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.007412161208789384) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.99175) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01156057508733171) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99915) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.004201813298434218) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9865625000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.012175391485948633) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999775) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0021268964117280294) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9809249999999998) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.010081889419878812) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999625) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.000683625445693766) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012774050701409578 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012774050701409578 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.84765625 - response_length/max:1465.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.84765625 - response_length_non_aborted/max:1465.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.011104702949524e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0925841629505157) - timing_s/agent_loop/generate_sequences/max:np.float64(10.3709436673671) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.332557140849531) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.3709436673671) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1465 - timing_s/gen:16.32388431020081 - timing_s/reward:0.06206193566322327 - timing_s/old_log_prob:2.4646734315901995 - timing_s/adv:0.5586496517062187 - timing_s/update_actor:15.928700476884842 - timing_s/step:35.42581527121365 - timing_s/testing:84.386498471722 - timing_s/save_checkpoint:6.650143900886178 - timing_s/stop_profile:0.00012451224029064178 - timing_per_token_ms/update_actor:0.08838230254895182 - timing_per_token_ms/adv:0.0030997345080106464 - timing_per_token_ms/gen:0.14697595381263953 - perf/total_num_tokens:180225 - perf/time_per_step:35.42581527121365 - perf/throughput:635.9239675228005 (TaskRunner pid=2074004) Training Progress: 60%|██████ | 60/100 [51:56<43:41, 65.54s/it] (TaskRunner pid=2074004) step:61 - global_seqlen/min:15258 - global_seqlen/max:27550 - global_seqlen/minmax_diff:12292 - global_seqlen/balanced_min:21784 - global_seqlen/balanced_max:21817 - global_seqlen/mean:21809.5 - actor/entropy:0.11104559153318405 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41980162262916565 - training/rollout_probs_diff_mean:0.0026704224292188883 - training/rollout_probs_diff_std:0.01082498300820589 - training/rollout_actor_probs_pearson_corr:0.9980793595314026 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:0.9999408721923828 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.5849687012378126e-05 - rollout_corr/rollout_is_max:1.967535376548767 - rollout_corr/rollout_is_min:0.2217474728822708 - rollout_corr/rollout_is_std:0.02934972383081913 - rollout_corr/rollout_is_eff_sample_size:0.9991390970476317 - rollout_corr/rollout_is_seq_mean:1.0000622272491455 - rollout_corr/rollout_is_seq_std:0.002196152927353978 - rollout_corr/rollout_is_seq_max:1.0167111158370972 - rollout_corr/rollout_is_seq_min:0.992173433303833 - rollout_corr/rollout_is_seq_max_deviation:0.016711115837097168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1188437603414059) - rollout_corr/training_log_ppl:np.float64(0.10840739371997188) - rollout_corr/kl:np.float64(0.0004275032880443064) - rollout_corr/k3_kl:np.float64(0.0005405173702293098) - rollout_corr/rollout_ppl:np.float64(1.1183525188826025) - rollout_corr/rollout_log_ppl:np.float64(0.10797988958074711) - rollout_corr/log_ppl_diff:np.float64(0.0004275041392247658) - rollout_corr/log_ppl_abs_diff:np.float64(0.001455118432204472) - rollout_corr/log_ppl_diff_max:np.float64(0.01237882673740387) - rollout_corr/log_ppl_diff_min:np.float64(-0.009973615407943726) - rollout_corr/ppl_ratio:np.float64(1.0004298684652895) - rollout_corr/chi2_token:np.float64(0.001337600639089942) - rollout_corr/chi2_seq:np.float64(0.26282138866372406) - actor/pg_loss:np.float64(0.002470805891789496) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000613119240369997) - actor/ppo_kl:np.float64(-8.526104423367542e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(115092.67977330904) - self_distillation/token_reweight_w_std:np.float64(2121252.4452306055) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004522716626525) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04705037997337058) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.32100969552993774) - perf/mfu/actor:np.float64(0.03935632243399488) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.00535583496094) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0035281332675367594 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0035281332675367594 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:425.984375 - response_length/max:1622.0 - response_length/min:47.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:425.984375 - response_length_non_aborted/max:1622.0 - response_length_non_aborted/min:47.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013445131480693817 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5521563682705164) - timing_s/agent_loop/generate_sequences/max:np.float64(10.833473060280085) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.15038654532691) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.833473060280085) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:316 - timing_s/agent_loop/slowest/response_length:1622 - timing_s/gen:16.92672893963754 - timing_s/reward:0.05643085576593876 - timing_s/old_log_prob:2.6533840224146843 - timing_s/adv:0.6161332204937935 - timing_s/update_actor:15.688221232965589 - timing_s/step:36.045780032873154 - timing_s/stop_profile:0.00011107698082923889 - timing_per_token_ms/update_actor:0.08991621330707712 - timing_per_token_ms/adv:0.003531335086165395 - timing_per_token_ms/gen:0.15521704269190426 - perf/total_num_tokens:174476 - perf/time_per_step:36.045780032873154 - perf/throughput:605.0500219473707 (TaskRunner pid=2074004) Training Progress: 61%|██████ | 61/100 [52:32<36:51, 56.71s/it] (TaskRunner pid=2074004) step:62 - global_seqlen/min:20453 - global_seqlen/max:31926 - global_seqlen/minmax_diff:11473 - global_seqlen/balanced_min:24980 - global_seqlen/balanced_max:25000 - global_seqlen/mean:24994.125 - actor/entropy:0.11476144939661026 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8106536269187927 - training/rollout_probs_diff_mean:0.0027153317350894213 - training/rollout_probs_diff_std:0.011159801855683327 - training/rollout_actor_probs_pearson_corr:0.9979556202888489 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:0.9999613165855408 - rollout_corr/rollout_is_ratio_fraction_high:7.923365046735853e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.338692037388682e-05 - rollout_corr/rollout_is_max:2.658514976501465 - rollout_corr/rollout_is_min:0.1538795679807663 - rollout_corr/rollout_is_std:0.030247831717133522 - rollout_corr/rollout_is_eff_sample_size:0.9990857860453506 - rollout_corr/rollout_is_seq_mean:0.9999266862869263 - rollout_corr/rollout_is_seq_std:0.0018450476927682757 - rollout_corr/rollout_is_seq_max:1.011099934577942 - rollout_corr/rollout_is_seq_min:0.9912953972816467 - rollout_corr/rollout_is_seq_max_deviation:0.011099934577941895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1163019719533622) - rollout_corr/training_log_ppl:np.float64(0.10734543754006154) - rollout_corr/kl:np.float64(0.0005882964252315048) - rollout_corr/k3_kl:np.float64(0.0005425430525072272) - rollout_corr/rollout_ppl:np.float64(1.1156547986902297) - rollout_corr/rollout_log_ppl:np.float64(0.1067571406965726) - rollout_corr/log_ppl_diff:np.float64(0.0005882968434889335) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013848803682776634) - rollout_corr/log_ppl_diff_max:np.float64(0.010647937655448914) - rollout_corr/log_ppl_diff_min:np.float64(-0.006409019231796265) - rollout_corr/ppl_ratio:np.float64(1.0005901597905904) - rollout_corr/chi2_token:np.float64(0.0009822722058743238) - rollout_corr/chi2_seq:np.float64(0.6805941013153642) - actor/pg_loss:np.float64(0.0017424660036340356) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005262740519356157) - actor/ppo_kl:np.float64(4.013105478151147e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(72289.93594874162) - self_distillation/token_reweight_w_std:np.float64(1211116.1148807127) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999488468747586) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04404561784758698) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.22749612480401993) - perf/mfu/actor:np.float64(0.04494797983979118) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.95013809204102) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005711755715310574 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005711755715310574 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:493.00390625 - response_length/max:1745.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:493.00390625 - response_length_non_aborted/max:1745.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.554384112358093e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6613222770392895) - timing_s/agent_loop/generate_sequences/max:np.float64(12.032618755474687) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.850435824046144) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.032618755474687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1745 - timing_s/gen:18.026200192049146 - timing_s/reward:0.06495929323136806 - timing_s/old_log_prob:2.460443489253521 - timing_s/adv:0.651430806145072 - timing_s/update_actor:16.151509700343013 - timing_s/step:37.442482167854905 - timing_s/stop_profile:0.00011717900633811951 - timing_per_token_ms/update_actor:0.0807765309864969 - timing_per_token_ms/adv:0.003257919641841193 - timing_per_token_ms/gen:0.14282816750032998 - perf/total_num_tokens:199953 - perf/time_per_step:37.442482167854905 - perf/throughput:667.533869361309 (TaskRunner pid=2074004) Training Progress: 62%|██████▏ | 62/100 [53:09<32:15, 50.94s/it] (TaskRunner pid=2074004) step:63 - global_seqlen/min:16031 - global_seqlen/max:28652 - global_seqlen/minmax_diff:12621 - global_seqlen/balanced_min:20376 - global_seqlen/balanced_max:20475 - global_seqlen/mean:20399.625 - actor/entropy:0.09949057549238205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9106009602546692 - training/rollout_probs_diff_mean:0.002759087597951293 - training/rollout_probs_diff_std:0.011734765022993088 - training/rollout_actor_probs_pearson_corr:0.9973864555358887 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.000015139579773 - rollout_corr/rollout_is_ratio_fraction_high:5.268204404274002e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.2145635234192014e-05 - rollout_corr/rollout_is_max:2.28180193901062 - rollout_corr/rollout_is_min:0.06190122291445732 - rollout_corr/rollout_is_std:0.03141549974679947 - rollout_corr/rollout_is_eff_sample_size:0.9990140393695902 - rollout_corr/rollout_is_seq_mean:0.9999146461486816 - rollout_corr/rollout_is_seq_std:0.001660522655583918 - rollout_corr/rollout_is_seq_max:1.0048866271972656 - rollout_corr/rollout_is_seq_min:0.9946311712265015 - rollout_corr/rollout_is_seq_max_deviation:0.005368828773498535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0948191625066102) - rollout_corr/training_log_ppl:np.float64(0.08782734138367232) - rollout_corr/kl:np.float64(0.000854949058012977) - rollout_corr/k3_kl:np.float64(0.0006357853782912315) - rollout_corr/rollout_ppl:np.float64(1.0938773178495467) - rollout_corr/rollout_log_ppl:np.float64(0.08697239161483594) - rollout_corr/log_ppl_diff:np.float64(0.0008549497688363772) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017329079382761847) - rollout_corr/log_ppl_diff_max:np.float64(0.008672401309013367) - rollout_corr/log_ppl_diff_min:np.float64(-0.0064188651740550995) - rollout_corr/ppl_ratio:np.float64(1.0008579187560827) - rollout_corr/chi2_token:np.float64(0.0008239620365202427) - rollout_corr/chi2_seq:np.float64(0.4398700457531959) - actor/pg_loss:np.float64(0.00171933404635638) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005747673458245117) - actor/ppo_kl:np.float64(0.00029454395632910746) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(141111.829759978) - self_distillation/token_reweight_w_std:np.float64(1970238.3386651743) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000252681784332) - self_distillation/token_reweight_w_clip_frac:np.float64(0.031212968737236224) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.273105651140213) - perf/mfu/actor:np.float64(0.037443201028578815) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.92625427246094) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005000842735171318 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005000842735171318 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:370.73828125 - response_length/max:1427.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:370.73828125 - response_length_non_aborted/max:1427.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010591745376586914 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8246997967362404) - timing_s/agent_loop/generate_sequences/max:np.float64(9.28955096192658) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.776446552736161) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.28955096192658) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1427 - timing_s/gen:15.555157018825412 - timing_s/reward:0.05943015590310097 - timing_s/old_log_prob:2.3526105862110853 - timing_s/adv:0.5264585241675377 - timing_s/update_actor:15.563456859439611 - timing_s/step:34.14370860904455 - timing_s/stop_profile:0.00012073293328285217 - timing_per_token_ms/update_actor:0.09536607204445922 - timing_per_token_ms/adv:0.0032259080998274336 - timing_per_token_ms/gen:0.1638954895618478 - perf/total_num_tokens:163197 - perf/time_per_step:34.14370860904455 - perf/throughput:597.4636567334168 (TaskRunner pid=2074004) Training Progress: 63%|██████▎ | 63/100 [53:44<28:18, 45.92s/it] (TaskRunner pid=2074004) step:64 - global_seqlen/min:17389 - global_seqlen/max:31739 - global_seqlen/minmax_diff:14350 - global_seqlen/balanced_min:23753 - global_seqlen/balanced_max:30196 - global_seqlen/mean:24563.75 - actor/entropy:0.11078858375549316 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27813637256622314 - training/rollout_probs_diff_mean:0.0025074740406125784 - training/rollout_probs_diff_std:0.010136975906789303 - training/rollout_actor_probs_pearson_corr:0.9982666969299316 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998918771743774 - rollout_corr/rollout_is_ratio_fraction_high:7.794596967869438e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.897298302035779e-05 - rollout_corr/rollout_is_max:2.2562098503112793 - rollout_corr/rollout_is_min:0.38679587841033936 - rollout_corr/rollout_is_std:0.02878168411552906 - rollout_corr/rollout_is_eff_sample_size:0.9991719432652497 - rollout_corr/rollout_is_seq_mean:0.9999525547027588 - rollout_corr/rollout_is_seq_std:0.0016079578781500459 - rollout_corr/rollout_is_seq_max:1.0066720247268677 - rollout_corr/rollout_is_seq_min:0.9929185509681702 - rollout_corr/rollout_is_seq_max_deviation:0.007081449031829834 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1173579916357994) - rollout_corr/training_log_ppl:np.float64(0.10709392841090448) - rollout_corr/kl:np.float64(0.0006130551650116445) - rollout_corr/k3_kl:np.float64(0.0006270227015860996) - rollout_corr/rollout_ppl:np.float64(1.1166537292301655) - rollout_corr/rollout_log_ppl:np.float64(0.1064808721675945) - rollout_corr/log_ppl_diff:np.float64(0.000613056243309984) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015000321327534039) - rollout_corr/log_ppl_diff_max:np.float64(0.00872485339641571) - rollout_corr/log_ppl_diff_min:np.float64(-0.005040250718593597) - rollout_corr/ppl_ratio:np.float64(1.0006150556728244) - rollout_corr/chi2_token:np.float64(0.0012433729134500027) - rollout_corr/chi2_seq:np.float64(1.6464725509285927) - actor/pg_loss:np.float64(0.0018512200331315398) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047728462982377096) - actor/ppo_kl:np.float64(0.00010965376971583396) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(103190.21550233453) - self_distillation/token_reweight_w_std:np.float64(1658807.6000547835) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004303914029151) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04449158857460134) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.2668180838227272) - perf/mfu/actor:np.float64(0.0422181008652012) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.02878189086914) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009036860428750515 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009036860428750515 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:501.1484375 - response_length/max:8192.0 - response_length/min:107.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:501.1484375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013284198939800262 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1968379076570272) - timing_s/agent_loop/generate_sequences/max:np.float64(51.116004686802626) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.855164358850743) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.116004686802626) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:307 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.01408413052559 - timing_s/reward:0.06438678875565529 - timing_s/old_log_prob:2.5394509602338076 - timing_s/adv:0.595608577132225 - timing_s/update_actor:16.805029960349202 - timing_s/step:77.1059439778328 - timing_s/stop_profile:0.00012509338557720184 - timing_per_token_ms/update_actor:0.08551742893669127 - timing_per_token_ms/adv:0.0030309326605883927 - timing_per_token_ms/gen:0.4444017968924937 - perf/total_num_tokens:196510 - perf/time_per_step:77.1059439778328 - perf/throughput:318.57141917699414 (TaskRunner pid=2074004) Training Progress: 64%|██████▍ | 64/100 [55:01<33:10, 55.29s/it] (TaskRunner pid=2074004) step:65 - global_seqlen/min:12881 - global_seqlen/max:28547 - global_seqlen/minmax_diff:15666 - global_seqlen/balanced_min:20761 - global_seqlen/balanced_max:20766 - global_seqlen/mean:20763.75 - actor/entropy:0.10208147764205933 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43991219997406006 - training/rollout_probs_diff_mean:0.00263621611520648 - training/rollout_probs_diff_std:0.01091404352337122 - training/rollout_actor_probs_pearson_corr:0.9977887272834778 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.83984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999848008155823 - rollout_corr/rollout_is_ratio_fraction_high:1.0132328498002607e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.052931399201043e-05 - rollout_corr/rollout_is_max:2.1628031730651855 - rollout_corr/rollout_is_min:0.17808698117733002 - rollout_corr/rollout_is_std:0.030277375131845474 - rollout_corr/rollout_is_eff_sample_size:0.9990840011771853 - rollout_corr/rollout_is_seq_mean:0.9999912977218628 - rollout_corr/rollout_is_seq_std:0.0019186957506462932 - rollout_corr/rollout_is_seq_max:1.0081483125686646 - rollout_corr/rollout_is_seq_min:0.9939520359039307 - rollout_corr/rollout_is_seq_max_deviation:0.00814831256866455 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0915365894325078) - rollout_corr/training_log_ppl:np.float64(0.08563249993312638) - rollout_corr/kl:np.float64(0.0005638776894053565) - rollout_corr/k3_kl:np.float64(0.0005349675133992093) - rollout_corr/rollout_ppl:np.float64(1.0908976537175477) - rollout_corr/rollout_log_ppl:np.float64(0.08506862222202471) - rollout_corr/log_ppl_diff:np.float64(0.000563877711101668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015504594302910846) - rollout_corr/log_ppl_diff_max:np.float64(0.007803872227668762) - rollout_corr/log_ppl_diff_min:np.float64(-0.00601176917552948) - rollout_corr/ppl_ratio:np.float64(1.0005660941824317) - rollout_corr/chi2_token:np.float64(0.0010084726382046938) - rollout_corr/chi2_seq:np.float64(0.3784930878318846) - actor/pg_loss:np.float64(0.0010833193082362413) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028873655492134276) - actor/ppo_kl:np.float64(7.600492999415565e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_w_mean:np.float64(153738.86696914816) - self_distillation/token_reweight_w_std:np.float64(1896279.4650243353) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996677823364735) - self_distillation/token_reweight_w_clip_frac:np.float64(0.026835397191462107) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.22542214393615723) - perf/mfu/actor:np.float64(0.038259118996520076) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.00650787353516) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010626278817653656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010626278817653656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:385.5234375 - response_length/max:1258.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:385.5234375 - response_length_non_aborted/max:1258.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013937614858150482 - timing_s/agent_loop/generate_sequences/min:np.float64(1.065683364868164) - timing_s/agent_loop/generate_sequences/max:np.float64(8.783862510696054) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.777955053730693) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.783862510696054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1258 - timing_s/gen:14.71335643529892 - timing_s/reward:0.06031355261802673 - timing_s/old_log_prob:2.3282906115055084 - timing_s/adv:0.5691004022955894 - timing_s/update_actor:15.625853784382343 - timing_s/step:33.38593070022762 - timing_s/stop_profile:0.00011672265827655792 - timing_per_token_ms/update_actor:0.09406931421577475 - timing_per_token_ms/adv:0.003426045405427665 - timing_per_token_ms/gen:0.1490805564198322 - perf/total_num_tokens:166110 - perf/time_per_step:33.38593070022762 - perf/throughput:621.9311417865741 (TaskRunner pid=2074004) Training Progress: 65%|██████▌ | 65/100 [55:34<28:25, 48.73s/it] (TaskRunner pid=2074004) step:66 - global_seqlen/min:18568 - global_seqlen/max:30571 - global_seqlen/minmax_diff:12003 - global_seqlen/balanced_min:23556 - global_seqlen/balanced_max:23573 - global_seqlen/mean:23566.625 - actor/entropy:0.10450827330350876 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797508478164673 - training/rollout_probs_diff_mean:0.0026548842433840036 - training/rollout_probs_diff_std:0.010791671462357044 - training/rollout_actor_probs_pearson_corr:0.9979157447814941 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.0000296831130981 - rollout_corr/rollout_is_ratio_fraction_high:2.5662740881671198e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.698821718804538e-05 - rollout_corr/rollout_is_max:3.0360376834869385 - rollout_corr/rollout_is_min:0.10401836037635803 - rollout_corr/rollout_is_std:0.030111556872725487 - rollout_corr/rollout_is_eff_sample_size:0.9990941155144449 - rollout_corr/rollout_is_seq_mean:1.000089406967163 - rollout_corr/rollout_is_seq_std:0.0016775390831753612 - rollout_corr/rollout_is_seq_max:1.008592128753662 - rollout_corr/rollout_is_seq_min:0.9943743348121643 - rollout_corr/rollout_is_seq_max_deviation:0.00859212875366211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1033647037111223) - rollout_corr/training_log_ppl:np.float64(0.09674622296006419) - rollout_corr/kl:np.float64(0.0004946426893255307) - rollout_corr/k3_kl:np.float64(0.00056559361085462) - rollout_corr/rollout_ppl:np.float64(1.1028108289465308) - rollout_corr/rollout_log_ppl:np.float64(0.09625157990376465) - rollout_corr/log_ppl_diff:np.float64(0.0004946430562995374) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014105098089203238) - rollout_corr/log_ppl_diff_max:np.float64(0.006463304162025452) - rollout_corr/log_ppl_diff_min:np.float64(-0.007465112954378128) - rollout_corr/ppl_ratio:np.float64(1.000496445223689) - rollout_corr/chi2_token:np.float64(0.001295700902119279) - rollout_corr/chi2_seq:np.float64(1.1554163373075426) - actor/pg_loss:np.float64(0.0020622570300474763) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004437317447809619) - actor/ppo_kl:np.float64(8.988743698523649e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(60744.646061601816) - self_distillation/token_reweight_w_std:np.float64(1164193.8388803264) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999702645232901) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0351556129171513) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.26315246149897575) - perf/mfu/actor:np.float64(0.04286121414241698) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.01708602905273) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013922036159783602 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013922036159783602 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.64453125 - response_length/max:1317.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.64453125 - response_length_non_aborted/max:1317.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.762759923934937e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1173687372356653) - timing_s/agent_loop/generate_sequences/max:np.float64(9.68836759403348) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.69582801679644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.68836759403348) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:1317 - timing_s/gen:15.383818035945296 - timing_s/reward:0.06612947955727577 - timing_s/old_log_prob:2.3371682837605476 - timing_s/adv:0.568218607455492 - timing_s/update_actor:15.863027606159449 - timing_s/step:34.31379409506917 - timing_s/stop_profile:0.00011265650391578674 - timing_per_token_ms/update_actor:0.08413926265512907 - timing_per_token_ms/adv:0.0030138946892877745 - timing_per_token_ms/gen:0.13159697552583208 - perf/total_num_tokens:188533 - perf/time_per_step:34.31379409506917 - perf/throughput:686.7974125713624 (TaskRunner pid=2074004) Training Progress: 66%|██████▌ | 66/100 [56:09<25:10, 44.42s/it] (TaskRunner pid=2074004) step:67 - global_seqlen/min:13079 - global_seqlen/max:27690 - global_seqlen/minmax_diff:14611 - global_seqlen/balanced_min:20237 - global_seqlen/balanced_max:20247 - global_seqlen/mean:20243.125 - actor/entropy:0.11189419776201248 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6611925363540649 - training/rollout_probs_diff_mean:0.0028133972082287073 - training/rollout_probs_diff_std:0.011245041154325008 - training/rollout_actor_probs_pearson_corr:0.9979284405708313 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.78125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.000016450881958 - rollout_corr/rollout_is_ratio_fraction_high:2.1283614842104726e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.385084270732477e-05 - rollout_corr/rollout_is_max:3.327455997467041 - rollout_corr/rollout_is_min:0.31944146752357483 - rollout_corr/rollout_is_std:0.03032655082643032 - rollout_corr/rollout_is_eff_sample_size:0.9990811454013858 - rollout_corr/rollout_is_seq_mean:0.9999868869781494 - rollout_corr/rollout_is_seq_std:0.0020908608566969633 - rollout_corr/rollout_is_seq_max:1.0100783109664917 - rollout_corr/rollout_is_seq_min:0.9929949045181274 - rollout_corr/rollout_is_seq_max_deviation:0.0100783109664917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1118552824482322) - rollout_corr/training_log_ppl:np.float64(0.10242664221368614) - rollout_corr/kl:np.float64(0.0007174603885786723) - rollout_corr/k3_kl:np.float64(0.0005228211956200823) - rollout_corr/rollout_ppl:np.float64(1.111049032304436) - rollout_corr/rollout_log_ppl:np.float64(0.10170918147196062) - rollout_corr/log_ppl_diff:np.float64(0.0007174607417255174) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015918946191959549) - rollout_corr/log_ppl_diff_max:np.float64(0.009511686861515045) - rollout_corr/log_ppl_diff_min:np.float64(-0.008098289370536804) - rollout_corr/ppl_ratio:np.float64(1.000719947507605) - rollout_corr/chi2_token:np.float64(0.0006735411006957293) - rollout_corr/chi2_seq:np.float64(0.3616406440269202) - actor/pg_loss:np.float64(0.0008698338642716408) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023361428748103208) - actor/ppo_kl:np.float64(0.00022379015435802785) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_w_mean:np.float64(29018.363461346366) - self_distillation/token_reweight_w_std:np.float64(446767.4180369621) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993730406276882) - self_distillation/token_reweight_w_clip_frac:np.float64(0.017900060265674256) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.10916494252160192) - perf/mfu/actor:np.float64(0.03730973843414816) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.93292999267578) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006233438849449158 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:0.006233438849449158 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:367.06640625 - response_length/max:1082.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:367.06640625 - response_length_non_aborted/max:1082.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00020816735923290253 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7952778041362762) - timing_s/agent_loop/generate_sequences/max:np.float64(7.780997171998024) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.720951149596658) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.780997171998024) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:1082 - timing_s/gen:13.588720228523016 - timing_s/reward:0.05631382390856743 - timing_s/old_log_prob:2.3990508504211903 - timing_s/adv:0.5778900533914566 - timing_s/update_actor:15.670771421864629 - timing_s/step:32.40315775945783 - timing_s/stop_profile:0.00010994821786880493 - timing_per_token_ms/update_actor:0.09676600958266467 - timing_per_token_ms/adv:0.0035684340571889013 - timing_per_token_ms/gen:0.14460854354652083 - perf/total_num_tokens:161945 - perf/time_per_step:32.40315775945783 - perf/throughput:624.7269216868668 (TaskRunner pid=2074004) Training Progress: 67%|██████▋ | 67/100 [56:42<22:32, 40.98s/it] (TaskRunner pid=2074004) step:68 - global_seqlen/min:17605 - global_seqlen/max:31191 - global_seqlen/minmax_diff:13586 - global_seqlen/balanced_min:21836 - global_seqlen/balanced_max:21847 - global_seqlen/mean:21843.875 - actor/entropy:0.09757616370916367 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34953224658966064 - training/rollout_probs_diff_mean:0.002497250447049737 - training/rollout_probs_diff_std:0.010688133537769318 - training/rollout_actor_probs_pearson_corr:0.9978483319282532 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000418424606323 - rollout_corr/rollout_is_ratio_fraction_high:2.808383942465298e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.872255961643532e-05 - rollout_corr/rollout_is_max:2.1123147010803223 - rollout_corr/rollout_is_min:0.3512527644634247 - rollout_corr/rollout_is_std:0.030320653691887856 - rollout_corr/rollout_is_eff_sample_size:0.999081502372468 - rollout_corr/rollout_is_seq_mean:1.0001800060272217 - rollout_corr/rollout_is_seq_std:0.001909724436700344 - rollout_corr/rollout_is_seq_max:1.0095264911651611 - rollout_corr/rollout_is_seq_min:0.9944645166397095 - rollout_corr/rollout_is_seq_max_deviation:0.009526491165161133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0923993606120348) - rollout_corr/training_log_ppl:np.float64(0.08670492109376937) - rollout_corr/kl:np.float64(0.0004997735171485829) - rollout_corr/k3_kl:np.float64(0.0006032356435454744) - rollout_corr/rollout_ppl:np.float64(1.0918437647633255) - rollout_corr/rollout_log_ppl:np.float64(0.08620514698122861) - rollout_corr/log_ppl_diff:np.float64(0.0004997741125407629) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014279280803748406) - rollout_corr/log_ppl_diff_max:np.float64(0.0074855126440525055) - rollout_corr/log_ppl_diff_min:np.float64(-0.011258374899625778) - rollout_corr/ppl_ratio:np.float64(1.0005019064992666) - rollout_corr/chi2_token:np.float64(0.0018328602891415358) - rollout_corr/chi2_seq:np.float64(0.43353482661768794) - actor/pg_loss:np.float64(0.0012217597104609013) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019400951759962481) - actor/ppo_kl:np.float64(0.0001972818646009955) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(60185.39716754225) - self_distillation/token_reweight_w_std:np.float64(943506.6543101566) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000661604572088) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01850332456524484) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.25972285121679306) - perf/mfu/actor:np.float64(0.03952048152380881) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.89424896240234) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004791805054992437 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.004791805054992437 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:417.27734375 - response_length/max:1557.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.27734375 - response_length_non_aborted/max:1557.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.857063949108124e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8643373139202595) - timing_s/agent_loop/generate_sequences/max:np.float64(10.241223026067019) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.15653961370117) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.241223026067019) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1557 - timing_s/gen:16.0516545958817 - timing_s/reward:0.05962240323424339 - timing_s/old_log_prob:2.291767431423068 - timing_s/adv:0.5920419842004776 - timing_s/update_actor:15.795362647622824 - timing_s/step:34.88568533025682 - timing_s/stop_profile:0.00011338107287883759 - timing_per_token_ms/update_actor:0.09038782409040763 - timing_per_token_ms/adv:0.0033879175752955784 - timing_per_token_ms/gen:0.15026403111578687 - perf/total_num_tokens:174751 - perf/time_per_step:34.88568533025682 - perf/throughput:626.1558227452828 (TaskRunner pid=2074004) Training Progress: 68%|██████▊ | 68/100 [57:16<20:53, 39.17s/it] (TaskRunner pid=2074004) step:69 - global_seqlen/min:18936 - global_seqlen/max:25688 - global_seqlen/minmax_diff:6752 - global_seqlen/balanced_min:22204 - global_seqlen/balanced_max:22257 - global_seqlen/mean:22220.75 - actor/entropy:0.10731451213359833 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4198373854160309 - training/rollout_probs_diff_mean:0.0025957662146538496 - training/rollout_probs_diff_std:0.010541104711592197 - training/rollout_actor_probs_pearson_corr:0.9981012940406799 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.000120997428894 - rollout_corr/rollout_is_ratio_fraction_high:1.8439637642586604e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.687927528517321e-05 - rollout_corr/rollout_is_max:24.342180252075195 - rollout_corr/rollout_is_min:0.34072938561439514 - rollout_corr/rollout_is_std:0.02951376512646675 - rollout_corr/rollout_is_eff_sample_size:0.9991294578066934 - rollout_corr/rollout_is_seq_mean:1.0000665187835693 - rollout_corr/rollout_is_seq_std:0.0023733978159725666 - rollout_corr/rollout_is_seq_max:1.0234099626541138 - rollout_corr/rollout_is_seq_min:0.9945483207702637 - rollout_corr/rollout_is_seq_max_deviation:0.02340996265411377 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1120971874333918) - rollout_corr/training_log_ppl:np.float64(0.1024366507299419) - rollout_corr/kl:np.float64(0.00044801794052590216) - rollout_corr/k3_kl:np.float64(0.0006100563592355002) - rollout_corr/rollout_ppl:np.float64(1.111582346726209) - rollout_corr/rollout_log_ppl:np.float64(0.10198863269033609) - rollout_corr/log_ppl_diff:np.float64(0.0004480180396058131) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014786184001422953) - rollout_corr/log_ppl_diff_max:np.float64(0.005983397364616394) - rollout_corr/log_ppl_diff_min:np.float64(-0.010212674736976624) - rollout_corr/ppl_ratio:np.float64(1.0004499880596995) - rollout_corr/chi2_token:np.float64(0.0031010701786726713) - rollout_corr/chi2_seq:np.float64(0.385674498276785) - actor/pg_loss:np.float64(0.001082204282283783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022557121974386973) - actor/ppo_kl:np.float64(-3.220176354723492e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(74030.12377202115) - self_distillation/token_reweight_w_std:np.float64(1098970.7254105331) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995012604631484) - self_distillation/token_reweight_w_clip_frac:np.float64(0.028009024827042595) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.1886149998754263) - perf/mfu/actor:np.float64(0.04061615996018782) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.03355026245117) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00016365178453270346 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00016365178453270346 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.6796875 - response_length/max:1407.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.6796875 - response_length_non_aborted/max:1407.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001237001270055771 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7061853520572186) - timing_s/agent_loop/generate_sequences/max:np.float64(9.594152119010687) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.148857997002779) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.594152119010687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:1407 - timing_s/gen:15.759770501405 - timing_s/reward:0.05864102765917778 - timing_s/old_log_prob:2.375713935121894 - timing_s/adv:0.5674699656665325 - timing_s/update_actor:15.860491406172514 - timing_s/step:34.719941690564156 - timing_s/stop_profile:0.00012359023094177246 - timing_per_token_ms/update_actor:0.08922117506256828 - timing_per_token_ms/adv:0.0031922300421145353 - timing_per_token_ms/gen:0.14530223028715128 - perf/total_num_tokens:177766 - perf/time_per_step:34.719941690564156 - perf/throughput:639.9996347355311 (TaskRunner pid=2074004) Training Progress: 69%|██████▉ | 69/100 [57:51<19:33, 37.85s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are to find the magnitude of the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Rearranging for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2074385) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/generation_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: 123c1826-04a8-41a1-bd46-2674c1e317a5)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/generation_config.json (WorkerDict pid=2074385) Retrying in 1s [Retry 1/5]. (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2074004) step:70 - global_seqlen/min:16730 - global_seqlen/max:27137 - global_seqlen/minmax_diff:10407 - global_seqlen/balanced_min:21643 - global_seqlen/balanced_max:21648 - global_seqlen/mean:21645.5 - actor/entropy:0.11388496309518814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830116271972656 - training/rollout_probs_diff_mean:0.0028158947825431824 - training/rollout_probs_diff_std:0.011242561973631382 - training/rollout_actor_probs_pearson_corr:0.9978851675987244 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0001726150512695 - rollout_corr/rollout_is_ratio_fraction_high:1.9112419977318496e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.556209988659248e-05 - rollout_corr/rollout_is_max:2.3941094875335693 - rollout_corr/rollout_is_min:0.20454047620296478 - rollout_corr/rollout_is_std:0.03099912963807583 - rollout_corr/rollout_is_eff_sample_size:0.9990402144088839 - rollout_corr/rollout_is_seq_mean:1.0003561973571777 - rollout_corr/rollout_is_seq_std:0.0019966252148151398 - rollout_corr/rollout_is_seq_max:1.0088205337524414 - rollout_corr/rollout_is_seq_min:0.9942991137504578 - rollout_corr/rollout_is_seq_max_deviation:0.008820533752441406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1065151896327734) - rollout_corr/training_log_ppl:np.float64(0.09889099766587606) - rollout_corr/kl:np.float64(-3.2485000064363945e-05) - rollout_corr/k3_kl:np.float64(0.0005556557533132889) - rollout_corr/rollout_ppl:np.float64(1.106540143955499) - rollout_corr/rollout_log_ppl:np.float64(0.09892348164066789) - rollout_corr/log_ppl_diff:np.float64(-3.2483974791830406e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015090550186869223) - rollout_corr/log_ppl_diff_max:np.float64(0.014128297567367554) - rollout_corr/log_ppl_diff_min:np.float64(-0.008920878171920776) - rollout_corr/ppl_ratio:np.float64(0.999969924101606) - rollout_corr/chi2_token:np.float64(0.0022924926597625017) - rollout_corr/chi2_seq:np.float64(0.9184615898411721) - actor/pg_loss:np.float64(0.00168591586407274) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006731084324655967) - actor/ppo_kl:np.float64(-0.00014355223837636544) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(99180.82018345594) - self_distillation/token_reweight_w_std:np.float64(1940147.8677054113) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003361816052347) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04243566015793476) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.27422183752059937) - perf/mfu/actor:np.float64(0.03944531836232677) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.0858383178711) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.64375) - val-aux/sciknoweval/reward/std@16:np.float64(0.17544167945053135) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7129875) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1322752335653467) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5741999999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1624310913125313) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6450374999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17432640534203808) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7630750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0897566649865629) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.510825) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13892894608045508) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6609999999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1328026930026406) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7967500000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05859835503578027) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45375000000000004) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.1090491129202408) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.666575) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09131931477917679) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8197624999999998) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03386172398893218) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40679999999999994) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07785458149403679) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6700250000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06801004833603) - val-aux/sciknoweval/score/mean@16:np.float64(0.64375) - val-aux/sciknoweval/score/std@16:np.float64(0.17544167945053135) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7129875) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1322752335653467) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5741999999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1624310913125313) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6450374999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17432640534203808) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7630750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0897566649865629) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.510825) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13892894608045508) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6609999999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1328026930026406) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7967500000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05859835503578027) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45375000000000004) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.1090491129202408) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.666575) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09131931477917679) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8197624999999998) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03386172398893218) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40679999999999994) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07785458149403679) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6700250000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06801004833603) - val-core/sciknoweval/acc/mean@16:np.float64(0.64375) - val-aux/sciknoweval/acc/std@16:np.float64(0.17544167945053135) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7129875) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1322752335653467) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5741999999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1624310913125313) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6450374999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17432640534203808) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7630750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0897566649865629) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.510825) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13892894608045508) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6609999999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1328026930026406) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7967500000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05859835503578027) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45375000000000004) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.1090491129202408) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.666575) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09131931477917679) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8197624999999998) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03386172398893218) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40679999999999994) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07785458149403679) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6700250000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06801004833603) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99296875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.023033514121596016) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9992374999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.006965874281756439) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9869375) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.030132003555088865) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9933) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.022640924934477863) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9765375000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.037335609975106154) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.997975) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.012752016030515461) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9597750000000002) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.04184278385125003) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996875000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0044696795477923035) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9392125) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03812973334861979) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0003950870157319778) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011775638908147812 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011775638908147812 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:408.765625 - response_length/max:1282.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:408.765625 - response_length_non_aborted/max:1282.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017072074115276337 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7840455882251263) - timing_s/agent_loop/generate_sequences/max:np.float64(8.811903426423669) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.002542052578065) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.811903426423669) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:350 - timing_s/agent_loop/slowest/response_length:1198 - timing_s/gen:14.664157785475254 - timing_s/reward:0.06123044900596142 - timing_s/old_log_prob:2.381863748654723 - timing_s/adv:0.5689894556999207 - timing_s/update_actor:15.963874887675047 - timing_s/step:33.727847304195166 - timing_s/testing:91.3584210369736 - timing_s/save_checkpoint:7.738546384498477 - timing_s/stop_profile:0.00014872848987579346 - timing_per_token_ms/update_actor:0.09218934009190737 - timing_per_token_ms/adv:0.003285841489570122 - timing_per_token_ms/gen:0.1401337657722875 - perf/total_num_tokens:173164 - perf/time_per_step:33.727847304195166 - perf/throughput:641.7693902838463 (TaskRunner pid=2074004) Training Progress: 70%|███████ | 70/100 [1:00:04<33:10, 66.36s/it] (TaskRunner pid=2074004) step:71 - global_seqlen/min:17341 - global_seqlen/max:23612 - global_seqlen/minmax_diff:6271 - global_seqlen/balanced_min:20701 - global_seqlen/balanced_max:20715 - global_seqlen/mean:20711.25 - actor/entropy:0.11080419272184372 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.531029462814331 - training/rollout_probs_diff_mean:0.002882769564166665 - training/rollout_probs_diff_std:0.011653199791908264 - training/rollout_actor_probs_pearson_corr:0.9976783990859985 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000351667404175 - rollout_corr/rollout_is_ratio_fraction_high:1.0566133823886048e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.452907059108838e-05 - rollout_corr/rollout_is_max:2.691452741622925 - rollout_corr/rollout_is_min:0.31489646434783936 - rollout_corr/rollout_is_std:0.031141091138124466 - rollout_corr/rollout_is_eff_sample_size:0.9990312909459407 - rollout_corr/rollout_is_seq_mean:1.0000897645950317 - rollout_corr/rollout_is_seq_std:0.0019137562485411763 - rollout_corr/rollout_is_seq_max:1.0093891620635986 - rollout_corr/rollout_is_seq_min:0.9933313727378845 - rollout_corr/rollout_is_seq_max_deviation:0.009389162063598633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1135450787842274) - rollout_corr/training_log_ppl:np.float64(0.1041237109056965) - rollout_corr/kl:np.float64(0.0005952388999048708) - rollout_corr/k3_kl:np.float64(0.0005321949834606876) - rollout_corr/rollout_ppl:np.float64(1.1128493160940707) - rollout_corr/rollout_log_ppl:np.float64(0.10352847058675252) - rollout_corr/log_ppl_diff:np.float64(0.0005952403189439792) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014877674730087165) - rollout_corr/log_ppl_diff_max:np.float64(0.00793486088514328) - rollout_corr/log_ppl_diff_min:np.float64(-0.004785224795341492) - rollout_corr/ppl_ratio:np.float64(1.0005972273647785) - rollout_corr/chi2_token:np.float64(0.0009104576893150806) - rollout_corr/chi2_seq:np.float64(0.7369661091361195) - actor/pg_loss:np.float64(0.0014374544844031334) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003361999638400448) - actor/ppo_kl:np.float64(0.00016070706497472997) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(28669.062551695388) - self_distillation/token_reweight_w_std:np.float64(576396.9004883079) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999415366910398) - self_distillation/token_reweight_w_clip_frac:np.float64(0.028172490041470155) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.14137845113873482) - perf/mfu/actor:np.float64(0.037758217715448655) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.2748680114746) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003997960593551397 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003997960593551397 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:369.6953125 - response_length/max:1046.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.6953125 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014478527009487152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.12398418225348) - timing_s/agent_loop/generate_sequences/max:np.float64(7.80044724419713) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.838224976650963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.80044724419713) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:310 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:13.64636124484241 - timing_s/reward:0.05765148997306824 - timing_s/old_log_prob:2.378165924921632 - timing_s/adv:0.5682110711932182 - timing_s/update_actor:15.703643430024385 - timing_s/step:32.44597504287958 - timing_s/stop_profile:0.00010780245065689087 - timing_per_token_ms/update_actor:0.09477725529618194 - timing_per_token_ms/adv:0.0034293624913586712 - timing_per_token_ms/gen:0.14418927373515364 - perf/total_num_tokens:165690 - perf/time_per_step:32.44597504287958 - perf/throughput:638.3303313470673 (TaskRunner pid=2074004) Training Progress: 71%|███████ | 71/100 [1:00:37<27:09, 56.20s/it] (TaskRunner pid=2074004) step:72 - global_seqlen/min:18266 - global_seqlen/max:27074 - global_seqlen/minmax_diff:8808 - global_seqlen/balanced_min:22610 - global_seqlen/balanced_max:22942 - global_seqlen/mean:22655.75 - actor/entropy:0.09876604378223419 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5981020331382751 - training/rollout_probs_diff_mean:0.0024195632431656122 - training/rollout_probs_diff_std:0.010534233413636684 - training/rollout_actor_probs_pearson_corr:0.9980013966560364 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000324249267578 - rollout_corr/rollout_is_ratio_fraction_high:2.6222400265396573e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.370399983599782e-05 - rollout_corr/rollout_is_max:2.0842576026916504 - rollout_corr/rollout_is_min:0.0317724235355854 - rollout_corr/rollout_is_std:0.02847144566476345 - rollout_corr/rollout_is_eff_sample_size:0.9991900334087397 - rollout_corr/rollout_is_seq_mean:1.0000596046447754 - rollout_corr/rollout_is_seq_std:0.0016953593585640192 - rollout_corr/rollout_is_seq_max:1.010093092918396 - rollout_corr/rollout_is_seq_min:0.9925937652587891 - rollout_corr/rollout_is_seq_max_deviation:0.010093092918395996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1005978477187455) - rollout_corr/training_log_ppl:np.float64(0.09388212007979746) - rollout_corr/kl:np.float64(0.000452447483150209) - rollout_corr/k3_kl:np.float64(0.0004727034649363304) - rollout_corr/rollout_ppl:np.float64(1.1000874377787113) - rollout_corr/rollout_log_ppl:np.float64(0.09342967234624666) - rollout_corr/log_ppl_diff:np.float64(0.00045244773355079815) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013087687766528688) - rollout_corr/log_ppl_diff_max:np.float64(0.008923083543777466) - rollout_corr/log_ppl_diff_min:np.float64(-0.006535425782203674) - rollout_corr/ppl_ratio:np.float64(1.0004541566595435) - rollout_corr/chi2_token:np.float64(0.0009954054839909077) - rollout_corr/chi2_seq:np.float64(0.5610992929432541) - actor/pg_loss:np.float64(0.0011489454191178083) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003143757853649731) - actor/ppo_kl:np.float64(5.7555564531242e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(69880.04204755742) - self_distillation/token_reweight_w_std:np.float64(1271668.7369876243) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998061931692064) - self_distillation/token_reweight_w_clip_frac:np.float64(0.025518682843539864) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.18177369236946106) - perf/mfu/actor:np.float64(0.04106344764567454) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.01401233673096) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008316871710121632 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008316871710121632 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:446.8984375 - response_length/max:1865.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:446.8984375 - response_length_non_aborted/max:1865.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010162405669689178 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4154884051531553) - timing_s/agent_loop/generate_sequences/max:np.float64(12.022635696455836) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.678455237481103) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.022635696455836) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:1865 - timing_s/gen:17.75904706120491 - timing_s/reward:0.0738532543182373 - timing_s/old_log_prob:2.3650817554444075 - timing_s/adv:0.630233459174633 - timing_s/update_actor:15.826980471611023 - timing_s/step:36.74509644508362 - timing_s/stop_profile:0.0001101028174161911 - timing_per_token_ms/update_actor:0.08732319870017007 - timing_per_token_ms/adv:0.0034772268583838153 - timing_per_token_ms/gen:0.15522828401661548 - perf/total_num_tokens:181246 - perf/time_per_step:36.74509644508362 - perf/throughput:616.5652615406666 (TaskRunner pid=2074004) Training Progress: 72%|███████▏ | 72/100 [1:01:13<23:30, 50.38s/it] (TaskRunner pid=2074004) step:73 - global_seqlen/min:16861 - global_seqlen/max:32908 - global_seqlen/minmax_diff:16047 - global_seqlen/balanced_min:26274 - global_seqlen/balanced_max:26303 - global_seqlen/mean:26293.375 - actor/entropy:0.11385305970907211 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5037192106246948 - training/rollout_probs_diff_mean:0.0026417120825499296 - training/rollout_probs_diff_std:0.011010304093360901 - training/rollout_actor_probs_pearson_corr:0.9979777932167053 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999186396598816 - rollout_corr/rollout_is_ratio_fraction_high:2.1509231373784132e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.603692549513653e-05 - rollout_corr/rollout_is_max:2.2957048416137695 - rollout_corr/rollout_is_min:0.3969012498855591 - rollout_corr/rollout_is_std:0.03009967692196369 - rollout_corr/rollout_is_eff_sample_size:0.9990945914883025 - rollout_corr/rollout_is_seq_mean:0.9999231696128845 - rollout_corr/rollout_is_seq_std:0.0021570948883891106 - rollout_corr/rollout_is_seq_max:1.0173187255859375 - rollout_corr/rollout_is_seq_min:0.9907941818237305 - rollout_corr/rollout_is_seq_max_deviation:0.0173187255859375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.119977319613099) - rollout_corr/training_log_ppl:np.float64(0.11065010864695068) - rollout_corr/kl:np.float64(0.0006033849298385796) - rollout_corr/k3_kl:np.float64(0.0006128351814460586) - rollout_corr/rollout_ppl:np.float64(1.1192855625413358) - rollout_corr/rollout_log_ppl:np.float64(0.11004672372655477) - rollout_corr/log_ppl_diff:np.float64(0.0006033849203959107) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016537990886718035) - rollout_corr/log_ppl_diff_max:np.float64(0.010391712188720703) - rollout_corr/log_ppl_diff_min:np.float64(-0.010457545518875122) - rollout_corr/ppl_ratio:np.float64(1.0006065221969038) - rollout_corr/chi2_token:np.float64(0.0012946291826665401) - rollout_corr/chi2_seq:np.float64(1.4108337932266295) - actor/pg_loss:np.float64(0.0022633997723460197) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048230918173430837) - actor/ppo_kl:np.float64(-3.4695178307231345e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(100532.0445677184) - self_distillation/token_reweight_w_std:np.float64(1938239.1147645125) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999280977062881) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04582360343192704) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.24655815586447716) - perf/mfu/actor:np.float64(0.04701854823971872) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.03591918945312) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002752285450696945 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002752285450696945 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:544.82421875 - response_length/max:1831.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:544.82421875 - response_length_non_aborted/max:1831.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011086836457252502 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8580625150352716) - timing_s/agent_loop/generate_sequences/max:np.float64(13.18264446593821) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.298752498922113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.18264446593821) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1831 - timing_s/gen:19.2261294554919 - timing_s/reward:0.06619958020746708 - timing_s/old_log_prob:2.412239581346512 - timing_s/adv:0.6206546351313591 - timing_s/update_actor:16.13218855857849 - timing_s/step:38.54634447954595 - timing_s/stop_profile:0.0001186523586511612 - timing_per_token_ms/update_actor:0.07669321910261849 - timing_per_token_ms/adv:0.002950622709767 - timing_per_token_ms/gen:0.13784642018635526 - perf/total_num_tokens:210347 - perf/time_per_step:38.54634447954595 - perf/throughput:682.1236969423181 (TaskRunner pid=2074004) Training Progress: 73%|███████▎ | 73/100 [1:01:52<21:04, 46.84s/it] (TaskRunner pid=2074004) step:74 - global_seqlen/min:15082 - global_seqlen/max:27595 - global_seqlen/minmax_diff:12513 - global_seqlen/balanced_min:20956 - global_seqlen/balanced_max:21007 - global_seqlen/mean:20963.375 - actor/entropy:0.1221182718873024 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4268117845058441 - training/rollout_probs_diff_mean:0.0029213139787316322 - training/rollout_probs_diff_std:0.011437395587563515 - training/rollout_actor_probs_pearson_corr:0.9979726672172546 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:0.9998773336410522 - rollout_corr/rollout_is_ratio_fraction_high:9.682135896582622e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.682135714683682e-05 - rollout_corr/rollout_is_max:2.0322699546813965 - rollout_corr/rollout_is_min:0.3633837103843689 - rollout_corr/rollout_is_std:0.03148183599114418 - rollout_corr/rollout_is_eff_sample_size:0.9990097563119252 - rollout_corr/rollout_is_seq_mean:1.0000077486038208 - rollout_corr/rollout_is_seq_std:0.0027337686624377966 - rollout_corr/rollout_is_seq_max:1.0147820711135864 - rollout_corr/rollout_is_seq_min:0.9881354570388794 - rollout_corr/rollout_is_seq_max_deviation:0.014782071113586426 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1157428310252726) - rollout_corr/training_log_ppl:np.float64(0.10688100929837674) - rollout_corr/kl:np.float64(0.0004768247579676199) - rollout_corr/k3_kl:np.float64(0.0006016286499246348) - rollout_corr/rollout_ppl:np.float64(1.1151970326900482) - rollout_corr/rollout_log_ppl:np.float64(0.10640418373441207) - rollout_corr/log_ppl_diff:np.float64(0.0004768255639646668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018265418002556544) - rollout_corr/log_ppl_diff_max:np.float64(0.014855928719043732) - rollout_corr/log_ppl_diff_min:np.float64(-0.011131703853607178) - rollout_corr/ppl_ratio:np.float64(1.0004803847987205) - rollout_corr/chi2_token:np.float64(0.0014686237554997206) - rollout_corr/chi2_seq:np.float64(0.33443050901405513) - actor/pg_loss:np.float64(0.0020010630832985044) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00046587054316660215) - actor/ppo_kl:np.float64(-7.767568073546727e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(61973.32169602462) - self_distillation/token_reweight_w_std:np.float64(1112938.32659155) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995205644518137) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03652457233692985) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.23524155095219612) - perf/mfu/actor:np.float64(0.03752256342730821) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.06950759887695) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0041500055231153965 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0041500055231153965 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:403.44921875 - response_length/max:1678.0 - response_length/min:48.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:403.44921875 - response_length_non_aborted/max:1678.0 - response_length_non_aborted/min:48.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013217143714427948 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5993990004062653) - timing_s/agent_loop/generate_sequences/max:np.float64(11.049694247543812) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.951693204406183) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.049694247543812) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:1678 - timing_s/gen:16.895056227222085 - timing_s/reward:0.058270735666155815 - timing_s/old_log_prob:2.356802634894848 - timing_s/adv:0.6194801684468985 - timing_s/update_actor:16.044949045404792 - timing_s/step:36.06812138669193 - timing_s/stop_profile:0.00010689161717891693 - timing_per_token_ms/update_actor:0.09567250648693729 - timing_per_token_ms/adv:0.003693824160272967 - timing_per_token_ms/gen:0.16358022353361235 - perf/total_num_tokens:167707 - perf/time_per_step:36.06812138669193 - perf/throughput:581.2161597009282 (TaskRunner pid=2074004) Training Progress: 74%|███████▍ | 74/100 [1:02:28<18:54, 43.62s/it] (TaskRunner pid=2074004) step:75 - global_seqlen/min:18314 - global_seqlen/max:24550 - global_seqlen/minmax_diff:6236 - global_seqlen/balanced_min:20293 - global_seqlen/balanced_max:27015 - global_seqlen/mean:21133.625 - actor/entropy:0.07867328822612762 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417161464691162 - training/rollout_probs_diff_mean:0.002312364522367716 - training/rollout_probs_diff_std:0.010702241212129593 - training/rollout_actor_probs_pearson_corr:0.9973950982093811 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.93359375 - self_distillation/correct_sample_fraction:0.875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.93359375 - rollout_corr/rollout_is_mean:0.9999236464500427 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.896926136803813e-05 - rollout_corr/rollout_is_max:1.7065232992172241 - rollout_corr/rollout_is_min:0.09278460592031479 - rollout_corr/rollout_is_std:0.027797745540738106 - rollout_corr/rollout_is_eff_sample_size:0.9992276439614955 - rollout_corr/rollout_is_seq_mean:0.999890923500061 - rollout_corr/rollout_is_seq_std:0.00215591280721128 - rollout_corr/rollout_is_seq_max:1.0128012895584106 - rollout_corr/rollout_is_seq_min:0.9893057942390442 - rollout_corr/rollout_is_seq_max_deviation:0.012801289558410645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.086495197378099) - rollout_corr/training_log_ppl:np.float64(0.0809084869997605) - rollout_corr/kl:np.float64(0.0006969666031730881) - rollout_corr/k3_kl:np.float64(0.000509651741253947) - rollout_corr/rollout_ppl:np.float64(1.085711132735014) - rollout_corr/rollout_log_ppl:np.float64(0.08021151977845875) - rollout_corr/log_ppl_diff:np.float64(0.0006969672213017475) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016032274397730362) - rollout_corr/log_ppl_diff_max:np.float64(0.012572050094604492) - rollout_corr/log_ppl_diff_min:np.float64(-0.01109679788351059) - rollout_corr/ppl_ratio:np.float64(1.0006995955482125) - rollout_corr/chi2_token:np.float64(0.0006079066079109907) - rollout_corr/chi2_seq:np.float64(0.20879352744668722) - actor/pg_loss:np.float64(0.0010695301461964846) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000338989874762774) - actor/ppo_kl:np.float64(0.00011593784117280848) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.93359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.93359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_w_mean:np.float64(98768.57021646714) - self_distillation/token_reweight_w_std:np.float64(1371672.2499703786) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993643839843571) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02514167891058605) - self_distillation/empty_target_batch:np.float64(0.06640625) - actor/grad_norm:np.float64(0.19157380517572165) - perf/mfu/actor:np.float64(0.0367450268799295) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.9773292541504) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.07149641960859299 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.07149641960859299 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:400.95703125 - response_length/max:8192.0 - response_length/min:76.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:400.95703125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013606809079647064 - timing_s/agent_loop/generate_sequences/min:np.float64(0.973297132179141) - timing_s/agent_loop/generate_sequences/max:np.float64(50.69700602442026) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.94060600863304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.69700602442026) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:261 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.43104676157236 - timing_s/reward:0.05962674133479595 - timing_s/old_log_prob:2.578733177855611 - timing_s/adv:0.56367601826787 - timing_s/update_actor:16.684449028223753 - timing_s/step:76.40252554044127 - timing_s/stop_profile:0.00010974332690238953 - timing_per_token_ms/update_actor:0.09868425925642048 - timing_per_token_ms/adv:0.003333999835971526 - timing_per_token_ms/gen:0.5497690755669771 - perf/total_num_tokens:169069 - perf/time_per_step:76.40252554044127 - perf/throughput:276.6089844610383 (TaskRunner pid=2074004) Training Progress: 75%|███████▌ | 75/100 [1:03:45<22:17, 53.48s/it] (TaskRunner pid=2074004) step:76 - global_seqlen/min:19671 - global_seqlen/max:26890 - global_seqlen/minmax_diff:7219 - global_seqlen/balanced_min:23504 - global_seqlen/balanced_max:23513 - global_seqlen/mean:23508.125 - actor/entropy:0.11660213023424149 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5008273124694824 - training/rollout_probs_diff_mean:0.00285673257894814 - training/rollout_probs_diff_std:0.011434005573391914 - training/rollout_actor_probs_pearson_corr:0.9978346228599548 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.9998456835746765 - rollout_corr/rollout_is_ratio_fraction_high:1.7103226127801463e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010261935676680878 - rollout_corr/rollout_is_max:3.2767059803009033 - rollout_corr/rollout_is_min:0.304049551486969 - rollout_corr/rollout_is_std:0.03136803209781647 - rollout_corr/rollout_is_eff_sample_size:0.9990166568117982 - rollout_corr/rollout_is_seq_mean:0.9999971985816956 - rollout_corr/rollout_is_seq_std:0.002045826055109501 - rollout_corr/rollout_is_seq_max:1.0144394636154175 - rollout_corr/rollout_is_seq_min:0.9931577444076538 - rollout_corr/rollout_is_seq_max_deviation:0.01443946361541748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.117191367316991) - rollout_corr/training_log_ppl:np.float64(0.108617360041535) - rollout_corr/kl:np.float64(0.0005786410445907109) - rollout_corr/k3_kl:np.float64(0.0006542783519307704) - rollout_corr/rollout_ppl:np.float64(1.1165225864388049) - rollout_corr/rollout_log_ppl:np.float64(0.10803871830285061) - rollout_corr/log_ppl_diff:np.float64(0.0005786417386843823) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014457525903708301) - rollout_corr/log_ppl_diff_max:np.float64(0.007953882217407227) - rollout_corr/log_ppl_diff_min:np.float64(-0.012358836829662323) - rollout_corr/ppl_ratio:np.float64(1.0005807224661112) - rollout_corr/chi2_token:np.float64(0.0018206238746643066) - rollout_corr/chi2_seq:np.float64(0.1669981717132032) - actor/pg_loss:np.float64(0.0019441124750301242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005475720254253247) - actor/ppo_kl:np.float64(3.105731396224609e-05) - actor/pg_clipfrac_lower:np.float64(1.1507097951835021e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_w_mean:np.float64(69920.34874183638) - self_distillation/token_reweight_w_std:np.float64(1212741.120844526) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996711185667664) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04925982659915462) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.2786094695329666) - perf/mfu/actor:np.float64(0.04275767300860228) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.03911590576172) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007215423509478569 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007215423509478569 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.78515625 - response_length/max:1205.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.78515625 - response_length_non_aborted/max:1205.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011574104428291321 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9071431942284107) - timing_s/agent_loop/generate_sequences/max:np.float64(9.096599321812391) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.676992036926094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.096599321812391) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1205 - timing_s/gen:15.07368259690702 - timing_s/reward:0.08101079240441322 - timing_s/old_log_prob:2.4621802624315023 - timing_s/adv:0.5551798567175865 - timing_s/update_actor:15.884726317599416 - timing_s/step:34.14583137631416 - timing_s/stop_profile:0.00012441538274288177 - timing_per_token_ms/update_actor:0.08446402210724704 - timing_per_token_ms/adv:0.0029520636839262303 - timing_per_token_ms/gen:0.1289043040005047 - perf/total_num_tokens:188065 - perf/time_per_step:34.14583137631416 - perf/throughput:688.4625165784311 (TaskRunner pid=2074004) Training Progress: 76%|███████▌ | 76/100 [1:04:19<19:04, 47.70s/it] (TaskRunner pid=2074004) step:77 - global_seqlen/min:15412 - global_seqlen/max:29304 - global_seqlen/minmax_diff:13892 - global_seqlen/balanced_min:21767 - global_seqlen/balanced_max:21798 - global_seqlen/mean:21789.125 - actor/entropy:0.0992889478802681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9991886615753174 - training/rollout_probs_diff_mean:0.00266986689530313 - training/rollout_probs_diff_std:0.011961651965975761 - training/rollout_actor_probs_pearson_corr:0.9973249435424805 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.73828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9998555779457092 - rollout_corr/rollout_is_ratio_fraction_high:2.7756959752878174e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.327088289661333e-05 - rollout_corr/rollout_is_max:4.194519519805908 - rollout_corr/rollout_is_min:6.96785764375818e-06 - rollout_corr/rollout_is_std:0.03096642531454563 - rollout_corr/rollout_is_eff_sample_size:0.999041761158671 - rollout_corr/rollout_is_seq_mean:0.9997901320457458 - rollout_corr/rollout_is_seq_std:0.002063008723780513 - rollout_corr/rollout_is_seq_max:1.0076104402542114 - rollout_corr/rollout_is_seq_min:0.991796612739563 - rollout_corr/rollout_is_seq_max_deviation:0.008203387260437012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.100688367150724) - rollout_corr/training_log_ppl:np.float64(0.09442752133691101) - rollout_corr/kl:np.float64(0.0008365866270294564) - rollout_corr/k3_kl:np.float64(0.0005724456864371064) - rollout_corr/rollout_ppl:np.float64(1.0997380977496505) - rollout_corr/rollout_log_ppl:np.float64(0.0935909333784366) - rollout_corr/log_ppl_diff:np.float64(0.0008365879584744107) - rollout_corr/log_ppl_abs_diff:np.float64(0.001732913853629725) - rollout_corr/log_ppl_diff_max:np.float64(0.011707089841365814) - rollout_corr/log_ppl_diff_min:np.float64(-0.0054268017411231995) - rollout_corr/ppl_ratio:np.float64(1.0008395314216614) - rollout_corr/chi2_token:np.float64(0.0006989024113863707) - rollout_corr/chi2_seq:np.float64(0.36190675757825375) - actor/pg_loss:np.float64(0.0014117311220616102) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031758257955516456) - actor/ppo_kl:np.float64(3.489058610828266e-05) - actor/pg_clipfrac_lower:np.float64(1.2245297511981335e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_w_mean:np.float64(78938.03803747497) - self_distillation/token_reweight_w_std:np.float64(1405159.017775763) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000561457127333) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02887801337055862) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.1942897979170084) - perf/mfu/actor:np.float64(0.039820481115549144) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.1161994934082) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002417168579995632 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002417168579995632 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:422.19140625 - response_length/max:1559.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:422.19140625 - response_length_non_aborted/max:1559.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013943202793598175 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1664074026048183) - timing_s/agent_loop/generate_sequences/max:np.float64(10.330939007923007) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.180417315277737) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.330939007923007) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:1540 - timing_s/gen:16.243871303275228 - timing_s/reward:0.05987227149307728 - timing_s/old_log_prob:2.3679722510278225 - timing_s/adv:0.5635029003024101 - timing_s/update_actor:15.628741595894098 - timing_s/step:34.95607234351337 - timing_s/stop_profile:0.00011238269507884979 - timing_per_token_ms/update_actor:0.08965907072848324 - timing_per_token_ms/adv:0.0032327072582217626 - timing_per_token_ms/gen:0.15029349564932992 - perf/total_num_tokens:174313 - perf/time_per_step:34.95607234351337 - perf/throughput:623.3287534674445 (TaskRunner pid=2074004) Training Progress: 77%|███████▋ | 77/100 [1:04:54<16:49, 43.89s/it] (TaskRunner pid=2074004) step:78 - global_seqlen/min:14364 - global_seqlen/max:26342 - global_seqlen/minmax_diff:11978 - global_seqlen/balanced_min:20665 - global_seqlen/balanced_max:20678 - global_seqlen/mean:20672.125 - actor/entropy:0.10278760641813278 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30106163024902344 - training/rollout_probs_diff_mean:0.0025821924209594727 - training/rollout_probs_diff_std:0.010455216281116009 - training/rollout_actor_probs_pearson_corr:0.9980041980743408 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:0.9999585151672363 - rollout_corr/rollout_is_ratio_fraction_high:1.0236567504762206e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0946270019048825e-05 - rollout_corr/rollout_is_max:2.61385440826416 - rollout_corr/rollout_is_min:0.18725009262561798 - rollout_corr/rollout_is_std:0.03052050806581974 - rollout_corr/rollout_is_eff_sample_size:0.9990692465112823 - rollout_corr/rollout_is_seq_mean:0.9999092817306519 - rollout_corr/rollout_is_seq_std:0.0021261873189359903 - rollout_corr/rollout_is_seq_max:1.0074591636657715 - rollout_corr/rollout_is_seq_min:0.9891470670700073 - rollout_corr/rollout_is_seq_max_deviation:0.010852932929992676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1040684799663723) - rollout_corr/training_log_ppl:np.float64(0.09624901220013271) - rollout_corr/kl:np.float64(0.0012303921190799372) - rollout_corr/k3_kl:np.float64(0.0009560748116967943) - rollout_corr/rollout_ppl:np.float64(1.1026978376321495) - rollout_corr/rollout_log_ppl:np.float64(0.09501861992612248) - rollout_corr/log_ppl_diff:np.float64(0.001230392274010228) - rollout_corr/log_ppl_abs_diff:np.float64(0.002099611494486453) - rollout_corr/log_ppl_diff_max:np.float64(0.024470895528793335) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052408576011657715) - rollout_corr/ppl_ratio:np.float64(1.0012363716959953) - rollout_corr/chi2_token:np.float64(0.0011034025810658932) - rollout_corr/chi2_seq:np.float64(0.31427349406294525) - actor/pg_loss:np.float64(0.0014356522588059306) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005964356096228585) - actor/ppo_kl:np.float64(0.0006484976534100184) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_w_mean:np.float64(93486.41973545495) - self_distillation/token_reweight_w_std:np.float64(1284691.6428641423) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996571952942759) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02805576975515578) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.22916332073509693) - perf/mfu/actor:np.float64(0.03750177780047932) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.05967712402344) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009613416157662868 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009613416157662868 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:381.59765625 - response_length/max:1424.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.59765625 - response_length_non_aborted/max:1424.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.280653178691864e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.881490908563137) - timing_s/agent_loop/generate_sequences/max:np.float64(9.613876366987824) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7271649866961525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.613876366987824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:1424 - timing_s/gen:15.345683528110385 - timing_s/reward:0.0597789641469717 - timing_s/old_log_prob:2.3274049311876297 - timing_s/adv:0.6073618121445179 - timing_s/update_actor:15.772901671007276 - timing_s/step:34.20114932395518 - timing_s/stop_profile:0.00012829527258872986 - timing_per_token_ms/update_actor:0.09537542506519815 - timing_per_token_ms/adv:0.003672589369407583 - timing_per_token_ms/gen:0.15708711859175942 - perf/total_num_tokens:165377 - perf/time_per_step:34.20114932395518 - perf/throughput:604.4277870369937 (TaskRunner pid=2074004) Training Progress: 78%|███████▊ | 78/100 [1:05:28<15:01, 41.00s/it] (TaskRunner pid=2074004) step:79 - global_seqlen/min:17456 - global_seqlen/max:34104 - global_seqlen/minmax_diff:16648 - global_seqlen/balanced_min:23010 - global_seqlen/balanced_max:23387 - global_seqlen/mean:23066.5 - actor/entropy:0.10228408128023148 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4396432042121887 - training/rollout_probs_diff_mean:0.0026395104359835386 - training/rollout_probs_diff_std:0.011051760986447334 - training/rollout_actor_probs_pearson_corr:0.9977607131004333 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.984375 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.984375 - rollout_corr/rollout_is_mean:1.000108003616333 - rollout_corr/rollout_is_ratio_fraction_high:8.729964974918403e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.364982305560261e-05 - rollout_corr/rollout_is_max:2.167071580886841 - rollout_corr/rollout_is_min:0.29309991002082825 - rollout_corr/rollout_is_std:0.030481424182653427 - rollout_corr/rollout_is_eff_sample_size:0.9990721022190584 - rollout_corr/rollout_is_seq_mean:1.0001380443572998 - rollout_corr/rollout_is_seq_std:0.0018053654348477721 - rollout_corr/rollout_is_seq_max:1.0080220699310303 - rollout_corr/rollout_is_seq_min:0.9950286149978638 - rollout_corr/rollout_is_seq_max_deviation:0.008022069931030273 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1028944421559572) - rollout_corr/training_log_ppl:np.float64(0.09561884048889624) - rollout_corr/kl:np.float64(0.000491056393927991) - rollout_corr/k3_kl:np.float64(0.0006740913643881186) - rollout_corr/rollout_ppl:np.float64(1.1023467793129385) - rollout_corr/rollout_log_ppl:np.float64(0.0951277833446511) - rollout_corr/log_ppl_diff:np.float64(0.0004910571442451328) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015971105312928557) - rollout_corr/log_ppl_diff_max:np.float64(0.012345612049102783) - rollout_corr/log_ppl_diff_min:np.float64(-0.006457008421421051) - rollout_corr/ppl_ratio:np.float64(1.00049376511015) - rollout_corr/chi2_token:np.float64(0.0030803862027823925) - rollout_corr/chi2_seq:np.float64(0.8471683100797236) - actor/pg_loss:np.float64(0.0012736148200929165) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005856991260770883) - actor/ppo_kl:np.float64(0.00015690973241966333) - actor/pg_clipfrac_lower:np.float64(1.272394092666218e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(105670.61742275464) - self_distillation/token_reweight_w_std:np.float64(1558575.511290349) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007708300836384) - self_distillation/token_reweight_w_clip_frac:np.float64(0.034760000635287724) - self_distillation/empty_target_batch:np.float64(0.015625) - actor/grad_norm:np.float64(0.2251501902937889) - perf/mfu/actor:np.float64(0.04172759803828769) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.39704513549805) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00025862519396468997 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00025862519396468997 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:447.453125 - response_length/max:1822.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:447.453125 - response_length_non_aborted/max:1822.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.048916399478912e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.363132268190384) - timing_s/agent_loop/generate_sequences/max:np.float64(12.375673668459058) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.632285126703209) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.375673668459058) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1822 - timing_s/gen:18.45864589139819 - timing_s/reward:0.39899917505681515 - timing_s/old_log_prob:2.5533297322690487 - timing_s/adv:0.6657597441226244 - timing_s/update_actor:15.879793219268322 - timing_s/step:37.99234522506595 - timing_s/stop_profile:0.00012224353849887848 - timing_per_token_ms/update_actor:0.08605441451492599 - timing_per_token_ms/adv:0.0036078281497118354 - timing_per_token_ms/gen:0.16114332761286265 - perf/total_num_tokens:184532 - perf/time_per_step:37.99234522506595 - perf/throughput:607.1354601395224 (TaskRunner pid=2074004) Training Progress: 79%|███████▉ | 79/100 [1:06:06<14:02, 40.11s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the known values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2074004) step:80 - global_seqlen/min:15487 - global_seqlen/max:29827 - global_seqlen/minmax_diff:14340 - global_seqlen/balanced_min:22537 - global_seqlen/balanced_max:22562 - global_seqlen/mean:22553.375 - actor/entropy:0.09155089408159256 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730071544647217 - training/rollout_probs_diff_mean:0.0024328790605068207 - training/rollout_probs_diff_std:0.010556120425462723 - training/rollout_actor_probs_pearson_corr:0.9977543950080872 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0000025033950806 - rollout_corr/rollout_is_ratio_fraction_high:3.5952147300122306e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.4940185034647584e-05 - rollout_corr/rollout_is_max:4.466158390045166 - rollout_corr/rollout_is_min:0.25997382402420044 - rollout_corr/rollout_is_std:0.029982615262269974 - rollout_corr/rollout_is_eff_sample_size:0.9991017311505869 - rollout_corr/rollout_is_seq_mean:0.9999626278877258 - rollout_corr/rollout_is_seq_std:0.0018385304138064384 - rollout_corr/rollout_is_seq_max:1.005773901939392 - rollout_corr/rollout_is_seq_min:0.9919787049293518 - rollout_corr/rollout_is_seq_max_deviation:0.008021295070648193 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0906855990178883) - rollout_corr/training_log_ppl:np.float64(0.08496327018656302) - rollout_corr/kl:np.float64(0.0005949745295730224) - rollout_corr/k3_kl:np.float64(0.0005148513625989892) - rollout_corr/rollout_ppl:np.float64(1.0900255064480007) - rollout_corr/rollout_log_ppl:np.float64(0.08436829397760448) - rollout_corr/log_ppl_diff:np.float64(0.000594976208958542) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013135308654455002) - rollout_corr/log_ppl_diff_max:np.float64(0.009091086685657501) - rollout_corr/log_ppl_diff_min:np.float64(-0.007834956049919128) - rollout_corr/ppl_ratio:np.float64(1.00059686251916) - rollout_corr/chi2_token:np.float64(0.0010652847122401) - rollout_corr/chi2_seq:np.float64(0.4150397318881005) - actor/pg_loss:np.float64(0.0015692964661866426) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005134487364557572) - actor/ppo_kl:np.float64(7.53170088216848e-05) - actor/pg_clipfrac_lower:np.float64(5.053363565821201e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(107108.59001243045) - self_distillation/token_reweight_w_std:np.float64(1823206.4579460123) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007828855887055) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0354066689760657) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.2679561898112297) - perf/mfu/actor:np.float64(0.04098502127101053) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.34692001342773) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6734375) - val-aux/sciknoweval/reward/std@16:np.float64(0.14740819888829457) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.73165) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1155182901181198) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6184875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13261326513354657) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.67505) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14789792369181032) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7746125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.08533694637512469) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5687374999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11152861009447176) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6805749999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1142760356947001) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8090125) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06002478260877771) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.52285) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09450955191647543) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6825875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07668878228681326) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8320874999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03566959645834318) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.48012499999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07047066140853427) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.67935) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05739226575463654) - val-aux/sciknoweval/score/mean@16:np.float64(0.6734375) - val-aux/sciknoweval/score/std@16:np.float64(0.14740819888829457) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.73165) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1155182901181198) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6184875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13261326513354657) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.67505) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14789792369181032) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7746125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.08533694637512469) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5687374999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11152861009447176) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6805749999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1142760356947001) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8090125) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06002478260877771) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.52285) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09450955191647543) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6825875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07668878228681326) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8320874999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03566959645834318) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.48012499999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07047066140853427) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.67935) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05739226575463654) - val-core/sciknoweval/acc/mean@16:np.float64(0.6734375) - val-aux/sciknoweval/acc/std@16:np.float64(0.14740819888829457) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.73165) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1155182901181198) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6184875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13261326513354657) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.67505) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14789792369181032) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7746125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.08533694637512469) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5687374999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11152861009447176) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6805749999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1142760356947001) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8090125) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06002478260877771) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.52285) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09450955191647543) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6825875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07668878228681326) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8320874999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03566959645834318) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.48012499999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07047066140853427) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.67935) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05739226575463654) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99921875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0030257682392245445) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0008816709987291178) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9984999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.00406201920231798) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9993124999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0028497532787945) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9971375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.005252365538497868) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.001615887913810856) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.995125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.006096874199128599) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9923249999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.006085382075104241) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00442548468708992 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00442548468708992 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.60546875 - response_length/max:1332.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.60546875 - response_length_non_aborted/max:1332.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.599832653999329e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1593389865010977) - timing_s/agent_loop/generate_sequences/max:np.float64(9.577507866546512) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.48384735804575) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.577507866546512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1332 - timing_s/gen:15.530941212549806 - timing_s/reward:0.05138901062309742 - timing_s/old_log_prob:2.2361939437687397 - timing_s/adv:0.5929265525192022 - timing_s/update_actor:15.585156086832285 - timing_s/step:34.08298168331385 - timing_s/testing:84.83454155549407 - timing_s/save_checkpoint:6.7697754595428705 - timing_s/stop_profile:0.00011528097093105316 - timing_per_token_ms/update_actor:0.08637928961204412 - timing_per_token_ms/adv:0.0032862407096454645 - timing_per_token_ms/gen:0.1395926730650986 - perf/total_num_tokens:180427 - perf/time_per_step:34.08298168331385 - perf/throughput:661.7195411351453 (TaskRunner pid=2074004) Training Progress: 80%|████████ | 80/100 [1:08:12<21:56, 65.81s/it] (TaskRunner pid=2074004) step:81 - global_seqlen/min:17592 - global_seqlen/max:32104 - global_seqlen/minmax_diff:14512 - global_seqlen/balanced_min:22879 - global_seqlen/balanced_max:22902 - global_seqlen/mean:22894.0 - actor/entropy:0.11737882345914841 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.0028525586239993572 - training/rollout_probs_diff_std:0.01126584131270647 - training/rollout_actor_probs_pearson_corr:0.9979852437973022 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.95703125 - self_distillation/correct_sample_fraction:0.76953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.95703125 - rollout_corr/rollout_is_mean:0.9998797178268433 - rollout_corr/rollout_is_ratio_fraction_high:8.564577001379803e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.2822885006899014e-05 - rollout_corr/rollout_is_max:2.0156476497650146 - rollout_corr/rollout_is_min:0.12281379103660583 - rollout_corr/rollout_is_std:0.03066662885248661 - rollout_corr/rollout_is_eff_sample_size:0.9990602035443582 - rollout_corr/rollout_is_seq_mean:1.000148892402649 - rollout_corr/rollout_is_seq_std:0.002365858294069767 - rollout_corr/rollout_is_seq_max:1.0138297080993652 - rollout_corr/rollout_is_seq_min:0.9926231503486633 - rollout_corr/rollout_is_seq_max_deviation:0.013829708099365234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1165403798222542) - rollout_corr/training_log_ppl:np.float64(0.10833677103437367) - rollout_corr/kl:np.float64(0.0006071828474922825) - rollout_corr/k3_kl:np.float64(0.0006022309770301604) - rollout_corr/rollout_ppl:np.float64(1.1158664338290691) - rollout_corr/rollout_log_ppl:np.float64(0.10772958788948017) - rollout_corr/log_ppl_diff:np.float64(0.0006071831448934972) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018550536478869617) - rollout_corr/log_ppl_diff_max:np.float64(0.009145308285951614) - rollout_corr/log_ppl_diff_min:np.float64(-0.012295559048652649) - rollout_corr/ppl_ratio:np.float64(1.0006107091903687) - rollout_corr/chi2_token:np.float64(0.001226232387125492) - rollout_corr/chi2_seq:np.float64(0.12261000624857843) - actor/pg_loss:np.float64(0.0018157117301598191) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004039403577280609) - actor/ppo_kl:np.float64(0.00021660448263460808) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.95703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.95703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_w_mean:np.float64(76784.49322666228) - self_distillation/token_reweight_w_std:np.float64(1652886.481738999) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002255805302411) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04571778689569328) - self_distillation/empty_target_batch:np.float64(0.04296875) - actor/grad_norm:np.float64(0.2025649379938841) - perf/mfu/actor:np.float64(0.04102161940188653) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.92991256713867) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009963815100491047 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009963815100491047 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.09375 - response_length/max:1588.0 - response_length/min:61.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.09375 - response_length_non_aborted/max:1588.0 - response_length_non_aborted/min:61.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.460662603378296e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6343090515583754) - timing_s/agent_loop/generate_sequences/max:np.float64(10.963617943227291) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.317745556931186) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.963617943227291) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1588 - timing_s/gen:16.757680144160986 - timing_s/reward:0.056961143389344215 - timing_s/old_log_prob:2.566490786150098 - timing_s/adv:0.6639319956302643 - timing_s/update_actor:15.944705080240965 - timing_s/step:36.09860094822943 - timing_s/stop_profile:0.00011157244443893433 - timing_per_token_ms/update_actor:0.08705722613043246 - timing_per_token_ms/adv:0.0036250327358165035 - timing_per_token_ms/gen:0.14352244042618179 - perf/total_num_tokens:183152 - perf/time_per_step:36.09860094822943 - perf/throughput:634.2074041271926 (TaskRunner pid=2074004) Training Progress: 81%|████████ | 81/100 [1:08:48<18:01, 56.91s/it] (TaskRunner pid=2074004) step:82 - global_seqlen/min:18726 - global_seqlen/max:28987 - global_seqlen/minmax_diff:10261 - global_seqlen/balanced_min:23632 - global_seqlen/balanced_max:23847 - global_seqlen/mean:23672.375 - actor/entropy:0.10426922142505646 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.506376326084137 - training/rollout_probs_diff_mean:0.0025403054896742105 - training/rollout_probs_diff_std:0.010555604472756386 - training/rollout_actor_probs_pearson_corr:0.9980065822601318 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.0000488758087158 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.208931204630062e-05 - rollout_corr/rollout_is_max:1.914077877998352 - rollout_corr/rollout_is_min:0.227353036403656 - rollout_corr/rollout_is_std:0.02918272092938423 - rollout_corr/rollout_is_eff_sample_size:0.999149212500466 - rollout_corr/rollout_is_seq_mean:0.9999838471412659 - rollout_corr/rollout_is_seq_std:0.0015772389015182853 - rollout_corr/rollout_is_seq_max:1.0057252645492554 - rollout_corr/rollout_is_seq_min:0.9941803216934204 - rollout_corr/rollout_is_seq_max_deviation:0.00581967830657959 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.109215888660401) - rollout_corr/training_log_ppl:np.float64(0.10109948862009333) - rollout_corr/kl:np.float64(0.0005538763568267713) - rollout_corr/k3_kl:np.float64(0.0005117594077042042) - rollout_corr/rollout_ppl:np.float64(1.1085950387641788) - rollout_corr/rollout_log_ppl:np.float64(0.10054561170181842) - rollout_corr/log_ppl_diff:np.float64(0.0005538769182749093) - rollout_corr/log_ppl_abs_diff:np.float64(0.001272055131266825) - rollout_corr/log_ppl_diff_max:np.float64(0.006336919963359833) - rollout_corr/log_ppl_diff_min:np.float64(-0.004927821457386017) - rollout_corr/ppl_ratio:np.float64(1.000555366044864) - rollout_corr/chi2_token:np.float64(0.0009450593497604132) - rollout_corr/chi2_seq:np.float64(1.0984282961580902) - actor/pg_loss:np.float64(0.0019886784721165895) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003411252544083254) - actor/ppo_kl:np.float64(9.175821236606652e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(50298.95451163477) - self_distillation/token_reweight_w_std:np.float64(977247.3528398918) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999259575270116) - self_distillation/token_reweight_w_clip_frac:np.float64(0.038464386205305345) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.25309666618704796) - perf/mfu/actor:np.float64(0.04338852253393083) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.42945861816406) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00773917231708765 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00773917231708765 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:464.04296875 - response_length/max:1880.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:464.04296875 - response_length_non_aborted/max:1880.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001201629638671875 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6977196279913187) - timing_s/agent_loop/generate_sequences/max:np.float64(12.517238644883037) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.792556140542729) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.517238644883037) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:349 - timing_s/agent_loop/slowest/response_length:1880 - timing_s/gen:18.328095687553287 - timing_s/reward:0.06305506825447083 - timing_s/old_log_prob:2.362060610204935 - timing_s/adv:0.6524072252213955 - timing_s/update_actor:15.651612892746925 - timing_s/step:37.145222717896104 - timing_s/stop_profile:4.989840090274811e-05 - timing_per_token_ms/update_actor:0.08264703527184601 - timing_per_token_ms/adv:0.0034449818893403992 - timing_per_token_ms/gen:0.1542833931356815 - perf/total_num_tokens:189379 - perf/time_per_step:37.145222717896104 - perf/throughput:637.2925848306987 (TaskRunner pid=2074004) Training Progress: 82%|████████▏ | 82/100 [1:09:25<15:17, 50.99s/it] (TaskRunner pid=2074004) step:83 - global_seqlen/min:16528 - global_seqlen/max:26927 - global_seqlen/minmax_diff:10399 - global_seqlen/balanced_min:21714 - global_seqlen/balanced_max:21743 - global_seqlen/mean:21736.25 - actor/entropy:0.1269112080335617 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6248966455459595 - training/rollout_probs_diff_mean:0.0030177931766957045 - training/rollout_probs_diff_std:0.011725279502570629 - training/rollout_actor_probs_pearson_corr:0.9978418350219727 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9999353289604187 - rollout_corr/rollout_is_ratio_fraction_high:3.8849284464959055e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.769856892991811e-05 - rollout_corr/rollout_is_max:2.3085124492645264 - rollout_corr/rollout_is_min:0.010780704207718372 - rollout_corr/rollout_is_std:0.03224320337176323 - rollout_corr/rollout_is_eff_sample_size:0.9989612175582933 - rollout_corr/rollout_is_seq_mean:0.9999208450317383 - rollout_corr/rollout_is_seq_std:0.001950851990841329 - rollout_corr/rollout_is_seq_max:1.0089315176010132 - rollout_corr/rollout_is_seq_min:0.9943766593933105 - rollout_corr/rollout_is_seq_max_deviation:0.008931517601013184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1371404435485601) - rollout_corr/training_log_ppl:np.float64(0.12448573324945755) - rollout_corr/kl:np.float64(0.0008972727727523111) - rollout_corr/k3_kl:np.float64(0.0008475244219425804) - rollout_corr/rollout_ppl:np.float64(1.1360785402357578) - rollout_corr/rollout_log_ppl:np.float64(0.12358846047573024) - rollout_corr/log_ppl_diff:np.float64(0.000897272773727309) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018847366955014877) - rollout_corr/log_ppl_diff_max:np.float64(0.01430445909500122) - rollout_corr/log_ppl_diff_min:np.float64(-0.007731936872005463) - rollout_corr/ppl_ratio:np.float64(1.0009009479545057) - rollout_corr/chi2_token:np.float64(0.001457967096939683) - rollout_corr/chi2_seq:np.float64(0.5747340724337846) - actor/pg_loss:np.float64(0.00172864377964288) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005559734236157965) - actor/ppo_kl:np.float64(0.0002423281556467316) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(34890.77068122127) - self_distillation/token_reweight_w_std:np.float64(689731.6690365797) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998807616066188) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03181662493443582) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.24900896847248077) - perf/mfu/actor:np.float64(0.039740719700471376) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.4448585510254) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005997358239255846 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005997358239255846 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:402.1953125 - response_length/max:1439.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:402.1953125 - response_length_non_aborted/max:1439.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.76057893037796e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6644078604876995) - timing_s/agent_loop/generate_sequences/max:np.float64(10.04941943101585) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.103168618654308) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.04941943101585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:346 - timing_s/agent_loop/slowest/response_length:1439 - timing_s/gen:15.86747464723885 - timing_s/reward:0.06120395101606846 - timing_s/old_log_prob:2.3169026523828506 - timing_s/adv:0.5539372637867928 - timing_s/update_actor:15.275738924741745 - timing_s/step:34.1625665910542 - timing_s/stop_profile:3.450922667980194e-05 - timing_per_token_ms/update_actor:0.08784713856312465 - timing_per_token_ms/adv:0.0031855613536534173 - timing_per_token_ms/gen:0.1541100080344093 - perf/total_num_tokens:173890 - perf/time_per_step:34.1625665910542 - perf/throughput:636.2592793508626 (TaskRunner pid=2074004) Training Progress: 83%|████████▎ | 83/100 [1:09:59<13:01, 45.94s/it] (TaskRunner pid=2074004) step:84 - global_seqlen/min:18342 - global_seqlen/max:28767 - global_seqlen/minmax_diff:10425 - global_seqlen/balanced_min:22295 - global_seqlen/balanced_max:22560 - global_seqlen/mean:22373.75 - actor/entropy:0.11953727900981903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016381561756134 - training/rollout_probs_diff_mean:0.0028403468895703554 - training/rollout_probs_diff_std:0.010894829407334328 - training/rollout_actor_probs_pearson_corr:0.9980471134185791 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000654458999634 - rollout_corr/rollout_is_ratio_fraction_high:8.987794899439905e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.6963383788825013e-05 - rollout_corr/rollout_is_max:2.1378886699676514 - rollout_corr/rollout_is_min:0.1871485859155655 - rollout_corr/rollout_is_std:0.030487289652228355 - rollout_corr/rollout_is_eff_sample_size:0.9990715072785922 - rollout_corr/rollout_is_seq_mean:1.0000371932983398 - rollout_corr/rollout_is_seq_std:0.0019689607433974743 - rollout_corr/rollout_is_seq_max:1.0071165561676025 - rollout_corr/rollout_is_seq_min:0.9907252788543701 - rollout_corr/rollout_is_seq_max_deviation:0.009274721145629883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1271462766453624) - rollout_corr/training_log_ppl:np.float64(0.11585301743616583) - rollout_corr/kl:np.float64(0.0005694015161132882) - rollout_corr/k3_kl:np.float64(0.0006418242667365348) - rollout_corr/rollout_ppl:np.float64(1.1264640409499407) - rollout_corr/rollout_log_ppl:np.float64(0.11528361437740386) - rollout_corr/log_ppl_diff:np.float64(0.0005694030587619636) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015966345054039266) - rollout_corr/log_ppl_diff_max:np.float64(0.010501980781555176) - rollout_corr/log_ppl_diff_min:np.float64(-0.006051577627658844) - rollout_corr/ppl_ratio:np.float64(1.0005718814209104) - rollout_corr/chi2_token:np.float64(0.0016377929132431746) - rollout_corr/chi2_seq:np.float64(0.6639628079719841) - actor/pg_loss:np.float64(0.002379689714871347) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000606918740686524) - actor/ppo_kl:np.float64(9.888500281718393e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(92119.91488653235) - self_distillation/token_reweight_w_std:np.float64(1798317.0580325122) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002044991124421) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05570600633654976) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.29781750962138176) - perf/mfu/actor:np.float64(0.040360571878840845) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.56901168823242) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005010695313103497 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005010695313103497 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:434.6171875 - response_length/max:2171.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.6171875 - response_length_non_aborted/max:2171.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.546158015727997e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7891156543046236) - timing_s/agent_loop/generate_sequences/max:np.float64(13.379426375031471) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.27586570179119) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.379426375031471) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:2171 - timing_s/gen:19.16517908498645 - timing_s/reward:0.06223759055137634 - timing_s/old_log_prob:2.38674658536911 - timing_s/adv:0.6694579813629389 - timing_s/update_actor:15.630993586033583 - timing_s/step:37.9998825173825 - timing_s/stop_profile:3.123842179775238e-05 - timing_per_token_ms/update_actor:0.08732886522170838 - timing_per_token_ms/adv:0.003740197672288613 - timing_per_token_ms/gen:0.17225269260831594 - perf/total_num_tokens:178990 - perf/time_per_step:37.9998825173825 - perf/throughput:588.7847150518281 (TaskRunner pid=2074004) Training Progress: 84%|████████▍ | 84/100 [1:10:37<11:37, 43.57s/it] (TaskRunner pid=2074004) step:85 - global_seqlen/min:13444 - global_seqlen/max:29698 - global_seqlen/minmax_diff:16254 - global_seqlen/balanced_min:21579 - global_seqlen/balanced_max:21625 - global_seqlen/mean:21607.0 - actor/entropy:0.1099928542971611 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7565039396286011 - training/rollout_probs_diff_mean:0.002620653947815299 - training/rollout_probs_diff_std:0.011461569927632809 - training/rollout_actor_probs_pearson_corr:0.9977118968963623 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.72265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9998971819877625 - rollout_corr/rollout_is_ratio_fraction_high:9.451795449422207e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00010396975267212838 - rollout_corr/rollout_is_max:2.1670234203338623 - rollout_corr/rollout_is_min:0.16362474858760834 - rollout_corr/rollout_is_std:0.030536126345396042 - rollout_corr/rollout_is_eff_sample_size:0.999068294612318 - rollout_corr/rollout_is_seq_mean:1.0000232458114624 - rollout_corr/rollout_is_seq_std:0.0025593643076717854 - rollout_corr/rollout_is_seq_max:1.008395791053772 - rollout_corr/rollout_is_seq_min:0.9886937141418457 - rollout_corr/rollout_is_seq_max_deviation:0.011306285858154297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1031886860728264) - rollout_corr/training_log_ppl:np.float64(0.09535550630971557) - rollout_corr/kl:np.float64(0.000747708574247774) - rollout_corr/k3_kl:np.float64(0.0006307248504953122) - rollout_corr/rollout_ppl:np.float64(1.1023486168123782) - rollout_corr/rollout_log_ppl:np.float64(0.09460779764049221) - rollout_corr/log_ppl_diff:np.float64(0.000747708669223357) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018803431230480783) - rollout_corr/log_ppl_diff_max:np.float64(0.013932567089796066) - rollout_corr/log_ppl_diff_min:np.float64(-0.007006004452705383) - rollout_corr/ppl_ratio:np.float64(1.0007511703297496) - rollout_corr/chi2_token:np.float64(0.000999337062239647) - rollout_corr/chi2_seq:np.float64(0.17764197965152562) - actor/pg_loss:np.float64(0.001810003537684679) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005623773683964828) - actor/ppo_kl:np.float64(0.000197975718934984) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.72265625) - self_distillation/token_reweight_w_mean:np.float64(103010.1768341891) - self_distillation/token_reweight_w_std:np.float64(1687668.886640767) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000071118120104) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03546456596814096) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.2238437756896019) - perf/mfu/actor:np.float64(0.03919303431293418) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.48588943481445) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027670131530612707 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027670131530612707 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:413.28125 - response_length/max:1808.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:413.28125 - response_length_non_aborted/max:1808.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.203617572784424e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7867520544677973) - timing_s/agent_loop/generate_sequences/max:np.float64(11.670810116454959) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.985955903917784) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.670810116454959) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:1808 - timing_s/gen:17.77034630998969 - timing_s/reward:0.08232808858156204 - timing_s/old_log_prob:2.3802652321755886 - timing_s/adv:0.6049736253917217 - timing_s/update_actor:15.622556304559112 - timing_s/step:36.56155405379832 - timing_s/stop_profile:5.652382969856262e-05 - timing_per_token_ms/update_actor:0.09037902244966395 - timing_per_token_ms/adv:0.003499870559261592 - timing_per_token_ms/gen:0.167961685349619 - perf/total_num_tokens:172856 - perf/time_per_step:36.56155405379832 - perf/throughput:590.9759735104938 (TaskRunner pid=2074004) Training Progress: 85%|████████▌ | 85/100 [1:11:14<10:22, 41.47s/it] (TaskRunner pid=2074004) step:86 - global_seqlen/min:17265 - global_seqlen/max:27278 - global_seqlen/minmax_diff:10013 - global_seqlen/balanced_min:22005 - global_seqlen/balanced_max:22347 - global_seqlen/mean:22060.875 - actor/entropy:0.1164645105600357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5546000003814697 - training/rollout_probs_diff_mean:0.0027262147050350904 - training/rollout_probs_diff_std:0.011048424988985062 - training/rollout_actor_probs_pearson_corr:0.9979538917541504 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999389052391052 - rollout_corr/rollout_is_ratio_fraction_high:2.80670246866066e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.6778372052358463e-05 - rollout_corr/rollout_is_max:2.691387176513672 - rollout_corr/rollout_is_min:0.28650444746017456 - rollout_corr/rollout_is_std:0.030255712568759918 - rollout_corr/rollout_is_eff_sample_size:0.9990854290712073 - rollout_corr/rollout_is_seq_mean:1.0000141859054565 - rollout_corr/rollout_is_seq_std:0.001993668731302023 - rollout_corr/rollout_is_seq_max:1.0104111433029175 - rollout_corr/rollout_is_seq_min:0.9925797581672668 - rollout_corr/rollout_is_seq_max_deviation:0.01041114330291748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.118015536107123) - rollout_corr/training_log_ppl:np.float64(0.10848677079411573) - rollout_corr/kl:np.float64(0.0006072961230856322) - rollout_corr/k3_kl:np.float64(0.0005712031893807534) - rollout_corr/rollout_ppl:np.float64(1.1173007288016379) - rollout_corr/rollout_log_ppl:np.float64(0.10787947439894197) - rollout_corr/log_ppl_diff:np.float64(0.000607296395173762) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015548389492323622) - rollout_corr/log_ppl_diff_max:np.float64(0.008465766906738281) - rollout_corr/log_ppl_diff_min:np.float64(-0.008714862167835236) - rollout_corr/ppl_ratio:np.float64(1.0006095981225371) - rollout_corr/chi2_token:np.float64(0.0010650106705725193) - rollout_corr/chi2_seq:np.float64(0.39449123083613813) - actor/pg_loss:np.float64(0.000594716053456068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002205379737461044) - actor/ppo_kl:np.float64(0.00013570546891372715) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(10468.989274371881) - self_distillation/token_reweight_w_std:np.float64(246581.90501564916) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998316585551947) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015069760556798428) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.09993577841669321) - perf/mfu/actor:np.float64(0.04011440240529307) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.5170135498047) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0022991569712758064 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0022991569712758064 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:417.52734375 - response_length/max:1781.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.52734375 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.181055843830109e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.044587777927518) - timing_s/agent_loop/generate_sequences/max:np.float64(11.397420486435294) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3413470640807645) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.397420486435294) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:1781 - timing_s/gen:17.311374351382256 - timing_s/reward:0.062451690435409546 - timing_s/old_log_prob:2.356656324118376 - timing_s/adv:0.579136572778225 - timing_s/update_actor:15.45087962038815 - timing_s/step:35.84615935757756 - timing_s/stop_profile:0.00011509284377098083 - timing_per_token_ms/update_actor:0.08754684265916556 - timing_per_token_ms/adv:0.0032814687358175103 - timing_per_token_ms/gen:0.16195958677278113 - perf/total_num_tokens:176487 - perf/time_per_step:35.84615935757756 - perf/throughput:615.4320405691252 (TaskRunner pid=2074004) Training Progress: 86%|████████▌ | 86/100 [1:11:50<09:17, 39.80s/it] (TaskRunner pid=2074004) step:87 - global_seqlen/min:18840 - global_seqlen/max:25215 - global_seqlen/minmax_diff:6375 - global_seqlen/balanced_min:22425 - global_seqlen/balanced_max:22454 - global_seqlen/mean:22447.0 - actor/entropy:0.12408103793859482 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33947664499282837 - training/rollout_probs_diff_mean:0.0029533642809838057 - training/rollout_probs_diff_std:0.011205787770450115 - training/rollout_actor_probs_pearson_corr:0.9980510473251343 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000004768371582 - rollout_corr/rollout_is_ratio_fraction_high:9.21557057154132e-06 - rollout_corr/rollout_is_ratio_fraction_low:7.372456457233056e-05 - rollout_corr/rollout_is_max:2.10186767578125 - rollout_corr/rollout_is_min:0.05793754756450653 - rollout_corr/rollout_is_std:0.03159523010253906 - rollout_corr/rollout_is_eff_sample_size:0.9990027369357228 - rollout_corr/rollout_is_seq_mean:1.000128984451294 - rollout_corr/rollout_is_seq_std:0.002129511907696724 - rollout_corr/rollout_is_seq_max:1.0119233131408691 - rollout_corr/rollout_is_seq_min:0.9937775135040283 - rollout_corr/rollout_is_seq_max_deviation:0.01192331314086914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1291866833344102) - rollout_corr/training_log_ppl:np.float64(0.11755550177258556) - rollout_corr/kl:np.float64(0.0004603647517873455) - rollout_corr/k3_kl:np.float64(0.0006310233444892788) - rollout_corr/rollout_ppl:np.float64(1.1286979308351874) - rollout_corr/rollout_log_ppl:np.float64(0.11709513665482518) - rollout_corr/log_ppl_diff:np.float64(0.00046036511776037514) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016326033364748582) - rollout_corr/log_ppl_diff_max:np.float64(0.007639244198799133) - rollout_corr/log_ppl_diff_min:np.float64(-0.010535374283790588) - rollout_corr/ppl_ratio:np.float64(1.0004629627801478) - rollout_corr/chi2_token:np.float64(0.0016186770517379045) - rollout_corr/chi2_seq:np.float64(0.7282668207772076) - actor/pg_loss:np.float64(0.0011052467161789536) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023867743630034965) - actor/ppo_kl:np.float64(1.1898970175394652e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_w_mean:np.float64(55895.11731084273) - self_distillation/token_reweight_w_std:np.float64(747223.8999097699) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997360575944185) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02877963568607811) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.2662094905972481) - perf/mfu/actor:np.float64(0.04090594702599697) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.4480094909668) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027278088964521885 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027278088964521885 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:423.875 - response_length/max:1575.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.875 - response_length_non_aborted/max:1575.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011235661804676056 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7568011898547411) - timing_s/agent_loop/generate_sequences/max:np.float64(10.530228452757001) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.055000598782499) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.530228452757001) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:239 - timing_s/agent_loop/slowest/response_length:1575 - timing_s/gen:16.49220446869731 - timing_s/reward:0.06013626419007778 - timing_s/old_log_prob:2.299874674528837 - timing_s/adv:0.572662265971303 - timing_s/update_actor:15.511149242520332 - timing_s/step:34.9925875030458 - timing_s/stop_profile:0.00011404044926166534 - timing_per_token_ms/update_actor:0.0863765160295381 - timing_per_token_ms/adv:0.0031889688264094477 - timing_per_token_ms/gen:0.1519850750948956 - perf/total_num_tokens:179576 - perf/time_per_step:34.9925875030458 - perf/throughput:641.4787131144899 (TaskRunner pid=2074004) Training Progress: 87%|████████▋ | 87/100 [1:12:25<08:18, 38.37s/it] (TaskRunner pid=2074004) step:88 - global_seqlen/min:17924 - global_seqlen/max:32180 - global_seqlen/minmax_diff:14256 - global_seqlen/balanced_min:22814 - global_seqlen/balanced_max:22832 - global_seqlen/mean:22825.125 - actor/entropy:0.10808072239160538 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2607862949371338 - training/rollout_probs_diff_mean:0.0027171641122549772 - training/rollout_probs_diff_std:0.010765996761620045 - training/rollout_actor_probs_pearson_corr:0.9980359077453613 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9296875 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9296875 - rollout_corr/rollout_is_mean:0.9999313950538635 - rollout_corr/rollout_is_ratio_fraction_high:1.7341692000627518e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3354230001568794e-05 - rollout_corr/rollout_is_max:3.181804656982422 - rollout_corr/rollout_is_min:0.01616358757019043 - rollout_corr/rollout_is_std:0.030833328142762184 - rollout_corr/rollout_is_eff_sample_size:0.9990502088766346 - rollout_corr/rollout_is_seq_mean:0.9999111890792847 - rollout_corr/rollout_is_seq_std:0.001777790836058557 - rollout_corr/rollout_is_seq_max:1.0082212686538696 - rollout_corr/rollout_is_seq_min:0.9945080876350403 - rollout_corr/rollout_is_seq_max_deviation:0.008221268653869629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.112278978805989) - rollout_corr/training_log_ppl:np.float64(0.10384648678154917) - rollout_corr/kl:np.float64(0.0007002655332808416) - rollout_corr/k3_kl:np.float64(0.0007456282953626214) - rollout_corr/rollout_ppl:np.float64(1.111463341396302) - rollout_corr/rollout_log_ppl:np.float64(0.10314622065925505) - rollout_corr/log_ppl_diff:np.float64(0.0007002661222941242) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017377006079186685) - rollout_corr/log_ppl_diff_max:np.float64(0.015303656458854675) - rollout_corr/log_ppl_diff_min:np.float64(-0.006968945264816284) - rollout_corr/ppl_ratio:np.float64(1.0007034207228571) - rollout_corr/chi2_token:np.float64(0.0014822578523308039) - rollout_corr/chi2_seq:np.float64(1.0283904024399817) - actor/pg_loss:np.float64(0.0028022772166877985) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011713389812939567) - actor/ppo_kl:np.float64(6.31784731428553e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(102159.92961579212) - self_distillation/token_reweight_w_std:np.float64(1634106.4369794251) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001399826724082) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04912651109043509) - self_distillation/empty_target_batch:np.float64(0.0703125) - actor/grad_norm:np.float64(0.34347517415881157) - perf/mfu/actor:np.float64(0.041204584153654134) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.4816131591797) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009243122301995754 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009243122301995754 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:450.50390625 - response_length/max:1537.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:450.50390625 - response_length_non_aborted/max:1537.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010040774941444397 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8878467436879873) - timing_s/agent_loop/generate_sequences/max:np.float64(10.596595136448741) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.400526633304253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.596595136448741) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1537 - timing_s/gen:16.373642224818468 - timing_s/reward:0.06224523112177849 - timing_s/old_log_prob:2.349311839789152 - timing_s/adv:0.6097414214164019 - timing_s/update_actor:15.592103796079755 - timing_s/step:35.07431404106319 - timing_s/stop_profile:0.00012129172682762146 - timing_per_token_ms/update_actor:0.08538892884529524 - timing_per_token_ms/adv:0.003339200888365353 - timing_per_token_ms/gen:0.1419733304270259 - perf/total_num_tokens:182601 - perf/time_per_step:35.07431404106319 - perf/throughput:650.764687037857 (TaskRunner pid=2074004) Training Progress: 88%|████████▊ | 88/100 [1:13:00<07:28, 37.40s/it] (TaskRunner pid=2074004) step:89 - global_seqlen/min:13752 - global_seqlen/max:27817 - global_seqlen/minmax_diff:14065 - global_seqlen/balanced_min:22301 - global_seqlen/balanced_max:22323 - global_seqlen/mean:22317.25 - actor/entropy:0.10688827931880951 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.601607620716095 - training/rollout_probs_diff_mean:0.002714034402742982 - training/rollout_probs_diff_std:0.011373176239430904 - training/rollout_actor_probs_pearson_corr:0.9978145360946655 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:1.0 - self_distillation/correct_sample_fraction:0.8125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999041557312012 - rollout_corr/rollout_is_ratio_fraction_high:3.661863593151793e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.239193266490474e-05 - rollout_corr/rollout_is_max:2.066577672958374 - rollout_corr/rollout_is_min:0.30741891264915466 - rollout_corr/rollout_is_std:0.030931759625673294 - rollout_corr/rollout_is_eff_sample_size:0.9990437838386964 - rollout_corr/rollout_is_seq_mean:0.9999012351036072 - rollout_corr/rollout_is_seq_std:0.00172822130843997 - rollout_corr/rollout_is_seq_max:1.0071568489074707 - rollout_corr/rollout_is_seq_min:0.9942929744720459 - rollout_corr/rollout_is_seq_max_deviation:0.007156848907470703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1084096948616207) - rollout_corr/training_log_ppl:np.float64(0.09965023837503395) - rollout_corr/kl:np.float64(0.0006493180134174992) - rollout_corr/k3_kl:np.float64(0.0006724119927685734) - rollout_corr/rollout_ppl:np.float64(1.107654474209994) - rollout_corr/rollout_log_ppl:np.float64(0.09900091987219639) - rollout_corr/log_ppl_diff:np.float64(0.0006493185028375592) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017076216572604608) - rollout_corr/log_ppl_diff_max:np.float64(0.01190245896577835) - rollout_corr/log_ppl_diff_min:np.float64(-0.013669520616531372) - rollout_corr/ppl_ratio:np.float64(1.0006527956575155) - rollout_corr/chi2_token:np.float64(0.0013369775842875242) - rollout_corr/chi2_seq:np.float64(5.031164270127192) - actor/pg_loss:np.float64(0.002956106443889439) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008950727274168457) - actor/ppo_kl:np.float64(4.916048081948787e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_w_mean:np.float64(71424.80499303155) - self_distillation/token_reweight_w_std:np.float64(1224926.4624935077) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000664854887873) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04491751454042969) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.3388006314635277) - perf/mfu/actor:np.float64(0.04076402686351015) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.86504364013672) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.8125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008070976473391056 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.008070976473391056 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:426.6953125 - response_length/max:1575.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.6953125 - response_length_non_aborted/max:1575.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003277640789747238 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9218454416841269) - timing_s/agent_loop/generate_sequences/max:np.float64(10.495202466845512) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.246527711606177) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.495202466845512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1575 - timing_s/gen:16.446799524128437 - timing_s/reward:0.056499673053622246 - timing_s/old_log_prob:2.5402090717107058 - timing_s/adv:0.6734065376222134 - timing_s/update_actor:15.645092817023396 - timing_s/step:35.48852607421577 - timing_s/stop_profile:0.00012188777327537537 - timing_per_token_ms/update_actor:0.08762892390988695 - timing_per_token_ms/adv:0.003771782688403664 - timing_per_token_ms/gen:0.15056483809188015 - perf/total_num_tokens:178538 - perf/time_per_step:35.48852607421577 - perf/throughput:628.8581823130328 (TaskRunner pid=2074004) Training Progress: 89%|████████▉ | 89/100 [1:13:36<06:46, 36.99s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We are to find the charge $ q $ using the formula: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Rearranging for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is **B: 56 pC**. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2074004) step:90 - global_seqlen/min:17503 - global_seqlen/max:30542 - global_seqlen/minmax_diff:13039 - global_seqlen/balanced_min:22163 - global_seqlen/balanced_max:22749 - global_seqlen/mean:22249.5 - actor/entropy:0.10518425703048706 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40461307764053345 - training/rollout_probs_diff_mean:0.0026247852947562933 - training/rollout_probs_diff_std:0.010737664066255093 - training/rollout_actor_probs_pearson_corr:0.9979662895202637 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.0000884532928467 - rollout_corr/rollout_is_ratio_fraction_high:1.823819002311211e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.20718560134992e-05 - rollout_corr/rollout_is_max:2.0903313159942627 - rollout_corr/rollout_is_min:0.19070646166801453 - rollout_corr/rollout_is_std:0.029787156730890274 - rollout_corr/rollout_is_eff_sample_size:0.9991136308144901 - rollout_corr/rollout_is_seq_mean:1.0000239610671997 - rollout_corr/rollout_is_seq_std:0.00162831402849406 - rollout_corr/rollout_is_seq_max:1.0044366121292114 - rollout_corr/rollout_is_seq_min:0.9923098683357239 - rollout_corr/rollout_is_seq_max_deviation:0.007690131664276123 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.102782507892698) - rollout_corr/training_log_ppl:np.float64(0.09468803418349125) - rollout_corr/kl:np.float64(0.0005543236077372171) - rollout_corr/k3_kl:np.float64(0.0005838268394882107) - rollout_corr/rollout_ppl:np.float64(1.1021574116311967) - rollout_corr/rollout_log_ppl:np.float64(0.0941337100121018) - rollout_corr/log_ppl_diff:np.float64(0.0005543241713894531) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015017515033832751) - rollout_corr/log_ppl_diff_max:np.float64(0.010810032486915588) - rollout_corr/log_ppl_diff_min:np.float64(-0.006302401423454285) - rollout_corr/ppl_ratio:np.float64(1.0005565818864852) - rollout_corr/chi2_token:np.float64(0.0012237117625772953) - rollout_corr/chi2_seq:np.float64(0.7812420942354947) - actor/pg_loss:np.float64(0.001676898100413382) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036616270745071233) - actor/ppo_kl:np.float64(0.00012324695833854094) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_w_mean:np.float64(32326.37215174688) - self_distillation/token_reweight_w_std:np.float64(704860.9180972856) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991597740445286) - self_distillation/token_reweight_w_clip_frac:np.float64(0.029383839399088174) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.14125302247703075) - perf/mfu/actor:np.float64(0.0403704412148885) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(144.92715072631836) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6921875) - val-aux/sciknoweval/reward/std@16:np.float64(0.1468032056932851) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7523500000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10240961009878324) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6337875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1445766161320927) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6938375000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1469724185844502) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.790025) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06162330599483268) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5755125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1281247414410848) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7121125000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11456407227764583) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8124499999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.030947104305230778) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5231125000000001) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10100008417206525) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7222500000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08294761183043528) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.821725) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.011989192674950796) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4808) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0733318252008867) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.729125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.060664540093849084) - val-aux/sciknoweval/score/mean@16:np.float64(0.6921875) - val-aux/sciknoweval/score/std@16:np.float64(0.1468032056932851) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7523500000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10240961009878324) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6337875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1445766161320927) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6938375000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1469724185844502) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.790025) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06162330599483268) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5755125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1281247414410848) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7121125000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11456407227764583) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8124499999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.030947104305230778) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5231125000000001) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10100008417206525) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7222500000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08294761183043528) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.821725) - val-aux/sciknoweval/score/best@16/std:np.float64(0.011989192674950796) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4808) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0733318252008867) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.729125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.060664540093849084) - val-core/sciknoweval/acc/mean@16:np.float64(0.6921875) - val-aux/sciknoweval/acc/std@16:np.float64(0.1468032056932851) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7523500000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10240961009878324) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6337875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1445766161320927) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6938375000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1469724185844502) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.790025) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06162330599483268) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5755125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1281247414410848) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7121125000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11456407227764583) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8124499999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.030947104305230778) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5231125000000001) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10100008417206525) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7222500000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08294761183043528) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.821725) - val-core/sciknoweval/acc/best@16/std:np.float64(0.011989192674950796) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4808) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0733318252008867) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.729125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.060664540093849084) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.009077304717673634) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998374999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.002405465425136726) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9957625) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.011852098487061466) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9977625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.008832748449106717) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9916125000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.015594544271371125) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9995750000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.003952916592698636) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9849375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.018366765656865063) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999925) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0013474443894951921) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9760875) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0180217269071149) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018307723104953766 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018307723104953766 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:428.359375 - response_length/max:2385.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:428.359375 - response_length_non_aborted/max:2385.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017513521015644073 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2588027901947498) - timing_s/agent_loop/generate_sequences/max:np.float64(14.54117732308805) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3901306394909625) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.54117732308805) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:2385 - timing_s/gen:21.131900370121002 - timing_s/reward:0.05953768454492092 - timing_s/old_log_prob:2.3052819333970547 - timing_s/adv:0.5906773693859577 - timing_s/update_actor:15.686287306249142 - timing_s/step:39.856287997215986 - timing_s/testing:87.98717088066041 - timing_s/save_checkpoint:6.872611632570624 - timing_s/stop_profile:4.615262150764465e-05 - timing_per_token_ms/update_actor:0.08812718997196084 - timing_per_token_ms/adv:0.0033184867602977465 - timing_per_token_ms/gen:0.19270381515703996 - perf/total_num_tokens:177996 - perf/time_per_step:39.856287997215986 - perf/throughput:558.24315604991 (TaskRunner pid=2074004) Training Progress: 90%|█████████ | 90/100 [1:15:51<11:03, 66.32s/it] (TaskRunner pid=2074004) step:91 - global_seqlen/min:17728 - global_seqlen/max:25869 - global_seqlen/minmax_diff:8141 - global_seqlen/balanced_min:21525 - global_seqlen/balanced_max:21553 - global_seqlen/mean:21545.25 - actor/entropy:0.08843715488910675 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41412943601608276 - training/rollout_probs_diff_mean:0.002419212367385626 - training/rollout_probs_diff_std:0.010600846260786057 - training/rollout_actor_probs_pearson_corr:0.9976838231086731 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.92578125 - self_distillation/correct_sample_fraction:0.83984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.92578125 - rollout_corr/rollout_is_mean:0.9999130368232727 - rollout_corr/rollout_is_ratio_fraction_high:9.281086931878235e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.712434772751294e-05 - rollout_corr/rollout_is_max:2.0842576026916504 - rollout_corr/rollout_is_min:0.18285298347473145 - rollout_corr/rollout_is_std:0.027998575940728188 - rollout_corr/rollout_is_eff_sample_size:0.999216455716407 - rollout_corr/rollout_is_seq_mean:0.9998674392700195 - rollout_corr/rollout_is_seq_std:0.0018235978204756975 - rollout_corr/rollout_is_seq_max:1.007836103439331 - rollout_corr/rollout_is_seq_min:0.9920446276664734 - rollout_corr/rollout_is_seq_max_deviation:0.007955372333526611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0833140267059207) - rollout_corr/training_log_ppl:np.float64(0.07899068885672023) - rollout_corr/kl:np.float64(0.0006408023299613319) - rollout_corr/k3_kl:np.float64(0.00047720418808694376) - rollout_corr/rollout_ppl:np.float64(1.0826310594566166) - rollout_corr/rollout_log_ppl:np.float64(0.07834988591639558) - rollout_corr/log_ppl_diff:np.float64(0.0006408029403246474) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014788467706239317) - rollout_corr/log_ppl_diff_max:np.float64(0.01340678334236145) - rollout_corr/log_ppl_diff_min:np.float64(-0.006584286689758301) - rollout_corr/ppl_ratio:np.float64(1.0006434579845518) - rollout_corr/chi2_token:np.float64(0.0005903118290007114) - rollout_corr/chi2_seq:np.float64(0.44575339392758906) - actor/pg_loss:np.float64(0.00038603588473051786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025614264700379863) - actor/ppo_kl:np.float64(7.664802197382414e-05) - actor/pg_clipfrac_lower:np.float64(5.7444854064669926e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.92578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.92578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_w_mean:np.float64(74129.82275516656) - self_distillation/token_reweight_w_std:np.float64(1313364.9157028943) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003251135349274) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0171360018721316) - self_distillation/empty_target_batch:np.float64(0.07421875) - actor/grad_norm:np.float64(0.11729132384061813) - perf/mfu/actor:np.float64(0.03910900715029075) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.49448776245117) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034270414616912603 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034270414616912603 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:420.8828125 - response_length/max:1701.0 - response_length/min:54.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.8828125 - response_length_non_aborted/max:1701.0 - response_length_non_aborted/min:54.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.895302355289459e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.649962367489934) - timing_s/agent_loop/generate_sequences/max:np.float64(11.452977204695344) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.1154736819225946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.452977204695344) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1701 - timing_s/gen:17.09042201936245 - timing_s/reward:0.056309279054403305 - timing_s/old_log_prob:2.4856445379555225 - timing_s/adv:0.686548063531518 - timing_s/update_actor:15.663952223956585 - timing_s/step:36.09207380563021 - timing_s/stop_profile:0.00012318789958953857 - timing_per_token_ms/update_actor:0.0908782227170524 - timing_per_token_ms/adv:0.003983175314347234 - timing_per_token_ms/gen:0.1586176936439631 - perf/total_num_tokens:172362 - perf/time_per_step:36.09207380563021 - perf/throughput:596.95239780428 (TaskRunner pid=2074004) Training Progress: 91%|█████████ | 91/100 [1:16:27<08:35, 57.27s/it] (TaskRunner pid=2074004) step:92 - global_seqlen/min:18485 - global_seqlen/max:32646 - global_seqlen/minmax_diff:14161 - global_seqlen/balanced_min:24274 - global_seqlen/balanced_max:24304 - global_seqlen/mean:24296.5 - actor/entropy:0.1161329373717308 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46171876788139343 - training/rollout_probs_diff_mean:0.0028441078029572964 - training/rollout_probs_diff_std:0.011383011005818844 - training/rollout_actor_probs_pearson_corr:0.997846245765686 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.000044584274292 - rollout_corr/rollout_is_ratio_fraction_high:3.2485462725162506e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.060682840645313e-05 - rollout_corr/rollout_is_max:2.6116247177124023 - rollout_corr/rollout_is_min:0.3187040090560913 - rollout_corr/rollout_is_std:0.03276945650577545 - rollout_corr/rollout_is_eff_sample_size:0.998927552516703 - rollout_corr/rollout_is_seq_mean:0.9999802708625793 - rollout_corr/rollout_is_seq_std:0.0019448020029813051 - rollout_corr/rollout_is_seq_max:1.0066347122192383 - rollout_corr/rollout_is_seq_min:0.9897977113723755 - rollout_corr/rollout_is_seq_max_deviation:0.010202288627624512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.125696862116456) - rollout_corr/training_log_ppl:np.float64(0.11609659137320705) - rollout_corr/kl:np.float64(0.0009234349944136078) - rollout_corr/k3_kl:np.float64(0.0007315961058225184) - rollout_corr/rollout_ppl:np.float64(1.1246338286437094) - rollout_corr/rollout_log_ppl:np.float64(0.11517315577657428) - rollout_corr/log_ppl_diff:np.float64(0.0009234355966327712) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018325499549973756) - rollout_corr/log_ppl_diff_max:np.float64(0.012700438499450684) - rollout_corr/log_ppl_diff_min:np.float64(-0.006820157170295715) - rollout_corr/ppl_ratio:np.float64(1.0009266710840166) - rollout_corr/chi2_token:np.float64(0.0010245046578347683) - rollout_corr/chi2_seq:np.float64(0.6241324651055038) - actor/pg_loss:np.float64(0.0013751451624557376) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006302250340013416) - actor/ppo_kl:np.float64(0.0003064914879740144) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(70968.99266718118) - self_distillation/token_reweight_w_std:np.float64(1136299.2140333871) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998132719192654) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03681001826771535) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.22948458045721054) - perf/mfu/actor:np.float64(0.04381601917617662) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.27560424804688) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007187408627942204 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007187408627942204 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.984375 - response_length/max:1689.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.984375 - response_length_non_aborted/max:1689.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012553296983242035 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0251992959529161) - timing_s/agent_loop/generate_sequences/max:np.float64(11.716115767136216) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.778788086819986) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.716115767136216) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1689 - timing_s/gen:17.857277484610677 - timing_s/reward:0.061120714992284775 - timing_s/old_log_prob:2.40597465634346 - timing_s/adv:0.6833560299128294 - timing_s/update_actor:15.71237531118095 - timing_s/step:36.81252425536513 - timing_s/stop_profile:0.00011310726404190063 - timing_per_token_ms/update_actor:0.08083661901498647 - timing_per_token_ms/adv:0.003515712293503331 - timing_per_token_ms/gen:0.14502548065986645 - perf/total_num_tokens:194372 - perf/time_per_step:36.81252425536513 - perf/throughput:660.0063563003011 (TaskRunner pid=2074004) Training Progress: 92%|█████████▏| 92/100 [1:17:04<06:49, 51.14s/it] (TaskRunner pid=2074004) step:93 - global_seqlen/min:14252 - global_seqlen/max:27080 - global_seqlen/minmax_diff:12828 - global_seqlen/balanced_min:19571 - global_seqlen/balanced_max:25808 - global_seqlen/mean:20389.375 - actor/entropy:0.11118520796298981 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2569490671157837 - training/rollout_probs_diff_mean:0.0027298631612211466 - training/rollout_probs_diff_std:0.01081868913024664 - training/rollout_actor_probs_pearson_corr:0.9979827404022217 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999743103981018 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9497259855270386 - rollout_corr/rollout_is_min:0.5504913330078125 - rollout_corr/rollout_is_std:0.030463820323348045 - rollout_corr/rollout_is_eff_sample_size:0.9990726971602333 - rollout_corr/rollout_is_seq_mean:1.000046968460083 - rollout_corr/rollout_is_seq_std:0.0025716039817780256 - rollout_corr/rollout_is_seq_max:1.0111936330795288 - rollout_corr/rollout_is_seq_min:0.9910350441932678 - rollout_corr/rollout_is_seq_max_deviation:0.011193633079528809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1135128010064363) - rollout_corr/training_log_ppl:np.float64(0.1049198287037143) - rollout_corr/kl:np.float64(0.0006423072564984977) - rollout_corr/k3_kl:np.float64(0.0005943681892190966) - rollout_corr/rollout_ppl:np.float64(1.1127843372523785) - rollout_corr/rollout_log_ppl:np.float64(0.10427752112082089) - rollout_corr/log_ppl_diff:np.float64(0.0006423075828934088) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018248588894493878) - rollout_corr/log_ppl_diff_max:np.float64(0.009763795882463455) - rollout_corr/log_ppl_diff_min:np.float64(-0.008692525327205658) - rollout_corr/ppl_ratio:np.float64(1.0006455411203206) - rollout_corr/chi2_token:np.float64(0.0011548018082976341) - rollout_corr/chi2_seq:np.float64(0.28039377136155963) - actor/pg_loss:np.float64(0.0013577784411609173) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004785834780705045) - actor/ppo_kl:np.float64(0.0001363070303765923) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(101776.64841872128) - self_distillation/token_reweight_w_std:np.float64(1415721.968034971) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006784941069782) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037072227569296956) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.1903113927692175) - perf/mfu/actor:np.float64(0.0352703899917267) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.2206916809082) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06920255720615387 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06920255720615387 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:379.94921875 - response_length/max:8192.0 - response_length/min:72.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:379.94921875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.636208415031433e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.785060852766037) - timing_s/agent_loop/generate_sequences/max:np.float64(50.456341341137886) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6300632040947676) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.456341341137886) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:280 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.36329558491707 - timing_s/reward:0.06057967245578766 - timing_s/old_log_prob:2.5846155025064945 - timing_s/adv:0.5508739780634642 - timing_s/update_actor:16.457574239000678 - timing_s/step:76.1049917601049 - timing_s/stop_profile:0.00011239387094974518 - timing_per_token_ms/update_actor:0.10089552916041246 - timing_per_token_ms/adv:0.0033772122616771245 - timing_per_token_ms/gen:0.5794698673231113 - perf/total_num_tokens:163115 - perf/time_per_step:76.1049917601049 - perf/throughput:267.91113865790265 (TaskRunner pid=2074004) Training Progress: 93%|█████████▎| 93/100 [1:18:20<06:50, 58.65s/it] (TaskRunner pid=2074004) step:94 - global_seqlen/min:16134 - global_seqlen/max:25356 - global_seqlen/minmax_diff:9222 - global_seqlen/balanced_min:19639 - global_seqlen/balanced_max:19643 - global_seqlen/mean:19641.0 - actor/entropy:0.09782024472951889 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7454107999801636 - training/rollout_probs_diff_mean:0.002670757472515106 - training/rollout_probs_diff_std:0.01144047174602747 - training/rollout_actor_probs_pearson_corr:0.9975129961967468 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.999981164932251 - rollout_corr/rollout_is_ratio_fraction_high:2.235536157968454e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010059912165161222 - rollout_corr/rollout_is_max:2.278707504272461 - rollout_corr/rollout_is_min:0.010440262034535408 - rollout_corr/rollout_is_std:0.03061799891293049 - rollout_corr/rollout_is_eff_sample_size:0.9990632971725152 - rollout_corr/rollout_is_seq_mean:0.9999101161956787 - rollout_corr/rollout_is_seq_std:0.0022188930306583643 - rollout_corr/rollout_is_seq_max:1.0089306831359863 - rollout_corr/rollout_is_seq_min:0.9909623265266418 - rollout_corr/rollout_is_seq_max_deviation:0.009037673473358154 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1009153043851256) - rollout_corr/training_log_ppl:np.float64(0.09386854151671287) - rollout_corr/kl:np.float64(0.0008569538749156891) - rollout_corr/k3_kl:np.float64(0.0006892196433412323) - rollout_corr/rollout_ppl:np.float64(1.0999419218860567) - rollout_corr/rollout_log_ppl:np.float64(0.093011587527144) - rollout_corr/log_ppl_diff:np.float64(0.0008569539895688649) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018225670246465597) - rollout_corr/log_ppl_diff_max:np.float64(0.012111961841583252) - rollout_corr/log_ppl_diff_min:np.float64(-0.009987182915210724) - rollout_corr/ppl_ratio:np.float64(1.0008604750037193) - rollout_corr/chi2_token:np.float64(0.0010376402642577887) - rollout_corr/chi2_seq:np.float64(0.17592160333879292) - actor/pg_loss:np.float64(0.0020128018222749233) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006171084928610071) - actor/ppo_kl:np.float64(0.00018625303451358377) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(47890.5027854254) - self_distillation/token_reweight_w_std:np.float64(780379.3705423808) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003406463656574) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03524100726644974) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.30077048763632774) - perf/mfu/actor:np.float64(0.03571617935583021) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.26049041748047) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005389038939028978 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005389038939028978 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:349.46875 - response_length/max:992.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:349.46875 - response_length_non_aborted/max:992.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011242926120758057 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7382768783718348) - timing_s/agent_loop/generate_sequences/max:np.float64(7.45197612978518) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5364193320201593) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.45197612978518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:992 - timing_s/gen:13.292611926794052 - timing_s/reward:0.058578019961714745 - timing_s/old_log_prob:2.304050751030445 - timing_s/adv:0.5737872198224068 - timing_s/update_actor:15.573226625099778 - timing_s/step:31.890709141269326 - timing_s/stop_profile:0.00011250190436840057 - timing_per_token_ms/update_actor:0.09911172181342459 - timing_per_token_ms/adv:0.0036517184704343386 - timing_per_token_ms/gen:0.14858056790210647 - perf/total_num_tokens:157128 - perf/time_per_step:31.890709141269326 - perf/throughput:615.8847052599044 (TaskRunner pid=2074004) Training Progress: 94%|█████████▍| 94/100 [1:18:52<05:03, 50.64s/it] (TaskRunner pid=2074004) step:95 - global_seqlen/min:19898 - global_seqlen/max:27127 - global_seqlen/minmax_diff:7229 - global_seqlen/balanced_min:22383 - global_seqlen/balanced_max:22390 - global_seqlen/mean:22386.125 - actor/entropy:0.11839713156223297 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4174291491508484 - training/rollout_probs_diff_mean:0.002880886197090149 - training/rollout_probs_diff_std:0.01125163584947586 - training/rollout_actor_probs_pearson_corr:0.9979742765426636 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.000062108039856 - rollout_corr/rollout_is_ratio_fraction_high:1.846091436163988e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.7691370632965118e-05 - rollout_corr/rollout_is_max:2.8053035736083984 - rollout_corr/rollout_is_min:0.35777518153190613 - rollout_corr/rollout_is_std:0.031417399644851685 - rollout_corr/rollout_is_eff_sample_size:0.9990140393695902 - rollout_corr/rollout_is_seq_mean:1.0001864433288574 - rollout_corr/rollout_is_seq_std:0.001943354494869709 - rollout_corr/rollout_is_seq_max:1.009150743484497 - rollout_corr/rollout_is_seq_min:0.9956559538841248 - rollout_corr/rollout_is_seq_max_deviation:0.00915074348449707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1184041863307357) - rollout_corr/training_log_ppl:np.float64(0.10852470160170924) - rollout_corr/kl:np.float64(0.0004109893218089411) - rollout_corr/k3_kl:np.float64(0.0005756196864368235) - rollout_corr/rollout_ppl:np.float64(1.1179265826940536) - rollout_corr/rollout_log_ppl:np.float64(0.10811371153613436) - rollout_corr/log_ppl_diff:np.float64(0.0004109900655748788) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015215025887300726) - rollout_corr/log_ppl_diff_max:np.float64(0.010581329464912415) - rollout_corr/log_ppl_diff_min:np.float64(-0.007771208882331848) - rollout_corr/ppl_ratio:np.float64(1.0004132050089538) - rollout_corr/chi2_token:np.float64(0.0015168595127761364) - rollout_corr/chi2_seq:np.float64(0.37046705558896065) - actor/pg_loss:np.float64(0.0014443554682657123) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039592202506355534) - actor/ppo_kl:np.float64(9.238178380144291e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(35129.66564378468) - self_distillation/token_reweight_w_std:np.float64(673946.7570342987) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999094212660566) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03862924790882971) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.22043711598962545) - perf/mfu/actor:np.float64(0.04063184804557128) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.2927703857422) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033506557811051607 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033506557811051607 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:423.19140625 - response_length/max:1177.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:423.19140625 - response_length_non_aborted/max:1177.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.422190487384796e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9937835987657309) - timing_s/agent_loop/generate_sequences/max:np.float64(8.682577695697546) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.320787707940326) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.682577695697546) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1171 - timing_s/gen:14.465107165277004 - timing_s/reward:0.061005301773548126 - timing_s/old_log_prob:2.500914089381695 - timing_s/adv:0.5702372565865517 - timing_s/update_actor:15.484657960012555 - timing_s/step:33.17570861801505 - timing_s/stop_profile:0.00012160651385784149 - timing_per_token_ms/update_actor:0.08646347882903224 - timing_per_token_ms/adv:0.0031840998419029177 - timing_per_token_ms/gen:0.13351954701788865 - perf/total_num_tokens:179089 - perf/time_per_step:33.17570861801505 - perf/throughput:674.7745845538293 (TaskRunner pid=2074004) Training Progress: 95%|█████████▌| 95/100 [1:19:25<03:47, 45.41s/it] (TaskRunner pid=2074004) step:96 - global_seqlen/min:14700 - global_seqlen/max:31388 - global_seqlen/minmax_diff:16688 - global_seqlen/balanced_min:22938 - global_seqlen/balanced_max:22942 - global_seqlen/mean:22940.5 - actor/entropy:0.10098060220479965 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6580971479415894 - training/rollout_probs_diff_mean:0.002621095860376954 - training/rollout_probs_diff_std:0.011137053370475769 - training/rollout_actor_probs_pearson_corr:0.9977489113807678 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.6796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0001559257507324 - rollout_corr/rollout_is_ratio_fraction_high:1.786160828487482e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.251563172554597e-05 - rollout_corr/rollout_is_max:2.2557666301727295 - rollout_corr/rollout_is_min:0.252839058637619 - rollout_corr/rollout_is_std:0.03050878830254078 - rollout_corr/rollout_is_eff_sample_size:0.9990705553753199 - rollout_corr/rollout_is_seq_mean:1.0001717805862427 - rollout_corr/rollout_is_seq_std:0.001738636172376573 - rollout_corr/rollout_is_seq_max:1.007121205329895 - rollout_corr/rollout_is_seq_min:0.9931297898292542 - rollout_corr/rollout_is_seq_max_deviation:0.0071212053298950195 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.103144368622452) - rollout_corr/training_log_ppl:np.float64(0.09644664141524117) - rollout_corr/kl:np.float64(0.0005017887852607483) - rollout_corr/k3_kl:np.float64(0.0005338758359130225) - rollout_corr/rollout_ppl:np.float64(1.1025838060304523) - rollout_corr/rollout_log_ppl:np.float64(0.09594485214620363) - rollout_corr/log_ppl_diff:np.float64(0.0005017892690375447) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013042025966569781) - rollout_corr/log_ppl_diff_max:np.float64(0.006274871528148651) - rollout_corr/log_ppl_diff_min:np.float64(-0.005490683019161224) - rollout_corr/ppl_ratio:np.float64(1.0005033176857978) - rollout_corr/chi2_token:np.float64(0.0011128755286335945) - rollout_corr/chi2_seq:np.float64(0.5927000071387738) - actor/pg_loss:np.float64(0.0014072328340262175) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000444404973222845) - actor/ppo_kl:np.float64(0.000197858096928627) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_w_mean:np.float64(106493.64426750923) - self_distillation/token_reweight_w_std:np.float64(2085160.6336320788) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998901744838804) - self_distillation/token_reweight_w_clip_frac:np.float64(0.036775027387193404) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.2108887266367674) - perf/mfu/actor:np.float64(0.0416768245283575) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.38690567016602) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002023943467065692 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002023943467065692 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:437.390625 - response_length/max:1046.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.390625 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.732507169246674e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1755850836634636) - timing_s/agent_loop/generate_sequences/max:np.float64(8.23095478862524) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.5998671856941655) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.23095478862524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:296 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:14.00526099652052 - timing_s/reward:0.06283171847462654 - timing_s/old_log_prob:2.3491326589137316 - timing_s/adv:0.5709024202078581 - timing_s/update_actor:15.67631054110825 - timing_s/step:32.75171426124871 - timing_s/stop_profile:0.0001319441944360733 - timing_per_token_ms/update_actor:0.08541831336015045 - timing_per_token_ms/adv:0.0031107779920220686 - timing_per_token_ms/gen:0.1250782427439049 - perf/total_num_tokens:183524 - perf/time_per_step:32.75171426124871 - perf/throughput:700.4366188899866 (TaskRunner pid=2074004) Training Progress: 96%|█████████▌| 96/100 [1:19:58<02:46, 41.63s/it] (TaskRunner pid=2074004) step:97 - global_seqlen/min:17529 - global_seqlen/max:26956 - global_seqlen/minmax_diff:9427 - global_seqlen/balanced_min:22453 - global_seqlen/balanced_max:22536 - global_seqlen/mean:22478.25 - actor/entropy:0.11855632066726685 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3018026649951935 - training/rollout_probs_diff_mean:0.0028152333106845617 - training/rollout_probs_diff_std:0.010734545066952705 - training/rollout_actor_probs_pearson_corr:0.9980775713920593 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.9609375 - self_distillation/correct_sample_fraction:0.79296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9609375 - rollout_corr/rollout_is_mean:0.9999098777770996 - rollout_corr/rollout_is_ratio_fraction_high:1.8008608094532974e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.4025826102588326e-05 - rollout_corr/rollout_is_max:4.380692005157471 - rollout_corr/rollout_is_min:0.2371114045381546 - rollout_corr/rollout_is_std:0.030879687517881393 - rollout_corr/rollout_is_eff_sample_size:0.9990471153295336 - rollout_corr/rollout_is_seq_mean:0.9999960660934448 - rollout_corr/rollout_is_seq_std:0.0019316772231832147 - rollout_corr/rollout_is_seq_max:1.0092357397079468 - rollout_corr/rollout_is_seq_min:0.9920583367347717 - rollout_corr/rollout_is_seq_max_deviation:0.009235739707946777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1194870863109827) - rollout_corr/training_log_ppl:np.float64(0.10984483735228423) - rollout_corr/kl:np.float64(0.0007211861242026885) - rollout_corr/k3_kl:np.float64(0.0006180598990965791) - rollout_corr/rollout_ppl:np.float64(1.118659039027989) - rollout_corr/rollout_log_ppl:np.float64(0.10912365088734077) - rollout_corr/log_ppl_diff:np.float64(0.0007211864649434574) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015570802715956233) - rollout_corr/log_ppl_diff_max:np.float64(0.011463932693004608) - rollout_corr/log_ppl_diff_min:np.float64(-0.005851447582244873) - rollout_corr/ppl_ratio:np.float64(1.000723579665646) - rollout_corr/chi2_token:np.float64(0.0009887183550745249) - rollout_corr/chi2_seq:np.float64(0.33689292683266103) - actor/pg_loss:np.float64(0.0028610387817025185) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005207407527905161) - actor/ppo_kl:np.float64(0.00018532980526941856) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_w_mean:np.float64(61391.32670080848) - self_distillation/token_reweight_w_std:np.float64(1146314.7785266174) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994598587509245) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060132228885777295) - self_distillation/empty_target_batch:np.float64(0.0390625) - actor/grad_norm:np.float64(0.27978259697556496) - perf/mfu/actor:np.float64(0.04065052381475308) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.33649063110352) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006340155377984047 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006340155377984047 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.8203125 - response_length/max:1625.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.8203125 - response_length_non_aborted/max:1625.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014923326671123505 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1448223646730185) - timing_s/agent_loop/generate_sequences/max:np.float64(10.64167152158916) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.490232729578565) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.64167152158916) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:1625 - timing_s/gen:16.820046979933977 - timing_s/reward:0.062204478308558464 - timing_s/old_log_prob:2.3776866644620895 - timing_s/adv:0.6371833365410566 - timing_s/update_actor:15.58851253055036 - timing_s/step:35.57156837172806 - timing_s/stop_profile:0.00011635571718215942 - timing_per_token_ms/update_actor:0.08668664448161201 - timing_per_token_ms/adv:0.003543332646786653 - timing_per_token_ms/gen:0.15145281726605897 - perf/total_num_tokens:179826 - perf/time_per_step:35.57156837172806 - perf/throughput:631.9161911867089 (TaskRunner pid=2074004) Training Progress: 97%|█████████▋| 97/100 [1:20:34<01:59, 39.83s/it] (TaskRunner pid=2074004) step:98 - global_seqlen/min:17481 - global_seqlen/max:26054 - global_seqlen/minmax_diff:8573 - global_seqlen/balanced_min:22060 - global_seqlen/balanced_max:22069 - global_seqlen/mean:22065.375 - actor/entropy:0.09252151846885681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6313449740409851 - training/rollout_probs_diff_mean:0.0024829101748764515 - training/rollout_probs_diff_std:0.01076439581811428 - training/rollout_actor_probs_pearson_corr:0.9977108836174011 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999709129333496 - rollout_corr/rollout_is_ratio_fraction_high:2.7911390134249814e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.582278026849963e-05 - rollout_corr/rollout_is_max:2.398716926574707 - rollout_corr/rollout_is_min:0.06967287510633469 - rollout_corr/rollout_is_std:0.02976113185286522 - rollout_corr/rollout_is_eff_sample_size:0.9991149397948247 - rollout_corr/rollout_is_seq_mean:0.9999794363975525 - rollout_corr/rollout_is_seq_std:0.0017344678053632379 - rollout_corr/rollout_is_seq_max:1.0062588453292847 - rollout_corr/rollout_is_seq_min:0.9945258498191833 - rollout_corr/rollout_is_seq_max_deviation:0.006258845329284668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.093379558995366) - rollout_corr/training_log_ppl:np.float64(0.08759709404694149) - rollout_corr/kl:np.float64(0.000628665536446249) - rollout_corr/k3_kl:np.float64(0.0005191412595024758) - rollout_corr/rollout_ppl:np.float64(1.0926776281557977) - rollout_corr/rollout_log_ppl:np.float64(0.08696842801145976) - rollout_corr/log_ppl_diff:np.float64(0.0006286660354817286) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013417766604106873) - rollout_corr/log_ppl_diff_max:np.float64(0.00907956063747406) - rollout_corr/log_ppl_diff_min:np.float64(-0.005724966526031494) - rollout_corr/ppl_ratio:np.float64(1.0006304366979748) - rollout_corr/chi2_token:np.float64(0.000801675021648407) - rollout_corr/chi2_seq:np.float64(0.10571352113038301) - actor/pg_loss:np.float64(0.0008303822251036763) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003051055082323728) - actor/ppo_kl:np.float64(0.00013278382333581362) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(32778.418274484575) - self_distillation/token_reweight_w_std:np.float64(637282.1950311251) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001593332272023) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02661549543699948) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.16087789833545685) - perf/mfu/actor:np.float64(0.03977215253160561) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.22827196121216) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006000949069857597 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006000949069857597 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:419.85546875 - response_length/max:1305.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:419.85546875 - response_length_non_aborted/max:1305.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011768564581871033 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7680980022996664) - timing_s/agent_loop/generate_sequences/max:np.float64(9.386873960494995) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.251178185098979) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.386873960494995) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:1305 - timing_s/gen:15.361656874418259 - timing_s/reward:0.06246807053685188 - timing_s/old_log_prob:2.382600510492921 - timing_s/adv:0.5709490403532982 - timing_s/update_actor:15.436897011473775 - timing_s/step:33.90267143212259 - timing_s/stop_profile:0.00010995566844940186 - timing_per_token_ms/update_actor:0.0874497771478718 - timing_per_token_ms/adv:0.003234417273405155 - timing_per_token_ms/gen:0.14292173529226257 - perf/total_num_tokens:176523 - perf/time_per_step:33.90267143212259 - perf/throughput:650.8447289818342 (TaskRunner pid=2074004) Training Progress: 98%|█████████▊| 98/100 [1:21:08<01:16, 38.07s/it] (TaskRunner pid=2074004) step:99 - global_seqlen/min:17883 - global_seqlen/max:26736 - global_seqlen/minmax_diff:8853 - global_seqlen/balanced_min:21621 - global_seqlen/balanced_max:21644 - global_seqlen/mean:21636.5 - actor/entropy:0.10580387711524963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4862961173057556 - training/rollout_probs_diff_mean:0.002484919736161828 - training/rollout_probs_diff_std:0.0104207843542099 - training/rollout_actor_probs_pearson_corr:0.9981136918067932 - self_distillation/success_group_fraction:0.96875 - self_distillation/success_sample_fraction:0.96875 - self_distillation/correct_sample_fraction:0.90234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.96875 - rollout_corr/rollout_is_mean:0.9999649524688721 - rollout_corr/rollout_is_ratio_fraction_high:9.457158739678562e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.674295607605018e-05 - rollout_corr/rollout_is_max:2.7389602661132812 - rollout_corr/rollout_is_min:0.13493947684764862 - rollout_corr/rollout_is_std:0.028364475816488266 - rollout_corr/rollout_is_eff_sample_size:0.9991959842570144 - rollout_corr/rollout_is_seq_mean:0.9998395442962646 - rollout_corr/rollout_is_seq_std:0.001808670349419117 - rollout_corr/rollout_is_seq_max:1.007258415222168 - rollout_corr/rollout_is_seq_min:0.9866185188293457 - rollout_corr/rollout_is_seq_max_deviation:0.013381481170654297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0939670382067561) - rollout_corr/training_log_ppl:np.float64(0.08732558643532684) - rollout_corr/kl:np.float64(0.000521703354597669) - rollout_corr/k3_kl:np.float64(0.00046708716944898754) - rollout_corr/rollout_ppl:np.float64(1.0933743044734001) - rollout_corr/rollout_log_ppl:np.float64(0.0868038823173265) - rollout_corr/log_ppl_diff:np.float64(0.0005217041180003434) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012858145564678125) - rollout_corr/log_ppl_diff_max:np.float64(0.015529707074165344) - rollout_corr/log_ppl_diff_min:np.float64(-0.006662759929895401) - rollout_corr/ppl_ratio:np.float64(1.000523641007021) - rollout_corr/chi2_token:np.float64(0.0009014918468892574) - rollout_corr/chi2_seq:np.float64(0.2561511362437159) - actor/pg_loss:np.float64(0.0010093091987073421) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018811171594279585) - actor/ppo_kl:np.float64(-4.058463273981516e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.96875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.96875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_w_mean:np.float64(28957.405509632314) - self_distillation/token_reweight_w_std:np.float64(466637.31905894005) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997467994689941) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0230792346119415) - self_distillation/empty_target_batch:np.float64(0.03125) - actor/grad_norm:np.float64(0.24430519342422485) - perf/mfu/actor:np.float64(0.039178741434394745) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.19931030273438) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.90234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.90234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005199073348194361 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005199073348194361 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:413.046875 - response_length/max:1520.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:413.046875 - response_length_non_aborted/max:1520.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013734027743339539 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8036548234522343) - timing_s/agent_loop/generate_sequences/max:np.float64(10.340177113190293) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.11781918026827) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.340177113190293) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:313 - timing_s/agent_loop/slowest/response_length:1520 - timing_s/gen:16.068710835650563 - timing_s/reward:0.06060960330069065 - timing_s/old_log_prob:2.314846996217966 - timing_s/adv:0.6209873277693987 - timing_s/update_actor:15.458110202103853 - timing_s/step:34.610270539298654 - timing_s/stop_profile:0.0001095551997423172 - timing_per_token_ms/update_actor:0.08930574608938514 - timing_per_token_ms/adv:0.0035876142616030706 - timing_per_token_ms/gen:0.15196435441318862 - perf/total_num_tokens:173092 - perf/time_per_step:34.610270539298654 - perf/throughput:625.1468036180928 (TaskRunner pid=2074004) Training Progress: 99%|█████████▉| 99/100 [1:21:42<00:37, 37.05s/it] (TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2074004) validation generation end (TaskRunner pid=2074004) [prompt] system (TaskRunner pid=2074004) (TaskRunner pid=2074004) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) ... (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2074004) (TaskRunner pid=2074004) A (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) user (TaskRunner pid=2074004) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2074004) (TaskRunner pid=2074004) A: 50 pC (TaskRunner pid=2074004) B: 56 pC (TaskRunner pid=2074004) C: 60 pC (TaskRunner pid=2074004) D: 64 pC (TaskRunner pid=2074004) Please reason step by step. (TaskRunner pid=2074004) assistant (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) [response] (TaskRunner pid=2074004) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) E = k \frac{q}{r^2} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Solving for $ q $: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Substituting the given values: (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2074004) (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2074004) $$ (TaskRunner pid=2074004) (TaskRunner pid=2074004) The closest answer is B: 56 pC. (TaskRunner pid=2074004) (TaskRunner pid=2074004) (TaskRunner pid=2074004) B (TaskRunner pid=2074004) (TaskRunner pid=2074004) [ground_truth] B (TaskRunner pid=2074004) [score] 1.0 (TaskRunner pid=2074004) [acc] 1.0 (TaskRunner pid=2074004) [pred] B (TaskRunner pid=2074004) [incorrect_format] 1 (TaskRunner pid=2074004) [feedback] (TaskRunner pid=2074004) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2074385) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2074004) step:100 - global_seqlen/min:14131 - global_seqlen/max:24831 - global_seqlen/minmax_diff:10700 - global_seqlen/balanced_min:20764 - global_seqlen/balanced_max:20778 - global_seqlen/mean:20773.375 - actor/entropy:0.10674719512462616 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6354759931564331 - training/rollout_probs_diff_mean:0.0026373525615781546 - training/rollout_probs_diff_std:0.011061684228479862 - training/rollout_actor_probs_pearson_corr:0.9978691935539246 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0000841617584229 - rollout_corr/rollout_is_ratio_fraction_high:3.084801210206933e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.169602420413867e-05 - rollout_corr/rollout_is_max:3.1517021656036377 - rollout_corr/rollout_is_min:0.13044416904449463 - rollout_corr/rollout_is_std:0.03166119009256363 - rollout_corr/rollout_is_eff_sample_size:0.998998810886964 - rollout_corr/rollout_is_seq_mean:1.0001347064971924 - rollout_corr/rollout_is_seq_std:0.0022613920737057924 - rollout_corr/rollout_is_seq_max:1.0161210298538208 - rollout_corr/rollout_is_seq_min:0.9922299981117249 - rollout_corr/rollout_is_seq_max_deviation:0.0161210298538208 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1050673802383244) - rollout_corr/training_log_ppl:np.float64(0.09719672136634472) - rollout_corr/kl:np.float64(0.00043732599254298066) - rollout_corr/k3_kl:np.float64(0.0005812929390707211) - rollout_corr/rollout_ppl:np.float64(1.1045606597326696) - rollout_corr/rollout_log_ppl:np.float64(0.09675939454609761) - rollout_corr/log_ppl_diff:np.float64(0.0004373268202471081) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015722126299806405) - rollout_corr/log_ppl_diff_max:np.float64(0.009256497025489807) - rollout_corr/log_ppl_diff_min:np.float64(-0.006794825196266174) - rollout_corr/ppl_ratio:np.float64(1.0004396380390972) - rollout_corr/chi2_token:np.float64(0.0014464769046753645) - rollout_corr/chi2_seq:np.float64(0.31380061944946647) - actor/pg_loss:np.float64(0.0012798310490325093) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005447886960610049) - actor/ppo_kl:np.float64(1.8937110884076347e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(181507.5251128471) - self_distillation/token_reweight_w_std:np.float64(3188081.7809230387) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007254967931658) - self_distillation/token_reweight_w_clip_frac:np.float64(0.029081717497319914) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.18284212984144688) - perf/mfu/actor:np.float64(0.038138719584986394) - perf/max_memory_allocated_gb:np.float64(125.17703342437744) - perf/max_memory_reserved_gb:np.float64(131.6875) - perf/cpu_memory_used_gb:np.float64(145.18253707885742) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7109375) - val-aux/sciknoweval/reward/std@16:np.float64(0.16604232971456567) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.78) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1094670184264562) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.645725) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.16903151272091094) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7134375000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16525033629837935) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8191750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0603992963721259) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.57575) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15001400456735414) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.733725) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1297479179197353) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8390250000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.02670945685411765) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5126625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12034852442580153) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7460875) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0934240016452198) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8469249999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.009961583262996359) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4606) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.08389710095714561) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7487375000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07219685215265889) - val-aux/sciknoweval/score/mean@16:np.float64(0.7109375) - val-aux/sciknoweval/score/std@16:np.float64(0.16604232971456567) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.78) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1094670184264562) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.645725) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.16903151272091094) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7134375000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16525033629837935) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8191750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0603992963721259) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.57575) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15001400456735414) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.733725) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1297479179197353) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8390250000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.02670945685411765) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5126625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.12034852442580153) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7460875) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.0934240016452198) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8469249999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.009961583262996359) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4606) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.08389710095714561) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7487375000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07219685215265889) - val-core/sciknoweval/acc/mean@16:np.float64(0.7109375) - val-aux/sciknoweval/acc/std@16:np.float64(0.16604232971456567) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.78) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1094670184264562) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.645725) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.16903151272091094) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7134375000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16525033629837935) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8191750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0603992963721259) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.57575) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15001400456735414) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.733725) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1297479179197353) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8390250000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.02670945685411765) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5126625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.12034852442580153) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7460875) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.0934240016452198) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8469249999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.009961583262996359) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4606) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.08389710095714561) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7487375000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07219685215265889) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9984375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.006051536478449089) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.001440128696321335) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9969625000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.008153644123697198) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9985125) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.005913422059734724) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9942624999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.010511931298848583) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9997125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.002649136032174741) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9901125000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.012221532639586832) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999874999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00039508701573197775) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9841000000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.012028687586990721) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031310732010751963 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0031310732010751963 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:379.88671875 - response_length/max:1422.0 - response_length/min:75.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:379.88671875 - response_length_non_aborted/max:1422.0 - response_length_non_aborted/min:75.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011692196130752563 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5598204713314772) - timing_s/agent_loop/generate_sequences/max:np.float64(9.012866754084826) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3422235102189006) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.012866754084826) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1422 - timing_s/gen:14.940290039405227 - timing_s/reward:0.05983664467930794 - timing_s/old_log_prob:2.2800443712621927 - timing_s/adv:0.5855069477111101 - timing_s/update_actor:15.32030195184052 - timing_s/step:33.273349760100245 - timing_s/testing:83.49365962296724 - timing_s/save_checkpoint:6.64394349232316 - timing_s/stop_profile:0.0001235567033290863 - timing_per_token_ms/update_actor:0.09218712626042061 - timing_per_token_ms/adv:0.0035231814023426026 - timing_per_token_ms/gen:0.1536260813709394 - perf/total_num_tokens:166187 - perf/time_per_step:33.273349760100245 - perf/throughput:624.3247268392076 (TaskRunner pid=2074004) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2074004) "np.float64(0.7109375), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2074004) "np.float64(0.16604232971456567), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2074004) "np.float64(0.78), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2074004) "np.float64(0.1094670184264562), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2074004) "np.float64(0.645725), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2074004) "np.float64(0.16903151272091094), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2074004) "np.float64(0.7134375000000001), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2074004) "np.float64(0.16525033629837935), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2074004) "np.float64(0.8191750000000001), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2074004) "np.float64(0.0603992963721259), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2074004) "np.float64(0.57575), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2074004) "np.float64(0.15001400456735414), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2074004) "np.float64(0.733725), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2074004) "np.float64(0.1297479179197353), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2074004) "np.float64(0.8390250000000001), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2074004) "np.float64(0.02670945685411765), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2074004) "np.float64(0.5126625), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2074004) "np.float64(0.12034852442580153), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2074004) "np.float64(0.7460875), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2074004) "np.float64(0.0934240016452198), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2074004) "np.float64(0.8469249999999999), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2074004) 'np.float64(0.009961583262996359), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.4606), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.08389710095714561), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7487375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.07219685215265889), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/mean@16': np.float64(0.7109375), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/std@16': np.float64(0.16604232971456567), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.78), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.1094670184264562), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.645725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.16903151272091094), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.7134375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.16525033629837935), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.8191750000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.0603992963721259), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.57575), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.15001400456735414), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.733725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.1297479179197353), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8390250000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.02670945685411765), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.5126625), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.12034852442580153), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.7460875), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.0934240016452198), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8469249999999999), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.009961583262996359), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.4606), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.08389710095714561), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7487375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.07219685215265889), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/mean@16': np.float64(0.7109375), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/std@16': np.float64(0.16604232971456567), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.78), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.1094670184264562), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.645725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.16903151272091094), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.7134375000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.16525033629837935), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.8191750000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.0603992963721259), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.57575), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.15001400456735414), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.733725), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.1297479179197353), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8390250000000001), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.02670945685411765), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.5126625), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.12034852442580153), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.7460875), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.0934240016452198), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8469249999999999), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.009961583262996359), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.4606), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.08389710095714561), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7487375000000001), " (TaskRunner pid=2074004) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.07219685215265889), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9984375), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2074004) 'np.float64(0.006051536478449089), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.9999125), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2074004) 'np.float64(0.001440128696321335), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2074004) 'np.float64(0.9969625000000001), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2074004) 'np.float64(0.008153644123697198), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9985125), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2074004) 'np.float64(0.005913422059734724), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2074004) 'np.float64(0.9942624999999999), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2074004) 'np.float64(0.010511931298848583), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(0.9997125), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2074004) 'np.float64(0.002649136032174741), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2074004) 'np.float64(0.9901125000000001), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2074004) 'np.float64(0.012221532639586832), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2074004) 'np.float64(0.9999874999999999), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2074004) 'np.float64(0.00039508701573197775), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2074004) 'np.float64(0.9841000000000001), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2074004) 'np.float64(0.012028687586990721), ' (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2074004) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2074004) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2074004) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2074004) Training Progress: 100%|██████████| 100/100 [1:23:46<00:00, 62.98s/it] Training Progress: 100%|██████████| 100/100 [1:23:46<00:00, 50.26s/it] (TaskRunner pid=2074004) wandb: updating run metadata (TaskRunner pid=2074004) wandb: updating run metadata; uploading output.log; uploading wandb-summary.json (TaskRunner pid=2074004) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2074004) wandb: uploading history steps 98-99, summary, console lines 791-912 (TaskRunner pid=2074004) wandb: (TaskRunner pid=2074004) wandb: Run history: (TaskRunner pid=2074004) wandb: actor/entropy ▇▆█▇▇█▅▆▄▃▂▆▃▂▄▁▄▃▂▁▂▁▃▄▄▅▃▅▃▄▆▄▄▆▄▄▅▆▃▄ (TaskRunner pid=2074004) wandb: actor/grad_norm ▇▃█▂▃▃▅▄▅▄▃▂▄▃▃▃▄▄▂▃▃▃▂▃▃▂▂▃▁▂▃▂▂▂▂▃▂▄▁▁ (TaskRunner pid=2074004) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2074004) wandb: actor/lr ▁▂▃█████████████████████████████████████ (TaskRunner pid=2074004) wandb: actor/pg_clipfrac ▁▄▄▅▄▄▄▄▅▇▅▆▄▄▄▆▄▆▃▃▅▅▅▆▅▄▆▅▄▃▂▂▅▃▄▂█▆▄▄ (TaskRunner pid=2074004) wandb: actor/pg_clipfrac_lower ▁▅▁▁▁▁▁█▅▁▁▁▃▁▆▁▁▁▁▃▃▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2074004) wandb: actor/pg_loss ▄▇█▄▇▅▇▇▅▅▄▆▆▅▄▄▆▄▄▆▄▃▂▃▄▃▂▂▃▃▄▄▄▃▅▁▃▃▅▂ (TaskRunner pid=2074004) wandb: actor/ppo_kl ▁▄▂▂▃▇▂▂▄▂▄▄▃▃▃▅▃▄▃▄▇▃▃▄▃▄▃▄▃█▃▃▅▄▄▃▄▃▄▃ (TaskRunner pid=2074004) wandb: critic/advantages/max ▇███▇████████▇██▅█▅▇██▇███▁██▇▇████▇███▇ (TaskRunner pid=2074004) wandb: critic/advantages/mean ▇▃▃▅▄▃▅▆▇▆▅█▃▆▃▃▃▄▃▃▅▆▇▄▁▇▆▇▃▅▅▅▇▇▆▅▅▄▅▅ (TaskRunner pid=2074004) wandb: +220 ... (TaskRunner pid=2074004) wandb: (TaskRunner pid=2074004) wandb: Run summary: (TaskRunner pid=2074004) wandb: actor/entropy 0.10675 (TaskRunner pid=2074004) wandb: actor/grad_norm 0.18284 (TaskRunner pid=2074004) wandb: actor/kl_loss 0 (TaskRunner pid=2074004) wandb: actor/lr 0.0 (TaskRunner pid=2074004) wandb: actor/pg_clipfrac 0.00054 (TaskRunner pid=2074004) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2074004) wandb: actor/pg_loss 0.00128 (TaskRunner pid=2074004) wandb: actor/ppo_kl 2e-05 (TaskRunner pid=2074004) wandb: critic/advantages/max 0.875 (TaskRunner pid=2074004) wandb: critic/advantages/mean -0.00313 (TaskRunner pid=2074004) wandb: +220 ... (TaskRunner pid=2074004) wandb: (TaskRunner pid=2074004) wandb: 🚀 View run qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/eajdt9hv (TaskRunner pid=2074004) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2074004) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2074004) wandb: Find logs at: ./wandb/run-20260702_085822-eajdt9hv/logs (TaskRunner pid=2074004) Exception ignored in atexit callback: .teardown_atexit at 0x7f1bd03c7880> (TaskRunner pid=2074004) Traceback (most recent call last): (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2074004) conn.teardown(hooks.exit_code) (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2074004) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2074004) return future.result() (TaskRunner pid=2074004) ^^^^^^^^^^^^^^^ (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2074004) return self.__get_result() (TaskRunner pid=2074004) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2074004) raise self._exception (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2074004) return await fn() (TaskRunner pid=2074004) ^^^^^^^^^^ (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2074004) await self._client.close() (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2074004) await self._writer.wait_closed() (TaskRunner pid=2074004) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2074004) await self._protocol._get_close_waiter(self) (TaskRunner pid=2074004) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 10:22:16] Finished physics rlsd_tr [2026-07-02 10:22:16] Starting biology grpo Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: biology Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet Ray tmp: /tmp/ray_q3g_biology_grpo_Qwen3_4B 2026-07-02 10:22:17,959 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,960 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,960 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,960 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,961 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,961 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,962 VINFO scripts.py:1286 -- Killed `ray::IDLE "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""` (via SIGKILL) 2026-07-02 10:22:17,970 VINFO scripts.py:1286 -- Killed `/mnt/mole/SDPO/L2T/.venv/bin/python -u /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/runtime_env/agent/main.py --node-ip-address=198.19.44.212 --runtime-env-agent-port=56317 --gcs-address=198.19.44.212:57594 --cluster-id-hex=1a1f12b856019ab7823cf8c9c76e73dc7eaa5439177dd16f1d3c0b76 --runtime-env-dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/runtime_resources --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --log-dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/logs --temp-dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B --stdout-filepath=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/logs/runtime_env_agent.out --stderr-filepath=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B/session_2026-07-02_08-56-26_561774_2071629/logs/runtime_env_agent.err` (via SIGKILL) 2026-07-02 10:22:25,626 WARN scripts.py:1369 -- Stopped only 0 out of 8 Ray processes within the grace period 16 seconds. Set `-v` to see more details. Remaining processes [psutil.Process(pid=2074225, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072135, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072138, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072091, name='python', status='zombie'), psutil.Process(pid=2072147, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072134, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072141, name='ray::IDLE', status='zombie'), psutil.Process(pid=2072137, name='ray::IDLE', status='zombie')] will be forcefully terminated. 2026-07-02 10:22:25,626 WARN scripts.py:1376 -- You can also use `--force` to forcefully terminate processes or set higher `--grace-period` to wait longer time for proper termination. Experiment: qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/biology Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_biology_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/biology +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/biology', 'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 10:22:36,329 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2099163) TaskRunner hostname: mole-workspace-rw, PID: 2099163 (TaskRunner pid=2099163) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2099163) 'calculate_entropy': False, (TaskRunner pid=2099163) 'calculate_sum_pi_squared': False, (TaskRunner pid=2099163) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2099163) 'async_save': False, (TaskRunner pid=2099163) 'load_contents': ['model', (TaskRunner pid=2099163) 'optimizer', (TaskRunner pid=2099163) 'extra'], (TaskRunner pid=2099163) 'save_contents': ['model', (TaskRunner pid=2099163) 'optimizer', (TaskRunner pid=2099163) 'extra']}, (TaskRunner pid=2099163) 'clip_ratio': 0.2, (TaskRunner pid=2099163) 'clip_ratio_c': 3.0, (TaskRunner pid=2099163) 'clip_ratio_high': 0.28, (TaskRunner pid=2099163) 'clip_ratio_low': 0.2, (TaskRunner pid=2099163) 'data_loader_seed': 42, (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_coeff': 0, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'freeze_vision_tower': False, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'forward_only': False, (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'full_determinism': False, (TaskRunner pid=2099163) 'model_dtype': 'fp32', (TaskRunner pid=2099163) 'offload_policy': False, (TaskRunner pid=2099163) 'optimizer_offload': False, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'seed': 42, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_orig_params': False, (TaskRunner pid=2099163) 'use_torch_compile': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'grad_clip': 1.0, (TaskRunner pid=2099163) 'kl_loss_coef': 0.0, (TaskRunner pid=2099163) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2099163) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2099163) 'loss_scale_factor': None, (TaskRunner pid=2099163) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2099163) 'betas': [0.9, 0.999], (TaskRunner pid=2099163) 'clip_grad': 1.0, (TaskRunner pid=2099163) 'lr': 1e-06, (TaskRunner pid=2099163) 'lr_scheduler_type': 'constant', (TaskRunner pid=2099163) 'lr_warmup_steps': 10, (TaskRunner pid=2099163) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2099163) 'min_lr_ratio': 0.0, (TaskRunner pid=2099163) 'num_cycles': 0.5, (TaskRunner pid=2099163) 'optimizer': 'AdamW', (TaskRunner pid=2099163) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2099163) 'override_optimizer_config': None, (TaskRunner pid=2099163) 'total_training_steps': -1, (TaskRunner pid=2099163) 'warmup_style': None, (TaskRunner pid=2099163) 'weight_decay': 0.01}, (TaskRunner pid=2099163) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2099163) 'clip_cov_lb': 1.0, (TaskRunner pid=2099163) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2099163) 'clip_cov_ub': 5.0, (TaskRunner pid=2099163) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2099163) 'loss_mode': 'vanilla', (TaskRunner pid=2099163) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2099163) 'ppo_epochs': 1, (TaskRunner pid=2099163) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2099163) 'ppo_micro_batch_size': None, (TaskRunner pid=2099163) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2099163) 'ppo_mini_batch_size': 8, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'rollout_n': 8, (TaskRunner pid=2099163) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2099163) 'mode': 'disabled', (TaskRunner pid=2099163) 'record_file': None, (TaskRunner pid=2099163) 'replay_file': None}, (TaskRunner pid=2099163) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2099163) 'alpha': 0.5, (TaskRunner pid=2099163) 'distillation_add_tail': True, (TaskRunner pid=2099163) 'distillation_topk': 100, (TaskRunner pid=2099163) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2099163) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2099163) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'loss_weight': 0.0, (TaskRunner pid=2099163) 'mask': 'all'}, (TaskRunner pid=2099163) 'feedback_template': '\n' (TaskRunner pid=2099163) 'The ' (TaskRunner pid=2099163) 'following ' (TaskRunner pid=2099163) 'is ' (TaskRunner pid=2099163) 'feedback ' (TaskRunner pid=2099163) 'from ' (TaskRunner pid=2099163) 'your ' (TaskRunner pid=2099163) 'unsuccessful ' (TaskRunner pid=2099163) 'earlier ' (TaskRunner pid=2099163) 'attempt:\n' (TaskRunner pid=2099163) '\n' (TaskRunner pid=2099163) '{feedback_raw}', (TaskRunner pid=2099163) 'full_logit_distillation': True, (TaskRunner pid=2099163) 'include_environment_feedback': True, (TaskRunner pid=2099163) 'is_clip': 2, (TaskRunner pid=2099163) 'max_reprompt_len': 22528, (TaskRunner pid=2099163) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2099163) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2099163) '\n' (TaskRunner pid=2099163) 'Correctly ' (TaskRunner pid=2099163) 'solve ' (TaskRunner pid=2099163) 'the ' (TaskRunner pid=2099163) 'original ' (TaskRunner pid=2099163) 'question.', (TaskRunner pid=2099163) 'reprompt_truncation': 'right', (TaskRunner pid=2099163) 'solution_template': '\n' (TaskRunner pid=2099163) 'Correct ' (TaskRunner pid=2099163) 'solution:\n' (TaskRunner pid=2099163) '\n' (TaskRunner pid=2099163) '{successful_previous_attempt}', (TaskRunner pid=2099163) 'srpo_dynamic_weighting': False, (TaskRunner pid=2099163) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2099163) 'success_reward_threshold': 0.5, (TaskRunner pid=2099163) 'teacher_regularization': 'ema', (TaskRunner pid=2099163) 'teacher_update_rate': 0.0, (TaskRunner pid=2099163) 'token_reweight_decay_steps': None, (TaskRunner pid=2099163) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2099163) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2099163) 'shuffle': False, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2099163) 'tau_neg': 1.05, (TaskRunner pid=2099163) 'tau_pos': 1.0, (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_dynamic_bsz': False, (TaskRunner pid=2099163) 'use_fused_kernels': False, (TaskRunner pid=2099163) 'use_kl_loss': False, (TaskRunner pid=2099163) 'use_prefix_grouper': False, (TaskRunner pid=2099163) 'use_remove_padding': True, (TaskRunner pid=2099163) 'use_torch_compile': True}, (TaskRunner pid=2099163) 'hybrid_engine': True, (TaskRunner pid=2099163) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2099163) 'custom_chat_template': None, (TaskRunner pid=2099163) 'enable_activation_offload': False, (TaskRunner pid=2099163) 'enable_gradient_checkpointing': True, (TaskRunner pid=2099163) 'exclude_modules': None, (TaskRunner pid=2099163) 'external_lib': None, (TaskRunner pid=2099163) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2099163) 'hf_config_path': None, (TaskRunner pid=2099163) 'lora_adapter_path': None, (TaskRunner pid=2099163) 'lora_alpha': 16, (TaskRunner pid=2099163) 'lora_rank': 0, (TaskRunner pid=2099163) 'override_config': {}, (TaskRunner pid=2099163) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2099163) 'target_modules': 'all-linear', (TaskRunner pid=2099163) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2099163) 'num_shards': 4}, (TaskRunner pid=2099163) 'tokenizer_path': None, (TaskRunner pid=2099163) 'trust_remote_code': True, (TaskRunner pid=2099163) 'use_fused_kernels': False, (TaskRunner pid=2099163) 'use_liger': False, (TaskRunner pid=2099163) 'use_remove_padding': True, (TaskRunner pid=2099163) 'use_shm': False}, (TaskRunner pid=2099163) 'nccl_timeout': 600, (TaskRunner pid=2099163) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'forward_only': True, (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'full_determinism': False, (TaskRunner pid=2099163) 'model_dtype': 'fp32', (TaskRunner pid=2099163) 'offload_policy': False, (TaskRunner pid=2099163) 'optimizer_offload': False, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'seed': 42, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_orig_params': False, (TaskRunner pid=2099163) 'use_torch_compile': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2099163) 'log_prob_micro_batch_size': None, (TaskRunner pid=2099163) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2099163) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'rollout_n': 8, (TaskRunner pid=2099163) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2099163) 'mode': 'disabled', (TaskRunner pid=2099163) 'record_file': None, (TaskRunner pid=2099163) 'replay_file': None}, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_torch_compile': True}, (TaskRunner pid=2099163) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2099163) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2099163) 'agent_loop_config_path': None, (TaskRunner pid=2099163) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2099163) 'name': None, (TaskRunner pid=2099163) 'path': None}, (TaskRunner pid=2099163) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2099163) 'num_workers': 8}, (TaskRunner pid=2099163) 'calculate_log_probs': True, (TaskRunner pid=2099163) 'cudagraph_capture_sizes': None, (TaskRunner pid=2099163) 'data_parallel_size': 1, (TaskRunner pid=2099163) 'disable_log_stats': True, (TaskRunner pid=2099163) 'do_sample': True, (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'enable_chunked_prefill': True, (TaskRunner pid=2099163) 'enable_prefix_caching': True, (TaskRunner pid=2099163) 'enable_rollout_routing_replay': False, (TaskRunner pid=2099163) 'enforce_eager': False, (TaskRunner pid=2099163) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2099163) 'expert_parallel_size': 1, (TaskRunner pid=2099163) 'free_cache_engine': True, (TaskRunner pid=2099163) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2099163) 'ignore_eos': False, (TaskRunner pid=2099163) 'layered_summon': False, (TaskRunner pid=2099163) 'load_format': 'dummy', (TaskRunner pid=2099163) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2099163) 'log_prob_micro_batch_size': None, (TaskRunner pid=2099163) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2099163) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2099163) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2099163) 'max_model_len': 10240, (TaskRunner pid=2099163) 'max_num_batched_tokens': 10240, (TaskRunner pid=2099163) 'max_num_seqs': 1024, (TaskRunner pid=2099163) 'mode': 'async', (TaskRunner pid=2099163) 'multi_stage_wake_up': False, (TaskRunner pid=2099163) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'format': 'hermes', (TaskRunner pid=2099163) 'interaction_config_path': None, (TaskRunner pid=2099163) 'max_assistant_turns': None, (TaskRunner pid=2099163) 'max_parallel_calls': 1, (TaskRunner pid=2099163) 'max_tool_response_length': 256, (TaskRunner pid=2099163) 'max_user_turns': None, (TaskRunner pid=2099163) 'num_repeat_rollouts': None, (TaskRunner pid=2099163) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2099163) 'tool_config_path': None, (TaskRunner pid=2099163) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2099163) 'use_inference_chat_template': False}, (TaskRunner pid=2099163) 'n': 8, (TaskRunner pid=2099163) 'name': 'vllm', (TaskRunner pid=2099163) 'over_sample_rate': 0, (TaskRunner pid=2099163) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2099163) 'port': 9090, (TaskRunner pid=2099163) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2099163) 'prompt_length': 2048, (TaskRunner pid=2099163) 'quantization': None, (TaskRunner pid=2099163) 'quantization_config_file': None, (TaskRunner pid=2099163) 'response_length': 8192, (TaskRunner pid=2099163) 'scheduling_policy': 'fcfs', (TaskRunner pid=2099163) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2099163) 'skip_rollout': False, (TaskRunner pid=2099163) 'skip_tokenizer_init': True, (TaskRunner pid=2099163) 'temperature': 1.0, (TaskRunner pid=2099163) 'tensor_model_parallel_size': 2, (TaskRunner pid=2099163) 'top_k': -1, (TaskRunner pid=2099163) 'top_p': 1.0, (TaskRunner pid=2099163) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2099163) 'backend': None, (TaskRunner pid=2099163) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2099163) 'token2text': False}, (TaskRunner pid=2099163) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2099163) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2099163) 'do_sample': True, (TaskRunner pid=2099163) 'n': 16, (TaskRunner pid=2099163) 'temperature': 0.6, (TaskRunner pid=2099163) 'top_k': -1, (TaskRunner pid=2099163) 'top_p': 0.95}}}, (TaskRunner pid=2099163) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2099163) 'adv_estimator': 'grpo', (TaskRunner pid=2099163) 'gamma': 1.0, (TaskRunner pid=2099163) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2099163) 'horizon': 10000, (TaskRunner pid=2099163) 'kl_coef': 0.001, (TaskRunner pid=2099163) 'target_kl': 0.1, (TaskRunner pid=2099163) 'type': 'fixed'}, (TaskRunner pid=2099163) 'kl_penalty': 'kl', (TaskRunner pid=2099163) 'lam': 1.0, (TaskRunner pid=2099163) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2099163) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2099163) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2099163) 'loss_type': 'ppo_clip', (TaskRunner pid=2099163) 'rollout_is': 'token', (TaskRunner pid=2099163) 'rollout_is_batch_normalize': False, (TaskRunner pid=2099163) 'rollout_is_threshold': 2.0, (TaskRunner pid=2099163) 'rollout_rs': None, (TaskRunner pid=2099163) 'rollout_rs_threshold': None}, (TaskRunner pid=2099163) 'use_kl_in_reward': False, (TaskRunner pid=2099163) 'use_pf_ppo': False}, (TaskRunner pid=2099163) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2099163) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2099163) 'async_save': False, (TaskRunner pid=2099163) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2099163) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2099163) 'cliprange_value': 0.5, (TaskRunner pid=2099163) 'data_loader_seed': 42, (TaskRunner pid=2099163) 'enable': None, (TaskRunner pid=2099163) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2099163) 'forward_micro_batch_size': None, (TaskRunner pid=2099163) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2099163) 'grad_clip': 1.0, (TaskRunner pid=2099163) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2099163) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2099163) 'enable_activation_offload': False, (TaskRunner pid=2099163) 'enable_gradient_checkpointing': True, (TaskRunner pid=2099163) 'external_lib': None, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'entropy_checkpointing': False, (TaskRunner pid=2099163) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2099163) 'forward_only': False, (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'full_determinism': False, (TaskRunner pid=2099163) 'model_dtype': 'fp32', (TaskRunner pid=2099163) 'offload_policy': False, (TaskRunner pid=2099163) 'optimizer_offload': False, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'seed': 42, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_orig_params': False, (TaskRunner pid=2099163) 'use_torch_compile': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'lora_alpha': 16, (TaskRunner pid=2099163) 'lora_rank': 0, (TaskRunner pid=2099163) 'override_config': {}, (TaskRunner pid=2099163) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2099163) 'target_modules': 'all-linear', (TaskRunner pid=2099163) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2099163) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2099163) 'trust_remote_code': True, (TaskRunner pid=2099163) 'use_remove_padding': False, (TaskRunner pid=2099163) 'use_shm': False}, (TaskRunner pid=2099163) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2099163) 'betas': [0.9, 0.999], (TaskRunner pid=2099163) 'clip_grad': 1.0, (TaskRunner pid=2099163) 'lr': 1e-05, (TaskRunner pid=2099163) 'lr_scheduler_type': 'constant', (TaskRunner pid=2099163) 'lr_warmup_steps': -1, (TaskRunner pid=2099163) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2099163) 'min_lr_ratio': 0.0, (TaskRunner pid=2099163) 'num_cycles': 0.5, (TaskRunner pid=2099163) 'optimizer': 'AdamW', (TaskRunner pid=2099163) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2099163) 'override_optimizer_config': None, (TaskRunner pid=2099163) 'total_training_steps': -1, (TaskRunner pid=2099163) 'warmup_style': None, (TaskRunner pid=2099163) 'weight_decay': 0.01}, (TaskRunner pid=2099163) 'ppo_epochs': 1, (TaskRunner pid=2099163) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2099163) 'ppo_micro_batch_size': None, (TaskRunner pid=2099163) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2099163) 'ppo_mini_batch_size': 8, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'rollout_n': 8, (TaskRunner pid=2099163) 'shuffle': False, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_dynamic_bsz': False}, (TaskRunner pid=2099163) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2099163) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2099163) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2099163) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2099163) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2099163) 'dataloader_num_workers': 8, (TaskRunner pid=2099163) 'filter_overlong_prompts': True, (TaskRunner pid=2099163) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2099163) 'image_key': 'images', (TaskRunner pid=2099163) 'image_patch_size': 14, (TaskRunner pid=2099163) 'max_prompt_length': 2048, (TaskRunner pid=2099163) 'max_response_length': 8192, (TaskRunner pid=2099163) 'prompt_key': 'prompt', (TaskRunner pid=2099163) 'return_full_prompt': False, (TaskRunner pid=2099163) 'return_multi_modal_inputs': True, (TaskRunner pid=2099163) 'return_raw_chat': True, (TaskRunner pid=2099163) 'return_raw_input_ids': False, (TaskRunner pid=2099163) 'reward_fn_key': 'data_source', (TaskRunner pid=2099163) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2099163) 'seed': None, (TaskRunner pid=2099163) 'shuffle': True, (TaskRunner pid=2099163) 'tokenizer': None, (TaskRunner pid=2099163) 'tool_config_path': None, (TaskRunner pid=2099163) 'train_batch_size': 32, (TaskRunner pid=2099163) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet'], (TaskRunner pid=2099163) 'train_max_samples': 3200, (TaskRunner pid=2099163) 'truncation': 'error', (TaskRunner pid=2099163) 'trust_remote_code': True, (TaskRunner pid=2099163) 'use_shm': False, (TaskRunner pid=2099163) 'val_batch_size': None, (TaskRunner pid=2099163) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet'], (TaskRunner pid=2099163) 'val_max_samples': -1, (TaskRunner pid=2099163) 'validation_shuffle': False, (TaskRunner pid=2099163) 'video_key': 'videos'}, (TaskRunner pid=2099163) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2099163) 'cuda-memory-usage': 'true', (TaskRunner pid=2099163) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2099163) 'capture-range-end': None, (TaskRunner pid=2099163) 'cuda-graph-trace': 'graph', (TaskRunner pid=2099163) 'cuda-memory-usage': 'true', (TaskRunner pid=2099163) 'kill': 'none', (TaskRunner pid=2099163) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2099163) 'torch_memory': {'context': 'all', (TaskRunner pid=2099163) 'kw_args': {}, (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'stacks': 'all', (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2099163) 'profile_continuous_steps': False, (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'steps': None, (TaskRunner pid=2099163) 'tool': None}, (TaskRunner pid=2099163) 'max_model_len': 10240, (TaskRunner pid=2099163) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_biology_grpo_Qwen3_4B', (TaskRunner pid=2099163) 'include_dashboard': False, (TaskRunner pid=2099163) 'num_cpus': None, (TaskRunner pid=2099163) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2099163) 'TASK': 'datasets/sciknoweval/biology', (TaskRunner pid=2099163) 'USER': 'root'}}}, (TaskRunner pid=2099163) 'timeline_json_file': None}, (TaskRunner pid=2099163) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2099163) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2099163) 'name': 'custom_reward_manager', (TaskRunner pid=2099163) 'path': None}, (TaskRunner pid=2099163) 'name': 'naive', (TaskRunner pid=2099163) 'source': 'register'}, (TaskRunner pid=2099163) 'reward_model': {'enable': False, (TaskRunner pid=2099163) 'enable_resource_pool': False, (TaskRunner pid=2099163) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2099163) 'launch_reward_fn_async': False, (TaskRunner pid=2099163) 'max_length': None, (TaskRunner pid=2099163) 'micro_batch_size': None, (TaskRunner pid=2099163) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2099163) 'model': {'external_lib': None, (TaskRunner pid=2099163) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2099163) 'forward_prefetch': False, (TaskRunner pid=2099163) 'fsdp_size': -1, (TaskRunner pid=2099163) 'param_offload': False, (TaskRunner pid=2099163) 'reshard_after_forward': True, (TaskRunner pid=2099163) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2099163) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2099163) 'override_config': {}, (TaskRunner pid=2099163) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2099163) 'trust_remote_code': False, (TaskRunner pid=2099163) 'use_fused_kernels': False, (TaskRunner pid=2099163) 'use_remove_padding': False, (TaskRunner pid=2099163) 'use_shm': False}, (TaskRunner pid=2099163) 'n_gpus_per_node': 8, (TaskRunner pid=2099163) 'nnodes': 0, (TaskRunner pid=2099163) 'num_workers': 1, (TaskRunner pid=2099163) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2099163) 'all_ranks': False, (TaskRunner pid=2099163) 'enable': False, (TaskRunner pid=2099163) 'ranks': [], (TaskRunner pid=2099163) 'save_path': 'outputs/profile', (TaskRunner pid=2099163) 'tool': None, (TaskRunner pid=2099163) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2099163) 'analysis': True, (TaskRunner pid=2099163) 'contents': [], (TaskRunner pid=2099163) 'discrete': False, (TaskRunner pid=2099163) 'level': 'level0'}, (TaskRunner pid=2099163) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2099163) 'discrete': False}, (TaskRunner pid=2099163) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2099163) 'step_end': None, (TaskRunner pid=2099163) 'step_start': 0}, (TaskRunner pid=2099163) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2099163) 'stack_depth': 32, (TaskRunner pid=2099163) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2099163) 'reward_loop_class_name': None, (TaskRunner pid=2099163) 'reward_loop_module_path': None, (TaskRunner pid=2099163) 'reward_loop_source': 'register', (TaskRunner pid=2099163) 'reward_manager': 'naive', (TaskRunner pid=2099163) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2099163) 'cudagraph_capture_sizes': None, (TaskRunner pid=2099163) 'data_parallel_size': 1, (TaskRunner pid=2099163) 'disable_log_stats': True, (TaskRunner pid=2099163) 'dtype': 'bfloat16', (TaskRunner pid=2099163) 'enable_chunked_prefill': True, (TaskRunner pid=2099163) 'enable_prefix_caching': True, (TaskRunner pid=2099163) 'enforce_eager': True, (TaskRunner pid=2099163) 'engine_kwargs': {}, (TaskRunner pid=2099163) 'expert_parallel_size': 1, (TaskRunner pid=2099163) 'free_cache_engine': True, (TaskRunner pid=2099163) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2099163) 'limit_images': None, (TaskRunner pid=2099163) 'load_format': 'auto', (TaskRunner pid=2099163) 'max_model_len': None, (TaskRunner pid=2099163) 'max_num_batched_tokens': 8192, (TaskRunner pid=2099163) 'max_num_seqs': 1024, (TaskRunner pid=2099163) 'name': '???', (TaskRunner pid=2099163) 'prompt_length': 2048, (TaskRunner pid=2099163) 'response_length': 2048, (TaskRunner pid=2099163) 'skip_tokenizer_init': False, (TaskRunner pid=2099163) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2099163) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2099163) 'memory_limit_mb': 1024, (TaskRunner pid=2099163) 'url': None}, (TaskRunner pid=2099163) 'strategy': 'fsdp', (TaskRunner pid=2099163) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2099163) 'use_dynamic_bsz': False, (TaskRunner pid=2099163) 'use_reward_loop': False}, (TaskRunner pid=2099163) 'trainer': {'balance_batch': True, (TaskRunner pid=2099163) 'critic_warmup': 0, (TaskRunner pid=2099163) 'default_hdfs_dir': None, (TaskRunner pid=2099163) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2099163) 'del_local_ckpt_after_load': False, (TaskRunner pid=2099163) 'device': 'cuda', (TaskRunner pid=2099163) 'esi_redundant_time': 0, (TaskRunner pid=2099163) 'experiment_name': 'qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2099163) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2099163) 'log_val_generations': 0, (TaskRunner pid=2099163) 'logger': ['console', 'wandb'], (TaskRunner pid=2099163) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2099163) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2099163) 'n_gpus_per_node': 8, (TaskRunner pid=2099163) 'nnodes': 1, (TaskRunner pid=2099163) 'project_name': 'SDPO-root', (TaskRunner pid=2099163) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2099163) 'resume_from_path': None, (TaskRunner pid=2099163) 'resume_mode': 'auto', (TaskRunner pid=2099163) 'rollout_data_dir': None, (TaskRunner pid=2099163) 'save_freq': 10, (TaskRunner pid=2099163) 'test_freq': 10, (TaskRunner pid=2099163) 'total_epochs': 30, (TaskRunner pid=2099163) 'total_training_steps': 100, (TaskRunner pid=2099163) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2099163) 'val_before_train': False, (TaskRunner pid=2099163) 'val_only': False, (TaskRunner pid=2099163) 'validation_data_dir': None}, (TaskRunner pid=2099163) 'transfer_queue': {'enable': False}, (TaskRunner pid=2099163) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/biology', (TaskRunner pid=2099163) 'dir': '/users/root/SDPO', (TaskRunner pid=2099163) 'log_dir': '/users/root/output', (TaskRunner pid=2099163) 'task': 'datasets/sciknoweval/biology'}} (TaskRunner pid=2099163) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2099163) use_critic=need_critic(config), (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2099163) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2099163) Using dataset class: RLHFDataset (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2099163) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2099163) dataset len: 450 (TaskRunner pid=2099163) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2099163) WARNING:2026-07-02 10:22:45,455:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2099163) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/450 [00:00 (TaskRunner pid=2099163) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2099163) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2099163) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 50/50 [00:00<00:00, 79.25 examples/s] (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2099163) self.use_critic = need_critic(self.config) (WorkerDict pid=2099540) [W702 10:22:53.887126096 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53015 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2099539) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2099546) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2099546) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2099543) [W702 10:22:54.133676817 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53015 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2099539) Model config after override: Qwen3Config { (WorkerDict pid=2099539) "architectures": [ (WorkerDict pid=2099539) "Qwen3ForCausalLM" (WorkerDict pid=2099539) ], (WorkerDict pid=2099539) "attention_bias": false, (WorkerDict pid=2099539) "attention_dropout": 0.0, (WorkerDict pid=2099539) "dtype": "bfloat16", (WorkerDict pid=2099539) "eos_token_id": 151645, (WorkerDict pid=2099539) "head_dim": 128, (WorkerDict pid=2099539) "hidden_act": "silu", (WorkerDict pid=2099539) "hidden_size": 2560, (WorkerDict pid=2099539) "initializer_range": 0.02, (WorkerDict pid=2099539) "intermediate_size": 9728, (WorkerDict pid=2099539) "layer_types": [ (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention", (WorkerDict pid=2099539) "full_attention" (WorkerDict pid=2099539) ], (WorkerDict pid=2099539) "max_position_embeddings": 40960, (WorkerDict pid=2099539) "max_window_layers": 36, (WorkerDict pid=2099539) "model_type": "qwen3", (WorkerDict pid=2099539) "num_attention_heads": 32, (WorkerDict pid=2099539) "num_hidden_layers": 36, (WorkerDict pid=2099539) "num_key_value_heads": 8, (WorkerDict pid=2099539) "pad_token_id": 151643, (WorkerDict pid=2099539) "rms_norm_eps": 1e-06, (WorkerDict pid=2099539) "rope_scaling": null, (WorkerDict pid=2099539) "rope_theta": 1000000, (WorkerDict pid=2099539) "sliding_window": null, (WorkerDict pid=2099539) "tie_word_embeddings": true, (WorkerDict pid=2099539) "transformers_version": "4.57.1", (WorkerDict pid=2099539) "use_cache": true, (WorkerDict pid=2099539) "use_sliding_window": false, (WorkerDict pid=2099539) "vocab_size": 151936 (WorkerDict pid=2099539) } (WorkerDict pid=2099539) (WorkerDict pid=2099540) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2099540) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2099540) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2099546) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2099539) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2099540) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2099540) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2099539) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.93s/it] [repeated 14x across cluster] (WorkerDict pid=2099539) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 6x across cluster] (WorkerDict pid=2099539) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2099539) Actor use_remove_padding=True (WorkerDict pid=2099539) Actor use_fused_kernels=False (WorkerDict pid=2099539) Actor use_prefix_grouper=False (WorkerDict pid=2099543) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2099543) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2099540) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2099540) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2099539) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2099544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2099544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2099542) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2099542) warnings.warn( (WorkerDict pid=2099544) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2099544) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2099163) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2099163) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2099543) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2099543) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2100408) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2100408) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2100407) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2100407) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2100407) ['serve', (vLLMHttpServer pid=2100407) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2100407) '--dtype', (vLLMHttpServer pid=2100407) 'bfloat16', (vLLMHttpServer pid=2100407) '--load_format', (vLLMHttpServer pid=2100407) 'dummy', (vLLMHttpServer pid=2100407) '--trust_remote_code', (vLLMHttpServer pid=2100407) '--max_model_len', (vLLMHttpServer pid=2100407) '40960', (vLLMHttpServer pid=2100407) '--max_num_seqs', (vLLMHttpServer pid=2100407) '1024', (vLLMHttpServer pid=2100407) '--enable_chunked_prefill', (vLLMHttpServer pid=2100407) '--max_num_batched_tokens', (vLLMHttpServer pid=2100407) '10240', (vLLMHttpServer pid=2100407) '--enable_prefix_caching', (vLLMHttpServer pid=2100407) '--enable_sleep_mode', (vLLMHttpServer pid=2100407) '--logprobs_mode', (vLLMHttpServer pid=2100407) 'processed_logprobs', (vLLMHttpServer pid=2100407) '--disable_custom_all_reduce', (vLLMHttpServer pid=2100407) '--gpu_memory_utilization', (vLLMHttpServer pid=2100407) '0.8', (vLLMHttpServer pid=2100407) '--disable_log_stats', (vLLMHttpServer pid=2100407) '--tensor_parallel_size', (vLLMHttpServer pid=2100407) '2', (vLLMHttpServer pid=2100407) '--seed', (vLLMHttpServer pid=2100407) '0', (vLLMHttpServer pid=2100407) '--override_generation_config', (vLLMHttpServer pid=2100407) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2100407) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2100407) '--hf_overrides', (vLLMHttpServer pid=2100407) '{}', (vLLMHttpServer pid=2100407) '--scheduling_policy', (vLLMHttpServer pid=2100407) 'fcfs'] (WorkerDict pid=2099544) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2100408) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2100408) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2100407) WARNING 07-02 10:23:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2099540) WARNING 07-02 10:23:47 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2100409) WARNING 07-02 10:23:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2099541) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2099540) [rank1]:W0702 10:23:58.556000 2099540 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2099541.136359408498368.tmp' (vLLMHttpServer pid=2100410) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2100410) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2100410) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2100410) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2099539) 2026-07-02 10:24:01,345 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2099539) 2026-07-02 10:24:01,370 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2099539) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score of a protein sequence is typically determined through computational methods such as energy minimization, molecular dynamics simulations, or machine learning models trained on known protein structures. However, without access to specific computational tools or databases that can calculate the folding stability score for the given sequence, it is not possible to determine the exact score. The provided options (A: -0.01, B: 1.69, C: 2.49, D: 0.45) are likely hypothetical values, and there is no standard or universally accepted method to calculate this score from the sequence alone. Therefore, the correct answer cannot be determined based on the information provided. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) B (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 1.0 (TaskRunner pid=2099163) [acc] 1.0 (TaskRunner pid=2099163) [pred] B (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2099163) step:10 - global_seqlen/min:12666 - global_seqlen/max:26901 - global_seqlen/minmax_diff:14235 - global_seqlen/balanced_min:18269 - global_seqlen/balanced_max:18382 - global_seqlen/mean:18293.25 - actor/entropy:0.192046120762825 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9020472168922424 - training/rollout_probs_diff_mean:0.00564963323995471 - training/rollout_probs_diff_std:0.0170443132519722 - training/rollout_actor_probs_pearson_corr:0.9967436790466309 - rollout_corr/rollout_is_mean:0.9999064207077026 - rollout_corr/rollout_is_ratio_fraction_high:6.567023228853941e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00027581496397033334 - rollout_corr/rollout_is_max:3.955087184906006 - rollout_corr/rollout_is_min:0.025797156617045403 - rollout_corr/rollout_is_std:0.045984067022800446 - rollout_corr/rollout_is_eff_sample_size:0.997889452616177 - rollout_corr/rollout_is_seq_mean:0.9998351335525513 - rollout_corr/rollout_is_seq_std:0.0035011477302759886 - rollout_corr/rollout_is_seq_max:1.0109554529190063 - rollout_corr/rollout_is_seq_min:0.9882628321647644 - rollout_corr/rollout_is_seq_max_deviation:0.011737167835235596 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2451128889806569) - rollout_corr/training_log_ppl:np.float64(0.21647670843231026) - rollout_corr/kl:np.float64(0.00146516180673828) - rollout_corr/k3_kl:np.float64(0.0013313775715459997) - rollout_corr/rollout_ppl:np.float64(1.2432878208346665) - rollout_corr/rollout_log_ppl:np.float64(0.21501154491852503) - rollout_corr/log_ppl_diff:np.float64(0.0014651635137852281) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028982661897316575) - rollout_corr/log_ppl_diff_max:np.float64(0.014495670795440674) - rollout_corr/log_ppl_diff_min:np.float64(-0.009721904993057251) - rollout_corr/ppl_ratio:np.float64(1.0014725218061358) - rollout_corr/chi2_token:np.float64(0.0023953597992658615) - rollout_corr/chi2_seq:np.float64(1.8521191228646785) - actor/pg_loss:np.float64(-0.0003558297175914049) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015348545880442543) - actor/ppo_kl:np.float64(3.0528818129482715e-05) - actor/pg_clipfrac_lower:np.float64(1.3610627320304047e-05) - actor/grad_norm:np.float64(0.7854970991611481) - perf/mfu/actor:np.float64(0.052699151040128595) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(105.79932022094727) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.32625) - val-aux/sciknoweval/reward/std@16:np.float64(0.26460828298638245) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.44315999999999994) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.23037433915304933) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.21118000000000003) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21955087215134436) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.3275) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.266025449388232) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.53736) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16763225376953209) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.12658) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14989660984549052) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.3408) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22535669809810538) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.60002) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10074795072317197) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.07074) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08389263468998247) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.34962000000000004) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.17799526365500928) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6341) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.052678976599392024) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.03935999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.044642921667641916) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.35164) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13534911056216117) - val-aux/sciknoweval/score/mean@16:np.float64(0.32625) - val-aux/sciknoweval/score/std@16:np.float64(0.26460828298638245) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.44315999999999994) - val-aux/sciknoweval/score/best@2/std:np.float64(0.23037433915304933) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.21118000000000003) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21955087215134436) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.3275) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.266025449388232) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.53736) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16763225376953209) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.12658) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14989660984549052) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.3408) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22535669809810538) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.60002) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10074795072317197) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.07074) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08389263468998247) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.34962000000000004) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.17799526365500928) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6341) - val-aux/sciknoweval/score/best@16/std:np.float64(0.052678976599392024) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.03935999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.044642921667641916) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.35164) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13534911056216117) - val-core/sciknoweval/acc/mean@16:np.float64(0.32625) - val-aux/sciknoweval/acc/std@16:np.float64(0.26460828298638245) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.44315999999999994) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.23037433915304933) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.21118000000000003) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21955087215134436) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.3275) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.266025449388232) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.53736) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16763225376953209) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.12658) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14989660984549052) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.3408) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22535669809810538) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.60002) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10074795072317197) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.07074) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08389263468998247) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.34962000000000004) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.17799526365500928) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6341) - val-core/sciknoweval/acc/best@16/std:np.float64(0.052678976599392024) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.03935999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.044642921667641916) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.35164) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13534911056216117) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.97875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.051687330456951015) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9963400000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.020813886045474284) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.96174) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06405299669111861) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9787599999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.05137839781051554) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9997800000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003963742623570884) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.93396) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07265674213411831) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99216) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03188606302646686) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.90036) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06915111790505142) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99804) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.015647532793846276) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.86874) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05102367714768366) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002157810766489876) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.25 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036463395226746798 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036463395226746798 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:297.4140625 - response_length/max:1569.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:297.4140625 - response_length_non_aborted/max:1569.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.25 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015241466462612152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.17083459533751) - timing_s/agent_loop/generate_sequences/max:np.float64(9.452028851956129) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0214375181894866) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.452028851956129) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1569 - timing_s/gen:15.476865164935589 - timing_s/reward:0.045249899849295616 - timing_s/old_log_prob:2.511210959404707 - timing_s/adv:0.15661288052797318 - timing_s/update_actor:10.127284951508045 - timing_s/step:28.350071949884295 - timing_s/testing:84.79899363964796 - timing_s/save_checkpoint:6.5641801711171865 - timing_s/stop_profile:0.00011404044926166534 - timing_per_token_ms/gen:0.20327386016096546 - timing_per_token_ms/update_actor:0.06920096860527822 - timing_per_token_ms/adv:0.0010701548421410436 - perf/total_num_tokens:146346 - perf/time_per_step:28.350071949884295 - perf/throughput:645.2629126422609 (TaskRunner pid=2099163) Training Progress: 10%|█ | 10/100 [10:46<1:44:31, 69.68s/it] (TaskRunner pid=2099163) step:11 - global_seqlen/min:16676 - global_seqlen/max:36461 - global_seqlen/minmax_diff:19785 - global_seqlen/balanced_min:22290 - global_seqlen/balanced_max:27044 - global_seqlen/mean:22898.125 - actor/entropy:0.16072751581668854 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.32411471009254456 - training/rollout_probs_diff_mean:0.0046199411153793335 - training/rollout_probs_diff_std:0.014347893185913563 - training/rollout_actor_probs_pearson_corr:0.9974071383476257 - rollout_corr/rollout_is_mean:0.9998364448547363 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00011153358354931697 - rollout_corr/rollout_is_max:1.8010543584823608 - rollout_corr/rollout_is_min:0.18950599431991577 - rollout_corr/rollout_is_std:0.040931250900030136 - rollout_corr/rollout_is_eff_sample_size:0.9983268407518708 - rollout_corr/rollout_is_seq_mean:0.9997417330741882 - rollout_corr/rollout_is_seq_std:0.0031297144014388323 - rollout_corr/rollout_is_seq_max:1.0107784271240234 - rollout_corr/rollout_is_seq_min:0.9905034303665161 - rollout_corr/rollout_is_seq_max_deviation:0.010778427124023438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2281816471368074) - rollout_corr/training_log_ppl:np.float64(0.2024953516520327) - rollout_corr/kl:np.float64(0.001424252133454651) - rollout_corr/k3_kl:np.float64(0.0011623206726625313) - rollout_corr/rollout_ppl:np.float64(1.226403083652258) - rollout_corr/rollout_log_ppl:np.float64(0.20107109841774218) - rollout_corr/log_ppl_diff:np.float64(0.0014242532342905179) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025687814049888402) - rollout_corr/log_ppl_diff_max:np.float64(0.01213720440864563) - rollout_corr/log_ppl_diff_min:np.float64(-0.009074568748474121) - rollout_corr/ppl_ratio:np.float64(1.0014300209004432) - rollout_corr/chi2_token:np.float64(0.0017990793567150831) - rollout_corr/chi2_seq:np.float64(0.5897399557288736) - actor/pg_loss:np.float64(0.00016360939480364323) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016948245472576673) - actor/ppo_kl:np.float64(9.670323938681946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6953910440206528) - perf/mfu/actor:np.float64(0.06338107402915627) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(111.92782211303711) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03447655215859413 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03447655215859413 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:385.25390625 - response_length/max:6669.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:385.25390625 - response_length_non_aborted/max:6669.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.3125 - prompt_length/max:500.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.39831817150116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2415860313922167) - timing_s/agent_loop/generate_sequences/max:np.float64(33.86591459810734) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7203849007346435) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(33.86591459810734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:6669 - timing_s/gen:39.782698061317205 - timing_s/reward:0.061702121049165726 - timing_s/old_log_prob:2.763470709323883 - timing_s/adv:0.13837043195962906 - timing_s/update_actor:10.400962248444557 - timing_s/step:53.240667348727584 - timing_s/stop_profile:0.00013394653797149658 - timing_per_token_ms/gen:0.403373364373305 - timing_per_token_ms/update_actor:0.05677846029120592 - timing_per_token_ms/adv:0.0007553589647603737 - perf/total_num_tokens:183185 - perf/time_per_step:53.240667348727584 - perf/throughput:430.0871146114071 (TaskRunner pid=2099163) Training Progress: 11%|█ | 11/100 [11:39<1:35:55, 64.66s/it] (TaskRunner pid=2099163) step:12 - global_seqlen/min:20156 - global_seqlen/max:34994 - global_seqlen/minmax_diff:14838 - global_seqlen/balanced_min:26866 - global_seqlen/balanced_max:32121 - global_seqlen/mean:28280.875 - actor/entropy:0.12687404453754425 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5569577217102051 - training/rollout_probs_diff_mean:0.00370349595323205 - training/rollout_probs_diff_std:0.013641891069710255 - training/rollout_actor_probs_pearson_corr:0.9971849918365479 - rollout_corr/rollout_is_mean:0.999925971031189 - rollout_corr/rollout_is_ratio_fraction_high:2.9888440622016788e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020174698147457093 - rollout_corr/rollout_is_max:2.8711585998535156 - rollout_corr/rollout_is_min:0.05395948514342308 - rollout_corr/rollout_is_std:0.038417838513851166 - rollout_corr/rollout_is_eff_sample_size:0.9985261260875207 - rollout_corr/rollout_is_seq_mean:0.9998483657836914 - rollout_corr/rollout_is_seq_std:0.002559172920882702 - rollout_corr/rollout_is_seq_max:1.0071240663528442 - rollout_corr/rollout_is_seq_min:0.9910182356834412 - rollout_corr/rollout_is_seq_max_deviation:0.008981764316558838 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2076720832847059) - rollout_corr/training_log_ppl:np.float64(0.1855281984044268) - rollout_corr/kl:np.float64(0.0015311053801809749) - rollout_corr/k3_kl:np.float64(0.0011974603096120973) - rollout_corr/rollout_ppl:np.float64(1.205766570288688) - rollout_corr/rollout_log_ppl:np.float64(0.1839970920154883) - rollout_corr/log_ppl_diff:np.float64(0.0015311063889384968) - rollout_corr/log_ppl_abs_diff:np.float64(0.002373620836806367) - rollout_corr/log_ppl_diff_max:np.float64(0.015954285860061646) - rollout_corr/log_ppl_diff_min:np.float64(-0.005851656198501587) - rollout_corr/ppl_ratio:np.float64(1.0015365811996162) - rollout_corr/chi2_token:np.float64(0.0016909947153180838) - rollout_corr/chi2_seq:np.float64(0.3893565512262285) - actor/pg_loss:np.float64(0.0003301744582131505) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001968728217093485) - actor/ppo_kl:np.float64(0.0002511454556559656) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8092714995145798) - perf/mfu/actor:np.float64(0.07718086515597372) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(110.70772171020508) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.45703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.45703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.019058551639318466 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.019058551639318466 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:522.77734375 - response_length/max:8192.0 - response_length/min:106.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:522.77734375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:361.0 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.196065366268158e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2030387092381716) - timing_s/agent_loop/generate_sequences/max:np.float64(49.85608519241214) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7326578640859225) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.85608519241214) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.627652468159795 - timing_s/reward:0.06745622679591179 - timing_s/old_log_prob:2.655902164056897 - timing_s/adv:0.12508256547152996 - timing_s/update_actor:10.585711555555463 - timing_s/step:69.14367071911693 - timing_s/stop_profile:0.00011152029037475586 - timing_per_token_ms/gen:0.41565595764927254 - timing_per_token_ms/update_actor:0.04678829578096268 - timing_per_token_ms/adv:0.0005528584488259732 - perf/total_num_tokens:226247 - perf/time_per_step:69.14367071911693 - perf/throughput:409.016106693058 (TaskRunner pid=2099163) Training Progress: 12%|█▏ | 12/100 [12:48<1:36:51, 66.04s/it] (TaskRunner pid=2099163) step:13 - global_seqlen/min:14490 - global_seqlen/max:27440 - global_seqlen/minmax_diff:12950 - global_seqlen/balanced_min:21991 - global_seqlen/balanced_max:22034 - global_seqlen/mean:22017.0 - actor/entropy:0.1795348972082138 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3799695372581482 - training/rollout_probs_diff_mean:0.004953271709382534 - training/rollout_probs_diff_std:0.014711440540850163 - training/rollout_actor_probs_pearson_corr:0.9974781274795532 - rollout_corr/rollout_is_mean:1.0001226663589478 - rollout_corr/rollout_is_ratio_fraction_high:4.09433341701515e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.165083661675453e-05 - rollout_corr/rollout_is_max:2.9943156242370605 - rollout_corr/rollout_is_min:0.36588054895401 - rollout_corr/rollout_is_std:0.04241306707262993 - rollout_corr/rollout_is_eff_sample_size:0.9982047182816606 - rollout_corr/rollout_is_seq_mean:1.0001139640808105 - rollout_corr/rollout_is_seq_std:0.003129767719656229 - rollout_corr/rollout_is_seq_max:1.0108983516693115 - rollout_corr/rollout_is_seq_min:0.9917095303535461 - rollout_corr/rollout_is_seq_max_deviation:0.010898351669311523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2477836473844945) - rollout_corr/training_log_ppl:np.float64(0.2179029462276958) - rollout_corr/kl:np.float64(0.0012722289181801294) - rollout_corr/k3_kl:np.float64(0.0011847673267197933) - rollout_corr/rollout_ppl:np.float64(1.2461294089443982) - rollout_corr/rollout_log_ppl:np.float64(0.2166307152074296) - rollout_corr/log_ppl_diff:np.float64(0.0012722310202661902) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027390209725126624) - rollout_corr/log_ppl_diff_max:np.float64(0.01292330026626587) - rollout_corr/log_ppl_diff_min:np.float64(-0.009444236755371094) - rollout_corr/ppl_ratio:np.float64(1.0012791764456779) - rollout_corr/chi2_token:np.float64(0.002235463121905923) - rollout_corr/chi2_seq:np.float64(1.5766265538986772) - actor/pg_loss:np.float64(-0.0002433034824207425) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002133346634764166) - actor/ppo_kl:np.float64(0.00032201276727761297) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8120427280664444) - perf/mfu/actor:np.float64(0.06433296210788333) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(110.85671615600586) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007056328002363443 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007056328002363443 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:381.625 - response_length/max:1800.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.625 - response_length_non_aborted/max:1800.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013713166117668152 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2604616954922676) - timing_s/agent_loop/generate_sequences/max:np.float64(11.544187435880303) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8230461675921106) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.544187435880303) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:1800 - timing_s/gen:17.578441470861435 - timing_s/reward:0.061520595103502274 - timing_s/old_log_prob:2.475315697491169 - timing_s/adv:0.12054650858044624 - timing_s/update_actor:9.704637175425887 - timing_s/step:30.0266327932477 - timing_s/stop_profile:0.0001079123467206955 - timing_per_token_ms/gen:0.17993000195362588 - timing_per_token_ms/update_actor:0.05509740868093909 - timing_per_token_ms/adv:0.0006843944939163274 - perf/total_num_tokens:176136 - perf/time_per_step:30.0266327932477 - perf/throughput:733.2490509875325 (TaskRunner pid=2099163) Training Progress: 13%|█▎ | 13/100 [13:18<1:19:57, 55.14s/it] (TaskRunner pid=2099163) step:14 - global_seqlen/min:12976 - global_seqlen/max:34060 - global_seqlen/minmax_diff:21084 - global_seqlen/balanced_min:22596 - global_seqlen/balanced_max:22654 - global_seqlen/mean:22633.375 - actor/entropy:0.16276216506958008 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31926417350769043 - training/rollout_probs_diff_mean:0.004311886616051197 - training/rollout_probs_diff_std:0.013316148892045021 - training/rollout_actor_probs_pearson_corr:0.9977449178695679 - rollout_corr/rollout_is_mean:0.9999713897705078 - rollout_corr/rollout_is_ratio_fraction_high:3.8242746086325496e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.692480383208022e-05 - rollout_corr/rollout_is_max:2.2574832439422607 - rollout_corr/rollout_is_min:0.28150883316993713 - rollout_corr/rollout_is_std:0.040303174406290054 - rollout_corr/rollout_is_eff_sample_size:0.9983781696179648 - rollout_corr/rollout_is_seq_mean:0.9999383687973022 - rollout_corr/rollout_is_seq_std:0.0024390218313783407 - rollout_corr/rollout_is_seq_max:1.008014440536499 - rollout_corr/rollout_is_seq_min:0.9899135828018188 - rollout_corr/rollout_is_seq_max_deviation:0.010086417198181152 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2327516474761069) - rollout_corr/training_log_ppl:np.float64(0.20615991155500524) - rollout_corr/kl:np.float64(0.0009014457496316908) - rollout_corr/k3_kl:np.float64(0.0010392485548891273) - rollout_corr/rollout_ppl:np.float64(1.231638588476926) - rollout_corr/rollout_log_ppl:np.float64(0.2052584641496651) - rollout_corr/log_ppl_diff:np.float64(0.00090144740534015) - rollout_corr/log_ppl_abs_diff:np.float64(0.00216034529148601) - rollout_corr/log_ppl_diff_max:np.float64(0.010936662554740906) - rollout_corr/log_ppl_diff_min:np.float64(-0.008887723088264465) - rollout_corr/ppl_ratio:np.float64(1.0009056760463864) - rollout_corr/chi2_token:np.float64(0.002428637119010091) - rollout_corr/chi2_seq:np.float64(1.1035329343285412) - actor/pg_loss:np.float64(0.0004473176086321473) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011060538008678122) - actor/ppo_kl:np.float64(-0.0001549605085848782) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.575629249215126) - perf/mfu/actor:np.float64(0.0652755181152363) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(109.99267196655273) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.26953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.26953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010551412589848042 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010551412589848042 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:408.57421875 - response_length/max:2188.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:408.57421875 - response_length_non_aborted/max:2188.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.820424020290375e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0610912814736366) - timing_s/agent_loop/generate_sequences/max:np.float64(13.642793396487832) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8376170305346022) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.642793396487832) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:2188 - timing_s/gen:19.731356034055352 - timing_s/reward:0.06039970554411411 - timing_s/old_log_prob:2.502957135438919 - timing_s/adv:0.12832614220678806 - timing_s/update_actor:9.91347792558372 - timing_s/step:32.42584792152047 - timing_s/stop_profile:0.00012178905308246613 - timing_per_token_ms/gen:0.18864530841871363 - timing_per_token_ms/update_actor:0.05475032957735932 - timing_per_token_ms/adv:0.0007087218665288985 - perf/total_num_tokens:181067 - perf/time_per_step:32.42584792152047 - perf/throughput:698.0041063160178 (TaskRunner pid=2099163) Training Progress: 14%|█▍ | 14/100 [13:51<1:09:13, 48.30s/it] (TaskRunner pid=2099163) step:15 - global_seqlen/min:15918 - global_seqlen/max:28170 - global_seqlen/minmax_diff:12252 - global_seqlen/balanced_min:19653 - global_seqlen/balanced_max:19679 - global_seqlen/mean:19673.25 - actor/entropy:0.19017662107944489 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6307171583175659 - training/rollout_probs_diff_mean:0.005055586341768503 - training/rollout_probs_diff_std:0.014726489782333374 - training/rollout_actor_probs_pearson_corr:0.9975889921188354 - rollout_corr/rollout_is_mean:1.0000306367874146 - rollout_corr/rollout_is_ratio_fraction_high:2.42359619733179e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013329778448678553 - rollout_corr/rollout_is_max:2.5952959060668945 - rollout_corr/rollout_is_min:0.22091145813465118 - rollout_corr/rollout_is_std:0.04240182042121887 - rollout_corr/rollout_is_eff_sample_size:0.9982053121902402 - rollout_corr/rollout_is_seq_mean:1.0000522136688232 - rollout_corr/rollout_is_seq_std:0.0030719421338289976 - rollout_corr/rollout_is_seq_max:1.0085277557373047 - rollout_corr/rollout_is_seq_min:0.9906306266784668 - rollout_corr/rollout_is_seq_max_deviation:0.009369373321533203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2559455754235387) - rollout_corr/training_log_ppl:np.float64(0.22441870825423393) - rollout_corr/kl:np.float64(0.001127970071593154) - rollout_corr/k3_kl:np.float64(0.001125821751145395) - rollout_corr/rollout_ppl:np.float64(1.254524188581854) - rollout_corr/rollout_log_ppl:np.float64(0.22329073723813053) - rollout_corr/log_ppl_diff:np.float64(0.0011279710161034018) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024534623080398887) - rollout_corr/log_ppl_diff_max:np.float64(0.014268159866333008) - rollout_corr/log_ppl_diff_min:np.float64(-0.008966684341430664) - rollout_corr/ppl_ratio:np.float64(1.00113360513933) - rollout_corr/chi2_token:np.float64(0.0023321015760302544) - rollout_corr/chi2_seq:np.float64(0.6067947293631732) - actor/pg_loss:np.float64(-0.00015570060350000858) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014229919640911248) - actor/ppo_kl:np.float64(0.00016318686129324078) - actor/pg_clipfrac_lower:np.float64(5.865240382263437e-06) - actor/grad_norm:np.float64(0.6899215131998062) - perf/mfu/actor:np.float64(0.05693643757760258) - perf/max_memory_allocated_gb:np.float64(123.58113050460815) - perf/max_memory_reserved_gb:np.float64(129.3828125) - perf/cpu_memory_used_gb:np.float64(110.72046184539795) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:1 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012832941487431526 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012832941487431526 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:322.3515625 - response_length/max:1553.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.3515625 - response_length_non_aborted/max:1553.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003503747284412384 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2059585116803646) - timing_s/agent_loop/generate_sequences/max:np.float64(9.883644664660096) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.228372771802242) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.883644664660096) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1553 - timing_s/gen:15.838685717433691 - timing_s/reward:0.051979394629597664 - timing_s/old_log_prob:2.7615113761276007 - timing_s/adv:0.1317354552447796 - timing_s/update_actor:9.826176561415195 - timing_s/step:28.72526820190251 - timing_s/stop_profile:0.0001147240400314331 - timing_per_token_ms/gen:0.1919328871989735 - timing_per_token_ms/update_actor:0.06243361265560594 - timing_per_token_ms/adv:0.0008370214329405384 - perf/total_num_tokens:157386 - perf/time_per_step:28.72526820190251 - perf/throughput:684.876111920759 (TaskRunner pid=2099163) Training Progress: 15%|█▌ | 15/100 [14:20<1:00:12, 42.50s/it] (TaskRunner pid=2099163) step:16 - global_seqlen/min:17621 - global_seqlen/max:49777 - global_seqlen/minmax_diff:32156 - global_seqlen/balanced_min:23638 - global_seqlen/balanced_max:29431 - global_seqlen/mean:27074.0 - actor/entropy:0.12440122663974762 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7334084510803223 - training/rollout_probs_diff_mean:0.003286235500127077 - training/rollout_probs_diff_std:0.012201991863548756 - training/rollout_actor_probs_pearson_corr:0.997710645198822 - rollout_corr/rollout_is_mean:0.999930739402771 - rollout_corr/rollout_is_ratio_fraction_high:7.548765097453725e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.793888314859942e-05 - rollout_corr/rollout_is_max:2.695401906967163 - rollout_corr/rollout_is_min:0.02908293716609478 - rollout_corr/rollout_is_std:0.034535329788923264 - rollout_corr/rollout_is_eff_sample_size:0.9988087318703696 - rollout_corr/rollout_is_seq_mean:0.9999657869338989 - rollout_corr/rollout_is_seq_std:0.002763616619631648 - rollout_corr/rollout_is_seq_max:1.0094408988952637 - rollout_corr/rollout_is_seq_min:0.9909818768501282 - rollout_corr/rollout_is_seq_max_deviation:0.009440898895263672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2334859780967236) - rollout_corr/training_log_ppl:np.float64(0.20614377667880035) - rollout_corr/kl:np.float64(0.0010040062433311903) - rollout_corr/k3_kl:np.float64(0.0012267602566424785) - rollout_corr/rollout_ppl:np.float64(1.2322044824250042) - rollout_corr/rollout_log_ppl:np.float64(0.20513977024347696) - rollout_corr/log_ppl_diff:np.float64(0.0010040064353233902) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025442858877795516) - rollout_corr/log_ppl_diff_max:np.float64(0.014998257160186768) - rollout_corr/log_ppl_diff_min:np.float64(-0.008154168725013733) - rollout_corr/ppl_ratio:np.float64(1.0010099613573402) - rollout_corr/chi2_token:np.float64(0.002799068344756961) - rollout_corr/chi2_seq:np.float64(0.6977890518028289) - actor/pg_loss:np.float64(0.0005094839725643396) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0022031578653241013) - actor/ppo_kl:np.float64(-6.424526176118661e-05) - actor/pg_clipfrac_lower:np.float64(1.4852661479380913e-05) - actor/grad_norm:np.float64(0.759627491235733) - perf/mfu/actor:np.float64(0.07361790070915576) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.84224700927734) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0865541398525238 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0865541398525238 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:517.46875 - response_length/max:8192.0 - response_length/min:85.0 - response_length/clip_ratio:0.015625 - response_length_non_aborted/mean:517.46875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.015625 - response/aborted_ratio:0.0 - prompt_length/mean:328.59375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.716115891933441e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6790603306144476) - timing_s/agent_loop/generate_sequences/max:np.float64(51.488350415602326) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.263302836421644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.488350415602326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.936794666573405 - timing_s/reward:0.06626211851835251 - timing_s/old_log_prob:2.6734151635318995 - timing_s/adv:0.12808487936854362 - timing_s/update_actor:10.97300206311047 - timing_s/step:71.86347073316574 - timing_s/stop_profile:0.00010935962200164795 - timing_per_token_ms/gen:0.43735124906828166 - timing_per_token_ms/update_actor:0.05066208384017171 - timing_per_token_ms/adv:0.0005913647751003898 - perf/total_num_tokens:216592 - perf/time_per_step:71.86347073316574 - perf/throughput:376.7421712837628 (TaskRunner pid=2099163) Training Progress: 16%|█▌ | 16/100 [15:32<1:11:53, 51.36s/it] (TaskRunner pid=2099163) step:17 - global_seqlen/min:11966 - global_seqlen/max:39775 - global_seqlen/minmax_diff:27809 - global_seqlen/balanced_min:20593 - global_seqlen/balanced_max:26443 - global_seqlen/mean:22197.625 - actor/entropy:0.15671347081661224 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34504491090774536 - training/rollout_probs_diff_mean:0.003963190596550703 - training/rollout_probs_diff_std:0.012594124302268028 - training/rollout_actor_probs_pearson_corr:0.9979807138442993 - rollout_corr/rollout_is_mean:0.9999871253967285 - rollout_corr/rollout_is_ratio_fraction_high:9.640877578931395e-06 - rollout_corr/rollout_is_ratio_fraction_low:9.640877578931395e-06 - rollout_corr/rollout_is_max:2.198077917098999 - rollout_corr/rollout_is_min:0.4044226408004761 - rollout_corr/rollout_is_std:0.03752783685922623 - rollout_corr/rollout_is_eff_sample_size:0.9985936420813145 - rollout_corr/rollout_is_seq_mean:0.9999175071716309 - rollout_corr/rollout_is_seq_std:0.0031496852170675993 - rollout_corr/rollout_is_seq_max:1.0094223022460938 - rollout_corr/rollout_is_seq_min:0.9900813102722168 - rollout_corr/rollout_is_seq_max_deviation:0.009918689727783203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2560751209966838) - rollout_corr/training_log_ppl:np.float64(0.22452797560981708) - rollout_corr/kl:np.float64(0.0013395728782059635) - rollout_corr/k3_kl:np.float64(0.0012000854812015405) - rollout_corr/rollout_ppl:np.float64(1.2543696821667254) - rollout_corr/rollout_log_ppl:np.float64(0.2231884019674908) - rollout_corr/log_ppl_diff:np.float64(0.001339573642326286) - rollout_corr/log_ppl_abs_diff:np.float64(0.002711944489419693) - rollout_corr/log_ppl_diff_max:np.float64(0.01098942756652832) - rollout_corr/log_ppl_diff_min:np.float64(-0.0078660249710083) - rollout_corr/ppl_ratio:np.float64(1.0013459962792695) - rollout_corr/chi2_token:np.float64(0.0021593261044472456) - rollout_corr/chi2_seq:np.float64(0.8300942496862262) - actor/pg_loss:np.float64(0.0006044314941391349) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015271970464709739) - actor/ppo_kl:np.float64(0.00025955110856701324) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.73015196621418) - perf/mfu/actor:np.float64(0.06131922719985038) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.87406206130981) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:1 - critic/score/mean:0.28515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06812485307455063 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06812485307455063 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:405.17578125 - response_length/max:8192.0 - response_length/min:85.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:405.17578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:288.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014323554933071136 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9844553098082542) - timing_s/agent_loop/generate_sequences/max:np.float64(49.343952156603336) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.660478450343362) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.343952156603336) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.40832286886871 - timing_s/reward:0.06138620153069496 - timing_s/old_log_prob:2.554391412064433 - timing_s/adv:0.17900926247239113 - timing_s/update_actor:10.665983552113175 - timing_s/step:68.9551175814122 - timing_s/stop_profile:0.0001098737120628357 - timing_per_token_ms/gen:0.5341848432766325 - timing_per_token_ms/update_actor:0.06006263931452788 - timing_per_token_ms/adv:0.001008042878868748 - perf/total_num_tokens:177581 - perf/time_per_step:68.9551175814122 - perf/throughput:321.9141055599284 (TaskRunner pid=2099163) Training Progress: 17%|█▋ | 17/100 [16:41<1:18:23, 56.67s/it] (TaskRunner pid=2099163) step:18 - global_seqlen/min:11273 - global_seqlen/max:36015 - global_seqlen/minmax_diff:24742 - global_seqlen/balanced_min:21872 - global_seqlen/balanced_max:27887 - global_seqlen/mean:22719.125 - actor/entropy:0.1572161316871643 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7516714334487915 - training/rollout_probs_diff_mean:0.003971869591623545 - training/rollout_probs_diff_std:0.01294691488146782 - training/rollout_actor_probs_pearson_corr:0.9978299140930176 - rollout_corr/rollout_is_mean:1.0000897645950317 - rollout_corr/rollout_is_ratio_fraction_high:3.9406924770446494e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9703462385223247e-05 - rollout_corr/rollout_is_max:2.524366617202759 - rollout_corr/rollout_is_min:0.09788884967565536 - rollout_corr/rollout_is_std:0.037467435002326965 - rollout_corr/rollout_is_eff_sample_size:0.998598397072895 - rollout_corr/rollout_is_seq_mean:1.0000863075256348 - rollout_corr/rollout_is_seq_std:0.0033069117926061153 - rollout_corr/rollout_is_seq_max:1.0111510753631592 - rollout_corr/rollout_is_seq_min:0.9837021827697754 - rollout_corr/rollout_is_seq_max_deviation:0.01629781723022461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2480866289697587) - rollout_corr/training_log_ppl:np.float64(0.2177488712950435) - rollout_corr/kl:np.float64(0.0012160134455356797) - rollout_corr/k3_kl:np.float64(0.0012282074120264497) - rollout_corr/rollout_ppl:np.float64(1.2464866428636014) - rollout_corr/rollout_log_ppl:np.float64(0.21653285443244386) - rollout_corr/log_ppl_diff:np.float64(0.0012160168625996448) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027001788184861653) - rollout_corr/log_ppl_diff_max:np.float64(0.01847335696220398) - rollout_corr/log_ppl_diff_min:np.float64(-0.008770108222961426) - rollout_corr/ppl_ratio:np.float64(1.0012231352739036) - rollout_corr/chi2_token:np.float64(0.0024735613260418177) - rollout_corr/chi2_seq:np.float64(1.276715693064034) - actor/pg_loss:np.float64(0.0005491708870977163) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019209535612390027) - actor/ppo_kl:np.float64(0.00030758063345004416) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7345165759325027) - perf/mfu/actor:np.float64(0.06209275037892638) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.8076171875) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:1 - critic/score/mean:0.3359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02325008623301983 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02325008623301983 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:396.50390625 - response_length/max:8192.0 - response_length/min:92.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:396.50390625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:313.46875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001363120973110199 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0805652625858784) - timing_s/agent_loop/generate_sequences/max:np.float64(49.01962508261204) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7108128744512214) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.01962508261204) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:54.90000583231449 - timing_s/reward:0.06206467002630234 - timing_s/old_log_prob:2.6254268158227205 - timing_s/adv:0.12246500886976719 - timing_s/update_actor:10.576374450698495 - timing_s/step:68.37408596090972 - timing_s/stop_profile:0.00011956505477428436 - timing_per_token_ms/gen:0.5408601136132652 - timing_per_token_ms/update_actor:0.05819092092399297 - timing_per_token_ms/adv:0.0006737991057631357 - perf/total_num_tokens:181753 - perf/time_per_step:68.37408596090972 - perf/throughput:332.27683676808186 (TaskRunner pid=2099163) Training Progress: 18%|█▊ | 18/100 [17:49<1:22:16, 60.20s/it] (TaskRunner pid=2099163) step:19 - global_seqlen/min:14639 - global_seqlen/max:26849 - global_seqlen/minmax_diff:12210 - global_seqlen/balanced_min:21052 - global_seqlen/balanced_max:21251 - global_seqlen/mean:21094.25 - actor/entropy:0.18874603509902954 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417160868644714 - training/rollout_probs_diff_mean:0.004822679795324802 - training/rollout_probs_diff_std:0.014397743158042431 - training/rollout_actor_probs_pearson_corr:0.9977278113365173 - rollout_corr/rollout_is_mean:0.9999311566352844 - rollout_corr/rollout_is_ratio_fraction_high:1.0811979336722288e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014055574138183147 - rollout_corr/rollout_is_max:2.0913357734680176 - rollout_corr/rollout_is_min:0.040095582604408264 - rollout_corr/rollout_is_std:0.0410170778632164 - rollout_corr/rollout_is_eff_sample_size:0.998320068587282 - rollout_corr/rollout_is_seq_mean:1.0000311136245728 - rollout_corr/rollout_is_seq_std:0.0029985024593770504 - rollout_corr/rollout_is_seq_max:1.010836124420166 - rollout_corr/rollout_is_seq_min:0.987970232963562 - rollout_corr/rollout_is_seq_max_deviation:0.012029767036437988 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.255211434327066) - rollout_corr/training_log_ppl:np.float64(0.22383148300286848) - rollout_corr/kl:np.float64(0.0012053162868781442) - rollout_corr/k3_kl:np.float64(0.0011591173564511337) - rollout_corr/rollout_ppl:np.float64(1.253677450120449) - rollout_corr/rollout_log_ppl:np.float64(0.2226261663454352) - rollout_corr/log_ppl_diff:np.float64(0.0012053166574332863) - rollout_corr/log_ppl_abs_diff:np.float64(0.002684240898815915) - rollout_corr/log_ppl_diff_max:np.float64(0.01323661208152771) - rollout_corr/log_ppl_diff_min:np.float64(-0.010094761848449707) - rollout_corr/ppl_ratio:np.float64(1.0012121815234423) - rollout_corr/chi2_token:np.float64(0.0021450589410960674) - rollout_corr/chi2_seq:np.float64(1.123602865729481) - actor/pg_loss:np.float64(0.00028023216873407364) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009424093339021056) - actor/ppo_kl:np.float64(0.0002521510247390779) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5630562603473663) - perf/mfu/actor:np.float64(0.06105586170219322) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.85847091674805) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:1 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012870309874415398 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012870309874415398 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:361.2890625 - response_length/max:1883.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.2890625 - response_length_non_aborted/max:1883.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.90625 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.548874080181122e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0813855342566967) - timing_s/agent_loop/generate_sequences/max:np.float64(11.688677107915282) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5753794663687586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.688677107915282) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:1883 - timing_s/gen:17.670184712857008 - timing_s/reward:0.06079593487083912 - timing_s/old_log_prob:2.4477415289729834 - timing_s/adv:0.12715278938412666 - timing_s/update_actor:9.766590259969234 - timing_s/step:30.157636089250445 - timing_s/stop_profile:0.00011079944670200348 - timing_per_token_ms/gen:0.19104967794201544 - timing_per_token_ms/update_actor:0.05787471858426606 - timing_per_token_ms/adv:0.0007534801508949516 - perf/total_num_tokens:168754 - perf/time_per_step:30.157636089250445 - perf/throughput:699.4662956198663 (TaskRunner pid=2099163) Training Progress: 19%|█▉ | 19/100 [18:20<1:09:06, 51.19s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) To determine the folding stability score of the given protein sequence, one would typically use a computational method or software that calculates the stability based on the amino acid sequence. This score is often derived from energy calculations that consider factors such as hydrogen bonding, hydrophobic interactions, and van der Waals forces. However, without access to specific computational tools or databases that provide such scores for the given sequence, it is not possible to calculate the exact folding stability score. Therefore, the correct answer cannot be determined from the information provided. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2099163) step:20 - global_seqlen/min:12298 - global_seqlen/max:30479 - global_seqlen/minmax_diff:18181 - global_seqlen/balanced_min:21810 - global_seqlen/balanced_max:21871 - global_seqlen/mean:21839.375 - actor/entropy:0.18792696297168732 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6172393560409546 - training/rollout_probs_diff_mean:0.004675295669585466 - training/rollout_probs_diff_std:0.013787684962153435 - training/rollout_actor_probs_pearson_corr:0.9978795051574707 - rollout_corr/rollout_is_mean:1.0000436305999756 - rollout_corr/rollout_is_ratio_fraction_high:5.289270120556466e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010578540241112933 - rollout_corr/rollout_is_max:2.6348955631256104 - rollout_corr/rollout_is_min:0.11010421812534332 - rollout_corr/rollout_is_std:0.040804363787174225 - rollout_corr/rollout_is_eff_sample_size:0.998337890270291 - rollout_corr/rollout_is_seq_mean:1.000036597251892 - rollout_corr/rollout_is_seq_std:0.0025871675461530685 - rollout_corr/rollout_is_seq_max:1.009564757347107 - rollout_corr/rollout_is_seq_min:0.9896543622016907 - rollout_corr/rollout_is_seq_max_deviation:0.010345637798309326 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2705717207863927) - rollout_corr/training_log_ppl:np.float64(0.2352731467835838) - rollout_corr/kl:np.float64(0.00124775865324378) - rollout_corr/k3_kl:np.float64(0.001086297427093541) - rollout_corr/rollout_ppl:np.float64(1.2689147526398301) - rollout_corr/rollout_log_ppl:np.float64(0.23402538590016775) - rollout_corr/log_ppl_diff:np.float64(0.0012477608834160492) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022727364994352683) - rollout_corr/log_ppl_diff_max:np.float64(0.012642994523048401) - rollout_corr/log_ppl_diff_min:np.float64(-0.007874488830566406) - rollout_corr/ppl_ratio:np.float64(1.0012527105864137) - rollout_corr/chi2_token:np.float64(0.0018538862932473421) - rollout_corr/chi2_seq:np.float64(0.5317726316861808) - actor/pg_loss:np.float64(-4.45549376308918e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001396393277673269) - actor/ppo_kl:np.float64(0.0002731104437279086) - actor/pg_clipfrac_lower:np.float64(7.614522473886609e-06) - actor/grad_norm:np.float64(0.5186081603169441) - perf/mfu/actor:np.float64(0.06376455117784033) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(110.90739440917969) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3575) - val-aux/sciknoweval/reward/std@16:np.float64(0.2845272325003602) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.4821000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.24641293690229316) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.23617999999999997) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.23810031156810768) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.35924000000000006) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2844295099662947) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.57886) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.17803900782144286) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.13665999999999998) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16515577551517271) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.38042000000000004) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23790217711971715) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.64558) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.11228107051862579) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.07342) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08529636863102308) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.39696) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.18341197265688536) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6885199999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06561633303370275) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.04667999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03078753156755329) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.4047799999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13095914178704102) - val-aux/sciknoweval/score/mean@16:np.float64(0.3575) - val-aux/sciknoweval/score/std@16:np.float64(0.2845272325003602) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.4821000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.24641293690229316) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.23617999999999997) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.23810031156810768) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.35924000000000006) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2844295099662947) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.57886) - val-aux/sciknoweval/score/best@4/std:np.float64(0.17803900782144286) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.13665999999999998) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16515577551517271) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.38042000000000004) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23790217711971715) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.64558) - val-aux/sciknoweval/score/best@8/std:np.float64(0.11228107051862579) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.07342) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08529636863102308) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.39696) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.18341197265688536) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6885199999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06561633303370275) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.04667999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03078753156755329) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.4047799999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13095914178704102) - val-core/sciknoweval/acc/mean@16:np.float64(0.3575) - val-aux/sciknoweval/acc/std@16:np.float64(0.2845272325003602) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.4821000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.24641293690229316) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.23617999999999997) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.23810031156810768) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.35924000000000006) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2844295099662947) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.57886) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.17803900782144286) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.13665999999999998) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16515577551517271) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.38042000000000004) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23790217711971715) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.64558) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.11228107051862579) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.07342) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08529636863102308) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.39696) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.18341197265688536) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6885199999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06561633303370275) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.04667999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03078753156755329) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.4047799999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13095914178704102) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04906017482220243) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99724) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.017382402655940103) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9658800000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06179985484869913) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.98052) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04963623931745396) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.99988) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.001544538766104626) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9399599999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.07259562722286844) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99358) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.02825209907076314) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.90684) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07207216802183986) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99862) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010856231271909862) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8730199999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05670226605201907) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.001787064632295094) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:1 - critic/score/mean:0.3359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017243020702153444 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017243020702153444 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:369.26171875 - response_length/max:1936.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.26171875 - response_length_non_aborted/max:1936.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011486373841762543 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1602171901613474) - timing_s/agent_loop/generate_sequences/max:np.float64(12.04150439798832) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.66143894479319) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.04150439798832) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1936 - timing_s/gen:17.918031726032495 - timing_s/reward:0.06160684861242771 - timing_s/old_log_prob:2.4307430386543274 - timing_s/adv:0.11749370023608208 - timing_s/update_actor:9.837341289967299 - timing_s/step:30.453087836503983 - timing_s/testing:84.54373020865023 - timing_s/save_checkpoint:6.91516363248229 - timing_s/stop_profile:0.00016070157289505005 - timing_per_token_ms/gen:0.18954662201851769 - timing_per_token_ms/update_actor:0.05630507563727956 - timing_per_token_ms/adv:0.0006724877671412418 - perf/total_num_tokens:174715 - perf/time_per_step:30.453087836503983 - perf/throughput:717.148130174873 (TaskRunner pid=2099163) Training Progress: 20%|██ | 20/100 [20:22<1:36:35, 72.45s/it] (TaskRunner pid=2099163) step:21 - global_seqlen/min:13059 - global_seqlen/max:30220 - global_seqlen/minmax_diff:17161 - global_seqlen/balanced_min:20197 - global_seqlen/balanced_max:20234 - global_seqlen/mean:20222.625 - actor/entropy:0.2039741724729538 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915150225162506 - training/rollout_probs_diff_mean:0.005005266051739454 - training/rollout_probs_diff_std:0.013932553119957447 - training/rollout_actor_probs_pearson_corr:0.9979515671730042 - rollout_corr/rollout_is_mean:0.9997883439064026 - rollout_corr/rollout_is_ratio_fraction_high:1.1129289305245038e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.3997433185577393 - rollout_corr/rollout_is_min:0.5131387114524841 - rollout_corr/rollout_is_std:0.04085326939821243 - rollout_corr/rollout_is_eff_sample_size:0.9983332565714974 - rollout_corr/rollout_is_seq_mean:0.9999094605445862 - rollout_corr/rollout_is_seq_std:0.0028472074773162603 - rollout_corr/rollout_is_seq_max:1.011277198791504 - rollout_corr/rollout_is_seq_min:0.9916235208511353 - rollout_corr/rollout_is_seq_max_deviation:0.011277198791503906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2804982154630125) - rollout_corr/training_log_ppl:np.float64(0.2435035327434889) - rollout_corr/kl:np.float64(0.0010719042137878887) - rollout_corr/k3_kl:np.float64(0.0011022668704754324) - rollout_corr/rollout_ppl:np.float64(1.2791178887709975) - rollout_corr/rollout_log_ppl:np.float64(0.24243162775383098) - rollout_corr/log_ppl_diff:np.float64(0.0010719049896579236) - rollout_corr/log_ppl_abs_diff:np.float64(0.002581465640105307) - rollout_corr/log_ppl_diff_max:np.float64(0.009952932596206665) - rollout_corr/log_ppl_diff_min:np.float64(-0.010108351707458496) - rollout_corr/ppl_ratio:np.float64(1.0010773693211377) - rollout_corr/chi2_token:np.float64(0.0022938339971005917) - rollout_corr/chi2_seq:np.float64(1.2692464508581907) - actor/pg_loss:np.float64(0.0004962929524481297) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018514558880724508) - actor/ppo_kl:np.float64(6.505578917792576e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7440718114376068) - perf/mfu/actor:np.float64(0.05869452164939222) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.24159622192383) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:1 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015914883697405457 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015914883697405457 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:350.98828125 - response_length/max:1700.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:350.98828125 - response_length_non_aborted/max:1700.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.96875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.297362804412842e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.264117207378149) - timing_s/agent_loop/generate_sequences/max:np.float64(10.401357980445027) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5284495416854043) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.401357980445027) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1700 - timing_s/gen:16.154271429404616 - timing_s/reward:0.05521078407764435 - timing_s/old_log_prob:2.5382862519472837 - timing_s/adv:0.14272014237940311 - timing_s/update_actor:9.808351395651698 - timing_s/step:28.792014434933662 - timing_s/stop_profile:0.00015562400221824646 - timing_per_token_ms/gen:0.17978555451019573 - timing_per_token_ms/update_actor:0.060627338164875345 - timing_per_token_ms/adv:0.0008821811113752735 - perf/total_num_tokens:161781 - perf/time_per_step:28.792014434933662 - perf/throughput:702.3692296938998 (TaskRunner pid=2099163) Training Progress: 21%|██ | 21/100 [20:50<1:18:09, 59.36s/it] (TaskRunner pid=2099163) step:22 - global_seqlen/min:18027 - global_seqlen/max:36166 - global_seqlen/minmax_diff:18139 - global_seqlen/balanced_min:25060 - global_seqlen/balanced_max:28210 - global_seqlen/mean:25542.25 - actor/entropy:0.17578984797000885 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4386941194534302 - training/rollout_probs_diff_mean:0.004188444931060076 - training/rollout_probs_diff_std:0.012796149589121342 - training/rollout_actor_probs_pearson_corr:0.9980850219726562 - rollout_corr/rollout_is_mean:1.000032901763916 - rollout_corr/rollout_is_ratio_fraction_high:1.730672738631256e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010384036431787536 - rollout_corr/rollout_is_max:2.1636812686920166 - rollout_corr/rollout_is_min:0.21608301997184753 - rollout_corr/rollout_is_std:0.038447305560112 - rollout_corr/rollout_is_eff_sample_size:0.9985239866454121 - rollout_corr/rollout_is_seq_mean:0.9999626874923706 - rollout_corr/rollout_is_seq_std:0.0027287364937365055 - rollout_corr/rollout_is_seq_max:1.0086798667907715 - rollout_corr/rollout_is_seq_min:0.990822970867157 - rollout_corr/rollout_is_seq_max_deviation:0.009177029132843018 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.265514174476266) - rollout_corr/training_log_ppl:np.float64(0.23049156166962348) - rollout_corr/kl:np.float64(0.001140852084887456) - rollout_corr/k3_kl:np.float64(0.0010895646327071518) - rollout_corr/rollout_ppl:np.float64(1.2640215558931231) - rollout_corr/rollout_log_ppl:np.float64(0.22935070820676629) - rollout_corr/log_ppl_diff:np.float64(0.0011408534628571942) - rollout_corr/log_ppl_abs_diff:np.float64(0.002326310393982567) - rollout_corr/log_ppl_diff_max:np.float64(0.016114696860313416) - rollout_corr/log_ppl_diff_min:np.float64(-0.005415067076683044) - rollout_corr/ppl_ratio:np.float64(1.0011460261885077) - rollout_corr/chi2_token:np.float64(0.0020373596344143152) - rollout_corr/chi2_seq:np.float64(1.662300607888028) - actor/pg_loss:np.float64(-4.751852247864008e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013797034152958076) - actor/ppo_kl:np.float64(0.0001431179896584922) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6352894306182861) - perf/mfu/actor:np.float64(0.07072783775794689) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(107.67041778564453) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:1 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.024204539135098457 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.024204539135098457 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.4140625 - response_length/max:5103.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:451.4140625 - response_length_non_aborted/max:5103.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.669573485851288e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2452805154025555) - timing_s/agent_loop/generate_sequences/max:np.float64(27.396138545125723) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.402181767611182) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.396138545125723) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:5103 - timing_s/gen:33.46731911972165 - timing_s/reward:0.068209582939744 - timing_s/old_log_prob:2.507258925586939 - timing_s/adv:0.13793614879250526 - timing_s/update_actor:10.272249413654208 - timing_s/step:46.539894139394164 - timing_s/stop_profile:3.1733885407447815e-05 - timing_per_token_ms/gen:0.2896048798023715 - timing_per_token_ms/update_actor:0.05027087185767801 - timing_per_token_ms/adv:0.0006750391449094406 - perf/total_num_tokens:204338 - perf/time_per_step:46.539894139394164 - perf/throughput:548.8248409740044 (TaskRunner pid=2099163) Training Progress: 22%|██▏ | 22/100 [21:37<1:12:10, 55.52s/it] (TaskRunner pid=2099163) step:23 - global_seqlen/min:18253 - global_seqlen/max:33113 - global_seqlen/minmax_diff:14860 - global_seqlen/balanced_min:23569 - global_seqlen/balanced_max:24069 - global_seqlen/mean:23647.375 - actor/entropy:0.20427009463310242 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.403209388256073 - training/rollout_probs_diff_mean:0.004876875318586826 - training/rollout_probs_diff_std:0.013606295920908451 - training/rollout_actor_probs_pearson_corr:0.9980311393737793 - rollout_corr/rollout_is_mean:0.9999285936355591 - rollout_corr/rollout_is_ratio_fraction_high:3.8674621464451775e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.76805866532959e-05 - rollout_corr/rollout_is_max:3.7063148021698 - rollout_corr/rollout_is_min:0.2912478744983673 - rollout_corr/rollout_is_std:0.0409952737390995 - rollout_corr/rollout_is_eff_sample_size:0.9983218507269501 - rollout_corr/rollout_is_seq_mean:0.9999192357063293 - rollout_corr/rollout_is_seq_std:0.002459433162584901 - rollout_corr/rollout_is_seq_max:1.008528470993042 - rollout_corr/rollout_is_seq_min:0.9915392994880676 - rollout_corr/rollout_is_seq_max_deviation:0.008528470993041992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.277975270524621) - rollout_corr/training_log_ppl:np.float64(0.24159730068640783) - rollout_corr/kl:np.float64(0.0014997820293984887) - rollout_corr/k3_kl:np.float64(0.0011694539433619866) - rollout_corr/rollout_ppl:np.float64(1.2760190423578024) - rollout_corr/rollout_log_ppl:np.float64(0.2400975169875892) - rollout_corr/log_ppl_diff:np.float64(0.0014997836988186464) - rollout_corr/log_ppl_abs_diff:np.float64(0.002539189110393636) - rollout_corr/log_ppl_diff_max:np.float64(0.011486321687698364) - rollout_corr/log_ppl_diff_min:np.float64(-0.00882810354232788) - rollout_corr/ppl_ratio:np.float64(1.001505514839664) - rollout_corr/chi2_token:np.float64(0.0016600224189460278) - rollout_corr/chi2_seq:np.float64(0.7354465429671109) - actor/pg_loss:np.float64(0.00017330946866422892) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015464231032638054) - actor/ppo_kl:np.float64(0.0004333707376389384) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6127719506621361) - perf/mfu/actor:np.float64(0.06741122513113522) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(106.24171829223633) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.25390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00273381220176816 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00273381220176816 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:404.01171875 - response_length/max:2216.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.01171875 - response_length_non_aborted/max:2216.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:334.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.080232560634613e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.536420190706849) - timing_s/agent_loop/generate_sequences/max:np.float64(13.316242065280676) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.049403940458433) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.316242065280676) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:2216 - timing_s/gen:19.068678671494126 - timing_s/reward:0.05746416188776493 - timing_s/old_log_prob:2.5150575172156096 - timing_s/adv:0.1416328214108944 - timing_s/update_actor:10.08072930201888 - timing_s/step:31.964212140068412 - timing_s/stop_profile:0.00014040805399417877 - timing_per_token_ms/gen:0.1843684789416122 - timing_per_token_ms/update_actor:0.0532867247528472 - timing_per_token_ms/adv:0.0007486709487358237 - perf/total_num_tokens:189179 - perf/time_per_step:31.964212140068412 - perf/throughput:739.8078481138934 (TaskRunner pid=2099163) Training Progress: 23%|██▎ | 23/100 [22:09<1:02:12, 48.47s/it] (TaskRunner pid=2099163) step:24 - global_seqlen/min:14555 - global_seqlen/max:31176 - global_seqlen/minmax_diff:16621 - global_seqlen/balanced_min:20123 - global_seqlen/balanced_max:20634 - global_seqlen/mean:20209.125 - actor/entropy:0.21117329597473145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29998478293418884 - training/rollout_probs_diff_mean:0.005020960234105587 - training/rollout_probs_diff_std:0.013722977600991726 - training/rollout_actor_probs_pearson_corr:0.9980407357215881 - rollout_corr/rollout_is_mean:0.9998765587806702 - rollout_corr/rollout_is_ratio_fraction_high:3.355366789037362e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.5922781029948965e-05 - rollout_corr/rollout_is_max:2.400970697402954 - rollout_corr/rollout_is_min:0.23983050882816315 - rollout_corr/rollout_is_std:0.04097927734255791 - rollout_corr/rollout_is_eff_sample_size:0.9983230388235971 - rollout_corr/rollout_is_seq_mean:0.9998420476913452 - rollout_corr/rollout_is_seq_std:0.002557374769821763 - rollout_corr/rollout_is_seq_max:1.0084576606750488 - rollout_corr/rollout_is_seq_min:0.9929150342941284 - rollout_corr/rollout_is_seq_max_deviation:0.008457660675048828 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2810057001188397) - rollout_corr/training_log_ppl:np.float64(0.2444376761704916) - rollout_corr/kl:np.float64(0.0012209779730945058) - rollout_corr/k3_kl:np.float64(0.0010693179642657924) - rollout_corr/rollout_ppl:np.float64(1.2793983216397464) - rollout_corr/rollout_log_ppl:np.float64(0.24321669599157758) - rollout_corr/log_ppl_diff:np.float64(0.0012209801789140329) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025764188467292115) - rollout_corr/log_ppl_diff_max:np.float64(0.011098459362983704) - rollout_corr/log_ppl_diff_min:np.float64(-0.008527815341949463) - rollout_corr/ppl_ratio:np.float64(1.0012267434503883) - rollout_corr/chi2_token:np.float64(0.0018536767456680536) - rollout_corr/chi2_seq:np.float64(1.0982552233617753) - actor/pg_loss:np.float64(0.0001084102550521493) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001204582532409404) - actor/ppo_kl:np.float64(9.153693397934148e-05) - actor/pg_clipfrac_lower:np.float64(1.0643732821336016e-05) - actor/grad_norm:np.float64(0.641492635011673) - perf/mfu/actor:np.float64(0.056898364927665215) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.3012466430664) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.36328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.36328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010942690074443817 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010942690074443817 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:349.25390625 - response_length/max:2150.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:349.25390625 - response_length_non_aborted/max:2150.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.868107736110687e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0074804201722145) - timing_s/agent_loop/generate_sequences/max:np.float64(12.549119051545858) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2516225457002292) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.549119051545858) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2150 - timing_s/gen:18.52240716293454 - timing_s/reward:0.070555554702878 - timing_s/old_log_prob:2.476743496954441 - timing_s/adv:0.16664943657815456 - timing_s/update_actor:10.124946380034089 - timing_s/step:31.44615630246699 - timing_s/stop_profile:2.8703361749649048e-05 - timing_per_token_ms/gen:0.2071649069213898 - timing_per_token_ms/update_actor:0.06262608091662857 - timing_per_token_ms/adv:0.0010307808760779756 - perf/total_num_tokens:161673 - perf/time_per_step:31.44615630246699 - perf/throughput:642.6580344388408 (TaskRunner pid=2099163) Training Progress: 24%|██▍ | 24/100 [22:41<54:56, 43.37s/it] (TaskRunner pid=2099163) step:25 - global_seqlen/min:19247 - global_seqlen/max:30356 - global_seqlen/minmax_diff:11109 - global_seqlen/balanced_min:26672 - global_seqlen/balanced_max:27390 - global_seqlen/mean:26767.875 - actor/entropy:0.18973146378993988 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33718350529670715 - training/rollout_probs_diff_mean:0.004361078143119812 - training/rollout_probs_diff_std:0.012972155585885048 - training/rollout_actor_probs_pearson_corr:0.9981417059898376 - rollout_corr/rollout_is_mean:0.999921441078186 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.597230822080746e-05 - rollout_corr/rollout_is_max:1.7342352867126465 - rollout_corr/rollout_is_min:0.2947933077812195 - rollout_corr/rollout_is_std:0.038447305560112 - rollout_corr/rollout_is_eff_sample_size:0.9985237489301881 - rollout_corr/rollout_is_seq_mean:0.9999851584434509 - rollout_corr/rollout_is_seq_std:0.0023914307821542025 - rollout_corr/rollout_is_seq_max:1.012315273284912 - rollout_corr/rollout_is_seq_min:0.9918898344039917 - rollout_corr/rollout_is_seq_max_deviation:0.01231527328491211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2648270050995052) - rollout_corr/training_log_ppl:np.float64(0.23086260134004988) - rollout_corr/kl:np.float64(0.0009177963597579719) - rollout_corr/k3_kl:np.float64(0.001010191431845442) - rollout_corr/rollout_ppl:np.float64(1.263638133648783) - rollout_corr/rollout_log_ppl:np.float64(0.22994480418856256) - rollout_corr/log_ppl_diff:np.float64(0.0009177971514873207) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021577174484264106) - rollout_corr/log_ppl_diff_max:np.float64(0.01115649938583374) - rollout_corr/log_ppl_diff_min:np.float64(-0.010651543736457825) - rollout_corr/ppl_ratio:np.float64(1.000921874307096) - rollout_corr/chi2_token:np.float64(0.0022637455258518457) - rollout_corr/chi2_seq:np.float64(0.9759733870159835) - actor/pg_loss:np.float64(0.0001767532667145133) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011058543057060888) - actor/ppo_kl:np.float64(-2.0648154892555226e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.553518146276474) - perf/mfu/actor:np.float64(0.07675197121879002) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.21955585479736) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.021515218541026115 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.021515218541026115 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:473.68359375 - response_length/max:2715.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:473.68359375 - response_length_non_aborted/max:2715.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.8125 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.77803361415863e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.284107943996787) - timing_s/agent_loop/generate_sequences/max:np.float64(15.839517647400498) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.669048825882783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.839517647400498) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:2715 - timing_s/gen:21.6606355458498 - timing_s/reward:0.05967309698462486 - timing_s/old_log_prob:2.458761554211378 - timing_s/adv:0.14967934787273407 - timing_s/update_actor:9.980403181165457 - timing_s/step:34.411740532144904 - timing_s/stop_profile:0.00013089925050735474 - timing_per_token_ms/gen:0.17862526529815195 - timing_per_token_ms/update_actor:0.04660625461100973 - timing_per_token_ms/adv:0.0006989691368512352 - perf/total_num_tokens:214143 - perf/time_per_step:34.411740532144904 - perf/throughput:777.8704182369221 (TaskRunner pid=2099163) Training Progress: 25%|██▌ | 25/100 [23:15<50:52, 40.70s/it] (TaskRunner pid=2099163) step:26 - global_seqlen/min:14395 - global_seqlen/max:27159 - global_seqlen/minmax_diff:12764 - global_seqlen/balanced_min:22165 - global_seqlen/balanced_max:22193 - global_seqlen/mean:22185.625 - actor/entropy:0.1997486650943756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33525192737579346 - training/rollout_probs_diff_mean:0.004470215644687414 - training/rollout_probs_diff_std:0.012864506803452969 - training/rollout_actor_probs_pearson_corr:0.9982172250747681 - rollout_corr/rollout_is_mean:1.0001190900802612 - rollout_corr/rollout_is_ratio_fraction_high:1.0165391358896159e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.0992344515398145e-05 - rollout_corr/rollout_is_max:3.4254493713378906 - rollout_corr/rollout_is_min:0.33838894963264465 - rollout_corr/rollout_is_std:0.03890654817223549 - rollout_corr/rollout_is_eff_sample_size:0.9984888060234459 - rollout_corr/rollout_is_seq_mean:1.0002080202102661 - rollout_corr/rollout_is_seq_std:0.0026749917306005955 - rollout_corr/rollout_is_seq_max:1.0075410604476929 - rollout_corr/rollout_is_seq_min:0.9928621649742126 - rollout_corr/rollout_is_seq_max_deviation:0.007541060447692871 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2674571070820093) - rollout_corr/training_log_ppl:np.float64(0.23334544732642826) - rollout_corr/kl:np.float64(0.0006923889370149539) - rollout_corr/k3_kl:np.float64(0.0010116354250726545) - rollout_corr/rollout_ppl:np.float64(1.266531832050532) - rollout_corr/rollout_log_ppl:np.float64(0.23265305937093217) - rollout_corr/log_ppl_diff:np.float64(0.0006923879554960877) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022414404957089573) - rollout_corr/log_ppl_diff_max:np.float64(0.009537696838378906) - rollout_corr/log_ppl_diff_min:np.float64(-0.008276775479316711) - rollout_corr/ppl_ratio:np.float64(1.0006968146190047) - rollout_corr/chi2_token:np.float64(0.0026694873813539743) - rollout_corr/chi2_seq:np.float64(0.8543046184349805) - actor/pg_loss:np.float64(-2.134358510375023e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006741704369233048) - actor/ppo_kl:np.float64(8.118165057968696e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37092382460832596) - perf/mfu/actor:np.float64(0.06470432805583136) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.13681030273438) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005917528178542852 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005917528178542852 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:384.26953125 - response_length/max:1567.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:384.26953125 - response_length_non_aborted/max:1567.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010296143591403961 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9918509125709534) - timing_s/agent_loop/generate_sequences/max:np.float64(10.191927688196301) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6913370271213353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.191927688196301) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1567 - timing_s/gen:15.927538784220815 - timing_s/reward:0.06093833968043327 - timing_s/old_log_prob:2.489799652248621 - timing_s/adv:0.15363946184515953 - timing_s/update_actor:9.85607378743589 - timing_s/step:28.577587887644768 - timing_s/stop_profile:0.00011636503040790558 - timing_per_token_ms/gen:0.16190965797750212 - timing_per_token_ms/update_actor:0.055531869101253004 - timing_per_token_ms/adv:0.0008656475862476239 - perf/total_num_tokens:177485 - perf/time_per_step:28.577587887644768 - perf/throughput:776.3295169356029 (TaskRunner pid=2099163) Training Progress: 26%|██▌ | 26/100 [23:44<45:43, 37.08s/it] (TaskRunner pid=2099163) step:27 - global_seqlen/min:12650 - global_seqlen/max:28413 - global_seqlen/minmax_diff:15763 - global_seqlen/balanced_min:22006 - global_seqlen/balanced_max:22028 - global_seqlen/mean:22020.875 - actor/entropy:0.1973375827074051 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977966308594 - training/rollout_probs_diff_mean:0.00446832412853837 - training/rollout_probs_diff_std:0.012732885777950287 - training/rollout_actor_probs_pearson_corr:0.9982201457023621 - rollout_corr/rollout_is_mean:1.0000790357589722 - rollout_corr/rollout_is_ratio_fraction_high:2.0880532247247174e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.132079655188136e-05 - rollout_corr/rollout_is_max:3.3716399669647217 - rollout_corr/rollout_is_min:0.41315898299217224 - rollout_corr/rollout_is_std:0.03870687633752823 - rollout_corr/rollout_is_eff_sample_size:0.9985042566669848 - rollout_corr/rollout_is_seq_mean:1.0001776218414307 - rollout_corr/rollout_is_seq_std:0.002695156028494239 - rollout_corr/rollout_is_seq_max:1.01008939743042 - rollout_corr/rollout_is_seq_min:0.9929530024528503 - rollout_corr/rollout_is_seq_max_deviation:0.010089397430419922 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2657123478129506) - rollout_corr/training_log_ppl:np.float64(0.23218309982621577) - rollout_corr/kl:np.float64(0.0008745037185420301) - rollout_corr/k3_kl:np.float64(0.0010085490487199422) - rollout_corr/rollout_ppl:np.float64(1.264582925941795) - rollout_corr/rollout_log_ppl:np.float64(0.2313085944915656) - rollout_corr/log_ppl_diff:np.float64(0.0008745053346501663) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021427856263471767) - rollout_corr/log_ppl_diff_max:np.float64(0.011478304862976074) - rollout_corr/log_ppl_diff_min:np.float64(-0.008908957242965698) - rollout_corr/ppl_ratio:np.float64(1.0008788730483502) - rollout_corr/chi2_token:np.float64(0.0022939874324947596) - rollout_corr/chi2_seq:np.float64(0.9378205249086022) - actor/pg_loss:np.float64(4.4271000660955906e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008205796011679922) - actor/ppo_kl:np.float64(0.00014555726753329168) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.377008780837059) - perf/mfu/actor:np.float64(0.0638297454911889) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.12808609008789) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014912614598870277 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014912614598870277 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.15234375 - response_length/max:1612.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.15234375 - response_length_non_aborted/max:1612.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.0 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.013922393321991e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1771093513816595) - timing_s/agent_loop/generate_sequences/max:np.float64(10.33962314389646) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6523388316636556) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.33962314389646) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:414 - timing_s/agent_loop/slowest/response_length:1592 - timing_s/gen:16.076986119151115 - timing_s/reward:0.05911053717136383 - timing_s/old_log_prob:2.4205230213701725 - timing_s/adv:0.15234399028122425 - timing_s/update_actor:9.950848061591387 - timing_s/step:28.75192417949438 - timing_s/stop_profile:0.0001438334584236145 - timing_per_token_ms/gen:0.16784801185127962 - timing_per_token_ms/update_actor:0.05648531258176268 - timing_per_token_ms/adv:0.0008647703047745846 - perf/total_num_tokens:176167 - perf/time_per_step:28.75192417949438 - perf/throughput:765.89221863993 (TaskRunner pid=2099163) Training Progress: 27%|██▋ | 27/100 [24:12<42:05, 34.60s/it] (TaskRunner pid=2099163) step:28 - global_seqlen/min:17510 - global_seqlen/max:32402 - global_seqlen/minmax_diff:14892 - global_seqlen/balanced_min:24287 - global_seqlen/balanced_max:24336 - global_seqlen/mean:24323.75 - actor/entropy:0.19081273674964905 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35365766286849976 - training/rollout_probs_diff_mean:0.004214483313262463 - training/rollout_probs_diff_std:0.012326523661613464 - training/rollout_actor_probs_pearson_corr:0.9983044266700745 - rollout_corr/rollout_is_mean:0.9999059438705444 - rollout_corr/rollout_is_ratio_fraction_high:1.8814322174875997e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.407160541741177e-05 - rollout_corr/rollout_is_max:2.637937545776367 - rollout_corr/rollout_is_min:0.28466564416885376 - rollout_corr/rollout_is_std:0.037621427327394485 - rollout_corr/rollout_is_eff_sample_size:0.9985862719338552 - rollout_corr/rollout_is_seq_mean:0.999921977519989 - rollout_corr/rollout_is_seq_std:0.0030063134618103504 - rollout_corr/rollout_is_seq_max:1.014600157737732 - rollout_corr/rollout_is_seq_min:0.990456759929657 - rollout_corr/rollout_is_seq_max_deviation:0.014600157737731934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2607678300701082) - rollout_corr/training_log_ppl:np.float64(0.22826670162612572) - rollout_corr/kl:np.float64(0.0009433091491317658) - rollout_corr/k3_kl:np.float64(0.0009751564648468047) - rollout_corr/rollout_ppl:np.float64(1.2595306900329888) - rollout_corr/rollout_log_ppl:np.float64(0.22732339147478342) - rollout_corr/log_ppl_diff:np.float64(0.0009433101513423026) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022531004797201604) - rollout_corr/log_ppl_diff_max:np.float64(0.010257154703140259) - rollout_corr/log_ppl_diff_min:np.float64(-0.012321293354034424) - rollout_corr/ppl_ratio:np.float64(1.0009483909234405) - rollout_corr/chi2_token:np.float64(0.002010138938203454) - rollout_corr/chi2_seq:np.float64(1.263109202729538) - actor/pg_loss:np.float64(0.0003952222177758813) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000980165132887123) - actor/ppo_kl:np.float64(-2.858693859408845e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5482866317033768) - perf/mfu/actor:np.float64(0.07006718373692834) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.16790771484375) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015940433368086815 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015940433368086815 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:415.2421875 - response_length/max:1856.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:415.2421875 - response_length_non_aborted/max:1856.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:344.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.274482727050781e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9281809013336897) - timing_s/agent_loop/generate_sequences/max:np.float64(11.828542461618781) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.940465694955492) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.828542461618781) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1856 - timing_s/gen:17.50202799960971 - timing_s/reward:0.06275253742933273 - timing_s/old_log_prob:2.5015085004270077 - timing_s/adv:0.1598478928208351 - timing_s/update_actor:9.977494372054935 - timing_s/step:30.29320042952895 - timing_s/stop_profile:0.00013585761189460754 - timing_per_token_ms/gen:0.1646443905063847 - timing_per_token_ms/update_actor:0.05127444561413708 - timing_per_token_ms/adv:0.0008214599559115839 - perf/total_num_tokens:194590 - perf/time_per_step:30.29320042952895 - perf/throughput:802.9442137216344 (TaskRunner pid=2099163) Training Progress: 28%|██▊ | 28/100 [24:43<39:59, 33.32s/it] (TaskRunner pid=2099163) step:29 - global_seqlen/min:11576 - global_seqlen/max:26504 - global_seqlen/minmax_diff:14928 - global_seqlen/balanced_min:20443 - global_seqlen/balanced_max:20468 - global_seqlen/mean:20457.125 - actor/entropy:0.19512885808944702 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2994707226753235 - training/rollout_probs_diff_mean:0.004517699591815472 - training/rollout_probs_diff_std:0.012806838378310204 - training/rollout_actor_probs_pearson_corr:0.9982004165649414 - rollout_corr/rollout_is_mean:0.9999142289161682 - rollout_corr/rollout_is_ratio_fraction_high:2.2683709175908007e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.5367418351816013e-05 - rollout_corr/rollout_is_max:2.5106632709503174 - rollout_corr/rollout_is_min:0.35614722967147827 - rollout_corr/rollout_is_std:0.03914633393287659 - rollout_corr/rollout_is_eff_sample_size:0.9984696716584548 - rollout_corr/rollout_is_seq_mean:0.9998313188552856 - rollout_corr/rollout_is_seq_std:0.0028376157861202955 - rollout_corr/rollout_is_seq_max:1.0107405185699463 - rollout_corr/rollout_is_seq_min:0.9908356666564941 - rollout_corr/rollout_is_seq_max_deviation:0.010740518569946289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2674566823989153) - rollout_corr/training_log_ppl:np.float64(0.23359788639936596) - rollout_corr/kl:np.float64(0.0009326249134602449) - rollout_corr/k3_kl:np.float64(0.0010803121514300074) - rollout_corr/rollout_ppl:np.float64(1.2662694193422794) - rollout_corr/rollout_log_ppl:np.float64(0.2326652607152937) - rollout_corr/log_ppl_diff:np.float64(0.0009326256840722635) - rollout_corr/log_ppl_abs_diff:np.float64(0.002509462457965128) - rollout_corr/log_ppl_diff_max:np.float64(0.009627044200897217) - rollout_corr/log_ppl_diff_min:np.float64(-0.010430455207824707) - rollout_corr/ppl_ratio:np.float64(1.0009380243718624) - rollout_corr/chi2_token:np.float64(0.0024118823930621147) - rollout_corr/chi2_seq:np.float64(1.6191739679779857) - actor/pg_loss:np.float64(0.00010256993118673563) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001801439915652736) - actor/ppo_kl:np.float64(-0.0001434271987896807) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5583970844745636) - perf/mfu/actor:np.float64(0.05960628937352358) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.0120849609375) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:2 - critic/score/mean:0.296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008349022828042507 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008349022828042507 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.41015625 - response_length/max:1501.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.41015625 - response_length_non_aborted/max:1501.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00034629926085472107 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7719959896057844) - timing_s/agent_loop/generate_sequences/max:np.float64(9.23289637081325) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.100054759372142) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.23289637081325) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1501 - timing_s/gen:15.16298632323742 - timing_s/reward:0.05156240798532963 - timing_s/old_log_prob:2.6052888948470354 - timing_s/adv:0.14285925030708313 - timing_s/update_actor:9.606981605291367 - timing_s/step:27.679332403466105 - timing_s/stop_profile:0.00011761859059333801 - timing_per_token_ms/gen:0.17197638992432054 - timing_per_token_ms/update_actor:0.05870192906683715 - timing_per_token_ms/adv:0.0008729186671335973 - perf/total_num_tokens:163657 - perf/time_per_step:27.679332403466105 - perf/throughput:739.075809409272 (TaskRunner pid=2099163) Training Progress: 29%|██▉ | 29/100 [25:11<37:36, 31.78s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score of a protein sequence typically requires computational analysis using tools like Rosetta, which calculate the energy of a protein structure. The sequence provided ('GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR') would need to be input into such a tool to determine its folding stability score. Since no computational tool or database is available here to calculate this score, it is not possible to determine the exact folding stability score without additional information or computational resources. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2099163) step:30 - global_seqlen/min:14663 - global_seqlen/max:34744 - global_seqlen/minmax_diff:20081 - global_seqlen/balanced_min:23097 - global_seqlen/balanced_max:23449 - global_seqlen/mean:23160.25 - actor/entropy:0.1929744929075241 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34270334243774414 - training/rollout_probs_diff_mean:0.004414478316903114 - training/rollout_probs_diff_std:0.012888044118881226 - training/rollout_actor_probs_pearson_corr:0.9981456995010376 - rollout_corr/rollout_is_mean:1.000051736831665 - rollout_corr/rollout_is_ratio_fraction_high:1.9112783775199205e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.556392615195364e-05 - rollout_corr/rollout_is_max:2.849820375442505 - rollout_corr/rollout_is_min:0.13807697594165802 - rollout_corr/rollout_is_std:0.03877303749322891 - rollout_corr/rollout_is_eff_sample_size:0.9984991460165831 - rollout_corr/rollout_is_seq_mean:0.9999996423721313 - rollout_corr/rollout_is_seq_std:0.002868861658498645 - rollout_corr/rollout_is_seq_max:1.0089269876480103 - rollout_corr/rollout_is_seq_min:0.989052414894104 - rollout_corr/rollout_is_seq_max_deviation:0.010947585105895996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2640721159987152) - rollout_corr/training_log_ppl:np.float64(0.23040971330192406) - rollout_corr/kl:np.float64(0.001080727721191721) - rollout_corr/k3_kl:np.float64(0.0009677562554202268) - rollout_corr/rollout_ppl:np.float64(1.2626397949643433) - rollout_corr/rollout_log_ppl:np.float64(0.22932898352155462) - rollout_corr/log_ppl_diff:np.float64(0.0010807297803694382) - rollout_corr/log_ppl_abs_diff:np.float64(0.002270795885124244) - rollout_corr/log_ppl_diff_max:np.float64(0.009988278150558472) - rollout_corr/log_ppl_diff_min:np.float64(-0.008100628852844238) - rollout_corr/ppl_ratio:np.float64(1.0010852506384254) - rollout_corr/chi2_token:np.float64(0.001721000298857689) - rollout_corr/chi2_seq:np.float64(0.7308730445802212) - actor/pg_loss:np.float64(0.00017032702453434467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00098742782915906) - actor/ppo_kl:np.float64(0.0001257946889916539) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.562660925090313) - perf/mfu/actor:np.float64(0.06606835743079029) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.89504623413086) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3825) - val-aux/sciknoweval/reward/std@16:np.float64(0.2710825289783396) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.49848) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.23249398932762172) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.26538000000000006) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.23000654365019318) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.38264) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2717389471729169) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5914200000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16401609741962153) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.17232) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16309037403563342) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4011) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2275492223932495) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.65118) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1002078489826257) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.11272) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0901338588728539) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4145999999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.17744782663880587) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.68952) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05884117194239494) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.08363999999999999) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04403769261963444) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.42375999999999997) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13151628701261814) - val-aux/sciknoweval/score/mean@16:np.float64(0.3825) - val-aux/sciknoweval/score/std@16:np.float64(0.2710825289783396) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.49848) - val-aux/sciknoweval/score/best@2/std:np.float64(0.23249398932762172) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.26538000000000006) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.23000654365019318) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.38264) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2717389471729169) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5914200000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16401609741962153) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.17232) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16309037403563342) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.4011) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2275492223932495) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.65118) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1002078489826257) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.11272) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0901338588728539) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4145999999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.17744782663880587) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.68952) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05884117194239494) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.08363999999999999) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04403769261963444) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.42375999999999997) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13151628701261814) - val-core/sciknoweval/acc/mean@16:np.float64(0.3825) - val-aux/sciknoweval/acc/std@16:np.float64(0.2710825289783396) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.49848) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.23249398932762172) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.26538000000000006) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.23000654365019318) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.38264) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2717389471729169) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5914200000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16401609741962153) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.17232) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16309037403563342) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.4011) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2275492223932495) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.65118) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1002078489826257) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.11272) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0901338588728539) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4145999999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.17744782663880587) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.68952) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05884117194239494) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.08363999999999999) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04403769261963444) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.42375999999999997) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13151628701261814) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.985) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.05195859001739712) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9987600000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.014549558441390551) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9708800000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06937229914476689) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9849800000000001) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.05188589399017503) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.94702) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08675116093152056) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99582) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.02779570240419907) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9096600000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.09741143940075572) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99928) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010537634702780806) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8622200000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08961444045723323) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711644) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:2 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012717168778181076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012717168778181076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:408.7578125 - response_length/max:2136.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:408.7578125 - response_length_non_aborted/max:2136.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.0 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.426474571228027e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.006382143124938) - timing_s/agent_loop/generate_sequences/max:np.float64(13.268630873411894) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9260140853366465) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.268630873411894) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:2136 - timing_s/gen:19.38935353793204 - timing_s/reward:0.06270048767328262 - timing_s/old_log_prob:2.4441376700997353 - timing_s/adv:0.1214206013828516 - timing_s/update_actor:9.896830067038536 - timing_s/step:32.002956476062536 - timing_s/testing:77.46762142144144 - timing_s/save_checkpoint:6.66411729156971 - timing_s/stop_profile:9.995326399803162e-05 - timing_per_token_ms/gen:0.18529226828550713 - timing_per_token_ms/update_actor:0.05341495702247674 - timing_per_token_ms/adv:0.0006553286416535422 - perf/total_num_tokens:185282 - perf/time_per_step:32.002956476062536 - perf/throughput:723.6909507821043 (TaskRunner pid=2099163) Training Progress: 30%|███ | 30/100 [27:07<1:06:36, 57.10s/it] (TaskRunner pid=2099163) step:31 - global_seqlen/min:15610 - global_seqlen/max:32987 - global_seqlen/minmax_diff:17377 - global_seqlen/balanced_min:22697 - global_seqlen/balanced_max:22706 - global_seqlen/mean:22704.125 - actor/entropy:0.20467621088027954 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36793190240859985 - training/rollout_probs_diff_mean:0.0046606664545834064 - training/rollout_probs_diff_std:0.013037621974945068 - training/rollout_actor_probs_pearson_corr:0.998202919960022 - rollout_corr/rollout_is_mean:0.9999342560768127 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.056140409782529e-05 - rollout_corr/rollout_is_max:1.8634686470031738 - rollout_corr/rollout_is_min:0.08700346946716309 - rollout_corr/rollout_is_std:0.03880991414189339 - rollout_corr/rollout_is_eff_sample_size:0.998495937030278 - rollout_corr/rollout_is_seq_mean:0.9998412132263184 - rollout_corr/rollout_is_seq_std:0.0024558771401643753 - rollout_corr/rollout_is_seq_max:1.008089303970337 - rollout_corr/rollout_is_seq_min:0.9901171326637268 - rollout_corr/rollout_is_seq_max_deviation:0.009882867336273193 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2660416262224317) - rollout_corr/training_log_ppl:np.float64(0.2328464862657711) - rollout_corr/kl:np.float64(0.0014669340525781394) - rollout_corr/k3_kl:np.float64(0.0012662450398579495) - rollout_corr/rollout_ppl:np.float64(1.2641333830542862) - rollout_corr/rollout_log_ppl:np.float64(0.2313795514346566) - rollout_corr/log_ppl_diff:np.float64(0.0014669348311144859) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024422181595582515) - rollout_corr/log_ppl_diff_max:np.float64(0.0128212571144104) - rollout_corr/log_ppl_diff_min:np.float64(-0.01384890079498291) - rollout_corr/ppl_ratio:np.float64(1.0014724703505635) - rollout_corr/chi2_token:np.float64(0.002137396950274706) - rollout_corr/chi2_seq:np.float64(0.24568918230943382) - actor/pg_loss:np.float64(-4.5661814510822296e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002057411175883317) - actor/ppo_kl:np.float64(0.00041522945140526346) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5537090376019478) - perf/mfu/actor:np.float64(0.0655864265337439) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.2952651977539) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:2 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009520757012069225 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009520757012069225 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:387.00390625 - response_length/max:1622.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:387.00390625 - response_length_non_aborted/max:1622.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000301973894238472 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0549929738044739) - timing_s/agent_loop/generate_sequences/max:np.float64(10.455312624573708) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.841500379559875) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.455312624573708) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1622 - timing_s/gen:16.36735489219427 - timing_s/reward:0.055881038308143616 - timing_s/old_log_prob:2.546142552047968 - timing_s/adv:0.16691702045500278 - timing_s/update_actor:9.986165532842278 - timing_s/step:29.223748456686735 - timing_s/stop_profile:0.00011165998876094818 - timing_per_token_ms/gen:0.16520499926513046 - timing_per_token_ms/update_actor:0.05497990746638705 - timing_per_token_ms/adv:0.0009189795932182081 - perf/total_num_tokens:181633 - perf/time_per_step:29.223748456686735 - perf/throughput:776.9066666328026 (TaskRunner pid=2099163) Training Progress: 31%|███ | 31/100 [27:36<56:03, 48.75s/it] (TaskRunner pid=2099163) step:32 - global_seqlen/min:16456 - global_seqlen/max:30003 - global_seqlen/minmax_diff:13547 - global_seqlen/balanced_min:22662 - global_seqlen/balanced_max:22713 - global_seqlen/mean:22703.375 - actor/entropy:0.19235235452651978 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6836323142051697 - training/rollout_probs_diff_mean:0.004288580734282732 - training/rollout_probs_diff_std:0.01270546205341816 - training/rollout_actor_probs_pearson_corr:0.9982287287712097 - rollout_corr/rollout_is_mean:1.000108003616333 - rollout_corr/rollout_is_ratio_fraction_high:4.2221261537633836e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.0555315384408459e-05 - rollout_corr/rollout_is_max:3.5419375896453857 - rollout_corr/rollout_is_min:0.4771340489387512 - rollout_corr/rollout_is_std:0.03740055859088898 - rollout_corr/rollout_is_eff_sample_size:0.9986033898627913 - rollout_corr/rollout_is_seq_mean:1.0001899003982544 - rollout_corr/rollout_is_seq_std:0.002721930155530572 - rollout_corr/rollout_is_seq_max:1.009945273399353 - rollout_corr/rollout_is_seq_min:0.9931846857070923 - rollout_corr/rollout_is_seq_max_deviation:0.009945273399353027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2627140362747014) - rollout_corr/training_log_ppl:np.float64(0.22928791525191627) - rollout_corr/kl:np.float64(0.0009696064506714563) - rollout_corr/k3_kl:np.float64(0.0009657189097538321) - rollout_corr/rollout_ppl:np.float64(1.2614360507577658) - rollout_corr/rollout_log_ppl:np.float64(0.228318307781592) - rollout_corr/log_ppl_diff:np.float64(0.000969607470324263) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024012566718738526) - rollout_corr/log_ppl_diff_max:np.float64(0.008903712034225464) - rollout_corr/log_ppl_diff_min:np.float64(-0.007482409477233887) - rollout_corr/ppl_ratio:np.float64(1.0009746660944074) - rollout_corr/chi2_token:np.float64(0.0019679118413478136) - rollout_corr/chi2_seq:np.float64(0.5128269151318818) - actor/pg_loss:np.float64(0.0006739883683621883) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011445526070019696) - actor/ppo_kl:np.float64(0.00029994351070516956) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5374793410301208) - perf/mfu/actor:np.float64(0.06520734277390705) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.37225103378296) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:2 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0055389017798006535 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0055389017798006535 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:370.07421875 - response_length/max:1794.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:370.07421875 - response_length_non_aborted/max:1794.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:339.40625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.293854236602783e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0003391448408365) - timing_s/agent_loop/generate_sequences/max:np.float64(11.236332735046744) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5134957447080524) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.236332735046744) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1794 - timing_s/gen:17.538983415812254 - timing_s/reward:0.05691481754183769 - timing_s/old_log_prob:2.4536229204386473 - timing_s/adv:0.14731214381754398 - timing_s/update_actor:10.00981381163001 - timing_s/step:30.308992091566324 - timing_s/stop_profile:0.0001265406608581543 - timing_per_token_ms/gen:0.18512949699503112 - timing_per_token_ms/update_actor:0.05511192615431632 - timing_per_token_ms/adv:0.0008110696307131868 - perf/total_num_tokens:181627 - perf/time_per_step:30.308992091566324 - perf/throughput:749.0640048804976 (TaskRunner pid=2099163) Training Progress: 32%|███▏ | 32/100 [28:07<49:00, 43.24s/it] (TaskRunner pid=2099163) step:33 - global_seqlen/min:14153 - global_seqlen/max:31822 - global_seqlen/minmax_diff:17669 - global_seqlen/balanced_min:24569 - global_seqlen/balanced_max:24592 - global_seqlen/mean:24586.5 - actor/entropy:0.19219401478767395 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38829466700553894 - training/rollout_probs_diff_mean:0.004298380576074123 - training/rollout_probs_diff_std:0.012502633035182953 - training/rollout_actor_probs_pearson_corr:0.9982357621192932 - rollout_corr/rollout_is_mean:0.9999257922172546 - rollout_corr/rollout_is_ratio_fraction_high:1.8696480765356682e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.674120282288641e-05 - rollout_corr/rollout_is_max:3.7463631629943848 - rollout_corr/rollout_is_min:0.3203428089618683 - rollout_corr/rollout_is_std:0.03699354827404022 - rollout_corr/rollout_is_eff_sample_size:0.9986331098833465 - rollout_corr/rollout_is_seq_mean:0.9999837875366211 - rollout_corr/rollout_is_seq_std:0.002320910105481744 - rollout_corr/rollout_is_seq_max:1.0067297220230103 - rollout_corr/rollout_is_seq_min:0.9923191666603088 - rollout_corr/rollout_is_seq_max_deviation:0.007680833339691162 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2546508102677763) - rollout_corr/training_log_ppl:np.float64(0.22293072564934846) - rollout_corr/kl:np.float64(0.0007883897724163154) - rollout_corr/k3_kl:np.float64(0.0008786465269849941) - rollout_corr/rollout_ppl:np.float64(1.2536278427578509) - rollout_corr/rollout_log_ppl:np.float64(0.22214233508566394) - rollout_corr/log_ppl_diff:np.float64(0.0007883905636845157) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019512200815370306) - rollout_corr/log_ppl_diff_max:np.float64(0.00915798544883728) - rollout_corr/log_ppl_diff_min:np.float64(-0.007831752300262451) - rollout_corr/ppl_ratio:np.float64(1.0007919785566628) - rollout_corr/chi2_token:np.float64(0.0019339756108820438) - rollout_corr/chi2_seq:np.float64(0.8230726562906057) - actor/pg_loss:np.float64(6.270431913435459e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013706558816011238) - actor/ppo_kl:np.float64(-7.741855117160412e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6440355181694031) - perf/mfu/actor:np.float64(0.0710191135212451) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.29852676391602) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:2 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00058893917594105 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00058893917594105 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:417.859375 - response_length/max:1500.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:417.859375 - response_length_non_aborted/max:1500.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:350.46875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.15168023109436e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3214661497622728) - timing_s/agent_loop/generate_sequences/max:np.float64(10.396715246140957) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.068895058728231) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.396715246140957) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1500 - timing_s/gen:16.118461679667234 - timing_s/reward:0.06390342302620411 - timing_s/old_log_prob:2.3601754792034626 - timing_s/adv:0.18045086413621902 - timing_s/update_actor:9.997449096292257 - timing_s/step:28.806253915652633 - timing_s/stop_profile:0.00010740384459495544 - timing_per_token_ms/gen:0.1506792588683696 - timing_per_token_ms/update_actor:0.05082793960248641 - timing_per_token_ms/adv:0.00091742858955229 - perf/total_num_tokens:196692 - perf/time_per_step:28.806253915652633 - perf/throughput:853.512576539509 (TaskRunner pid=2099163) Training Progress: 33%|███▎ | 33/100 [28:36<43:27, 38.93s/it] (TaskRunner pid=2099163) step:34 - global_seqlen/min:13165 - global_seqlen/max:31218 - global_seqlen/minmax_diff:18053 - global_seqlen/balanced_min:20844 - global_seqlen/balanced_max:21371 - global_seqlen/mean:20953.375 - actor/entropy:0.20621417462825775 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47441184520721436 - training/rollout_probs_diff_mean:0.004632072988897562 - training/rollout_probs_diff_std:0.013067140243947506 - training/rollout_actor_probs_pearson_corr:0.9982191324234009 - rollout_corr/rollout_is_mean:0.9996575117111206 - rollout_corr/rollout_is_ratio_fraction_high:1.123557649407303e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.741345714544877e-05 - rollout_corr/rollout_is_max:3.048807144165039 - rollout_corr/rollout_is_min:0.1631394475698471 - rollout_corr/rollout_is_std:0.039033498615026474 - rollout_corr/rollout_is_eff_sample_size:0.9984777531641464 - rollout_corr/rollout_is_seq_mean:0.9995238184928894 - rollout_corr/rollout_is_seq_std:0.003114572959020734 - rollout_corr/rollout_is_seq_max:1.0097143650054932 - rollout_corr/rollout_is_seq_min:0.9892820119857788 - rollout_corr/rollout_is_seq_max_deviation:0.010717988014221191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2769425138831139) - rollout_corr/training_log_ppl:np.float64(0.24088622524868697) - rollout_corr/kl:np.float64(0.0017434109029323963) - rollout_corr/k3_kl:np.float64(0.0011406559008833028) - rollout_corr/rollout_ppl:np.float64(1.2746948134154081) - rollout_corr/rollout_log_ppl:np.float64(0.23914281158067752) - rollout_corr/log_ppl_diff:np.float64(0.0017434136680094525) - rollout_corr/log_ppl_abs_diff:np.float64(0.002788226425764151) - rollout_corr/log_ppl_diff_max:np.float64(0.012063741683959961) - rollout_corr/log_ppl_diff_min:np.float64(-0.007839560508728027) - rollout_corr/ppl_ratio:np.float64(1.001750238938257) - rollout_corr/chi2_token:np.float64(0.0010821421165019274) - rollout_corr/chi2_seq:np.float64(0.3799931020475924) - actor/pg_loss:np.float64(0.00013993948232382536) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016885243967408314) - actor/ppo_kl:np.float64(0.0003524259806644281) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5224123150110245) - perf/mfu/actor:np.float64(0.06054932786011055) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.48657608032227) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:2 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01508656982332468 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01508656982332468 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:347.66796875 - response_length/max:2290.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.66796875 - response_length_non_aborted/max:2290.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.014829993247986e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0219873953610659) - timing_s/agent_loop/generate_sequences/max:np.float64(13.463359158486128) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3108865806207177) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.463359158486128) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:414 - timing_s/agent_loop/slowest/response_length:2290 - timing_s/gen:19.49041328765452 - timing_s/reward:0.05500324070453644 - timing_s/old_log_prob:2.408570285886526 - timing_s/adv:0.15743279829621315 - timing_s/update_actor:9.805676659569144 - timing_s/step:32.01260624267161 - timing_s/stop_profile:0.00011429190635681152 - timing_per_token_ms/gen:0.21898602617501117 - timing_per_token_ms/update_actor:0.05849700024202034 - timing_per_token_ms/adv:0.0009391852046282111 - perf/total_num_tokens:167627 - perf/time_per_step:32.01260624267161 - perf/throughput:654.5351178583496 (TaskRunner pid=2099163) Training Progress: 34%|███▍ | 34/100 [29:08<40:33, 36.86s/it] (TaskRunner pid=2099163) step:35 - global_seqlen/min:10930 - global_seqlen/max:29063 - global_seqlen/minmax_diff:18133 - global_seqlen/balanced_min:21182 - global_seqlen/balanced_max:21279 - global_seqlen/mean:21202.375 - actor/entropy:0.20077916979789734 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4047938883304596 - training/rollout_probs_diff_mean:0.004468213766813278 - training/rollout_probs_diff_std:0.012723792344331741 - training/rollout_actor_probs_pearson_corr:0.9982172250747681 - rollout_corr/rollout_is_mean:0.9999480247497559 - rollout_corr/rollout_is_ratio_fraction_high:1.1354991329426412e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.4064974897773936e-05 - rollout_corr/rollout_is_max:2.361192226409912 - rollout_corr/rollout_is_min:0.35472583770751953 - rollout_corr/rollout_is_std:0.03832307830452919 - rollout_corr/rollout_is_eff_sample_size:0.9985334953471807 - rollout_corr/rollout_is_seq_mean:0.9999346137046814 - rollout_corr/rollout_is_seq_std:0.0025622511748224497 - rollout_corr/rollout_is_seq_max:1.0083149671554565 - rollout_corr/rollout_is_seq_min:0.9895864129066467 - rollout_corr/rollout_is_seq_max_deviation:0.010413587093353271 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2627762155607343) - rollout_corr/training_log_ppl:np.float64(0.22958877607015893) - rollout_corr/kl:np.float64(0.001482734944080022) - rollout_corr/k3_kl:np.float64(0.001303993512976831) - rollout_corr/rollout_ppl:np.float64(1.2608598791994154) - rollout_corr/rollout_log_ppl:np.float64(0.22810603884863667) - rollout_corr/log_ppl_diff:np.float64(0.0014827372215222567) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028337313269730657) - rollout_corr/log_ppl_diff_max:np.float64(0.01417878270149231) - rollout_corr/log_ppl_diff_min:np.float64(-0.015369802713394165) - rollout_corr/ppl_ratio:np.float64(1.0014910355675966) - rollout_corr/chi2_token:np.float64(0.002457682741805911) - rollout_corr/chi2_seq:np.float64(1.00893239909783) - actor/pg_loss:np.float64(0.0003223039675503969) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018587432675758464) - actor/ppo_kl:np.float64(0.000577584969583711) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.604684367775917) - perf/mfu/actor:np.float64(0.06179983949184756) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.36828994750977) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:2 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004048054106533527 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004048054106533527 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.01171875 - response_length/max:1733.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.01171875 - response_length_non_aborted/max:1733.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.5625 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.777901530265808e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7171750254929066) - timing_s/agent_loop/generate_sequences/max:np.float64(8.89642646163702) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0253594234527554) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.89642646163702) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1293 - timing_s/gen:15.099879954010248 - timing_s/reward:0.04901489429175854 - timing_s/old_log_prob:2.3395367600023746 - timing_s/adv:0.11664748005568981 - timing_s/update_actor:9.723327906802297 - timing_s/step:27.36307036317885 - timing_s/stop_profile:0.00013891421258449554 - timing_per_token_ms/gen:0.17145900228246957 - timing_per_token_ms/update_actor:0.05732452087798122 - timing_per_token_ms/adv:0.00068770291096923 - perf/total_num_tokens:169619 - perf/time_per_step:27.36307036317885 - perf/throughput:774.853651969225 (TaskRunner pid=2099163) Training Progress: 35%|███▌ | 35/100 [29:35<36:51, 34.03s/it] (TaskRunner pid=2099163) step:36 - global_seqlen/min:14194 - global_seqlen/max:29137 - global_seqlen/minmax_diff:14943 - global_seqlen/balanced_min:20936 - global_seqlen/balanced_max:21371 - global_seqlen/mean:21018.125 - actor/entropy:0.20421844720840454 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5412930250167847 - training/rollout_probs_diff_mean:0.00441008573397994 - training/rollout_probs_diff_std:0.012608489021658897 - training/rollout_actor_probs_pearson_corr:0.9983038306236267 - rollout_corr/rollout_is_mean:0.9999710917472839 - rollout_corr/rollout_is_ratio_fraction_high:2.242931986984331e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.607329876511358e-05 - rollout_corr/rollout_is_max:2.2185213565826416 - rollout_corr/rollout_is_min:0.007402495015412569 - rollout_corr/rollout_is_std:0.03787250071763992 - rollout_corr/rollout_is_eff_sample_size:0.9985674904370404 - rollout_corr/rollout_is_seq_mean:0.999952495098114 - rollout_corr/rollout_is_seq_std:0.0027767117135226727 - rollout_corr/rollout_is_seq_max:1.008007526397705 - rollout_corr/rollout_is_seq_min:0.9892098307609558 - rollout_corr/rollout_is_seq_max_deviation:0.01079016923904419 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.27850745851174) - rollout_corr/training_log_ppl:np.float64(0.24139065398776438) - rollout_corr/kl:np.float64(0.0012924192194674333) - rollout_corr/k3_kl:np.float64(0.0011641595515641256) - rollout_corr/rollout_ppl:np.float64(1.2768099810928106) - rollout_corr/rollout_log_ppl:np.float64(0.24009823182132095) - rollout_corr/log_ppl_diff:np.float64(0.0012924221664434299) - rollout_corr/log_ppl_abs_diff:np.float64(0.002743791919783689) - rollout_corr/log_ppl_diff_max:np.float64(0.017144635319709778) - rollout_corr/log_ppl_diff_min:np.float64(-0.00719086080789566) - rollout_corr/ppl_ratio:np.float64(1.0013000178150833) - rollout_corr/chi2_token:np.float64(0.002002155175432563) - rollout_corr/chi2_seq:np.float64(0.773878968320787) - actor/pg_loss:np.float64(-6.605871021747589e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014383478630861646) - actor/ppo_kl:np.float64(0.00034995766268330897) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5709159523248672) - perf/mfu/actor:np.float64(0.061025477795435934) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.43119430541992) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:2 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012700601946562529 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012700601946562529 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:348.31640625 - response_length/max:2327.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.31640625 - response_length_non_aborted/max:2327.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:308.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013381987810134888 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9047284964472055) - timing_s/agent_loop/generate_sequences/max:np.float64(13.386295091360807) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3439106781152077) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.386295091360807) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:2327 - timing_s/gen:19.22059991583228 - timing_s/reward:0.06047280505299568 - timing_s/old_log_prob:2.4186214227229357 - timing_s/adv:0.13746901601552963 - timing_s/update_actor:9.880311040207744 - timing_s/step:31.805876852944493 - timing_s/stop_profile:0.00011717155575752258 - timing_per_token_ms/gen:0.215552489271297 - timing_per_token_ms/update_actor:0.05876065919419396 - timing_per_token_ms/adv:0.0008175623183295943 - perf/total_num_tokens:168145 - perf/time_per_step:31.805876852944493 - perf/throughput:660.8252021215445 (TaskRunner pid=2099163) Training Progress: 36%|███▌ | 36/100 [30:07<35:36, 33.38s/it] (TaskRunner pid=2099163) step:37 - global_seqlen/min:10542 - global_seqlen/max:30603 - global_seqlen/minmax_diff:20061 - global_seqlen/balanced_min:19813 - global_seqlen/balanced_max:26160 - global_seqlen/mean:20656.375 - actor/entropy:0.19215349853038788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102194607257843 - training/rollout_probs_diff_mean:0.004060864448547363 - training/rollout_probs_diff_std:0.012080005370080471 - training/rollout_actor_probs_pearson_corr:0.9984062314033508 - rollout_corr/rollout_is_mean:0.9999275803565979 - rollout_corr/rollout_is_ratio_fraction_high:1.1167319826199673e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.583660094998777e-05 - rollout_corr/rollout_is_max:2.5254178047180176 - rollout_corr/rollout_is_min:0.23548689484596252 - rollout_corr/rollout_is_std:0.03610971197485924 - rollout_corr/rollout_is_eff_sample_size:0.9986974300643999 - rollout_corr/rollout_is_seq_mean:0.9998934268951416 - rollout_corr/rollout_is_seq_std:0.002943673636764288 - rollout_corr/rollout_is_seq_max:1.009252905845642 - rollout_corr/rollout_is_seq_min:0.9870862364768982 - rollout_corr/rollout_is_seq_max_deviation:0.012913763523101807 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2790742232464254) - rollout_corr/training_log_ppl:np.float64(0.2423434283118695) - rollout_corr/kl:np.float64(0.0010189136970133461) - rollout_corr/k3_kl:np.float64(0.0010748106194000684) - rollout_corr/rollout_ppl:np.float64(1.277731901500374) - rollout_corr/rollout_log_ppl:np.float64(0.24132451312470948) - rollout_corr/log_ppl_diff:np.float64(0.0010189151871600188) - rollout_corr/log_ppl_abs_diff:np.float64(0.002592919896414969) - rollout_corr/log_ppl_diff_max:np.float64(0.015276342630386353) - rollout_corr/log_ppl_diff_min:np.float64(-0.011511757969856262) - rollout_corr/ppl_ratio:np.float64(1.0010252590291202) - rollout_corr/chi2_token:np.float64(0.0022940633352845907) - rollout_corr/chi2_seq:np.float64(0.5807141268160194) - actor/pg_loss:np.float64(4.6521308831870556e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013833769742177537) - actor/ppo_kl:np.float64(4.1046847659931274e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5195856019854546) - perf/mfu/actor:np.float64(0.05586420816152182) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.32366943359375) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:2 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03285565227270126 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03285565227270126 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:349.79296875 - response_length/max:8192.0 - response_length/min:81.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:349.79296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:295.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016735494136810303 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6763673182576895) - timing_s/agent_loop/generate_sequences/max:np.float64(49.16137577034533) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9194193026196444) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.16137577034533) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.212846806272864 - timing_s/reward:0.05881442688405514 - timing_s/old_log_prob:2.536781370639801 - timing_s/adv:0.1617901399731636 - timing_s/update_actor:10.77561523206532 - timing_s/step:68.83385827951133 - timing_s/stop_profile:0.00013233162462711334 - timing_per_token_ms/gen:0.6165795259056458 - timing_per_token_ms/update_actor:0.06520756444478593 - timing_per_token_ms/adv:0.0009790569495686174 - perf/total_num_tokens:165251 - perf/time_per_step:68.83385827951133 - perf/throughput:300.0903264222289 (TaskRunner pid=2099163) Training Progress: 37%|███▋ | 37/100 [31:16<46:14, 44.04s/it] (TaskRunner pid=2099163) step:38 - global_seqlen/min:19998 - global_seqlen/max:31407 - global_seqlen/minmax_diff:11409 - global_seqlen/balanced_min:24964 - global_seqlen/balanced_max:28950 - global_seqlen/mean:25513.75 - actor/entropy:0.17665337026119232 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29611214995384216 - training/rollout_probs_diff_mean:0.0035523248370736837 - training/rollout_probs_diff_std:0.0110044926404953 - training/rollout_actor_probs_pearson_corr:0.9986032247543335 - rollout_corr/rollout_is_mean:0.999925434589386 - rollout_corr/rollout_is_ratio_fraction_high:1.739039726089686e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.478079452179372e-05 - rollout_corr/rollout_is_max:2.075227975845337 - rollout_corr/rollout_is_min:0.4160393178462982 - rollout_corr/rollout_is_std:0.03343033045530319 - rollout_corr/rollout_is_eff_sample_size:0.9988834226305476 - rollout_corr/rollout_is_seq_mean:0.9999387264251709 - rollout_corr/rollout_is_seq_std:0.0025514268781989813 - rollout_corr/rollout_is_seq_max:1.0072029829025269 - rollout_corr/rollout_is_seq_min:0.9921553730964661 - rollout_corr/rollout_is_seq_max_deviation:0.007844626903533936 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2722632107324898) - rollout_corr/training_log_ppl:np.float64(0.2361144043825334) - rollout_corr/kl:np.float64(0.0008850332726968446) - rollout_corr/k3_kl:np.float64(0.0009130484113342163) - rollout_corr/rollout_ppl:np.float64(1.2710885656997561) - rollout_corr/rollout_log_ppl:np.float64(0.23522936822337215) - rollout_corr/log_ppl_diff:np.float64(0.0008850361591612455) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021792255029140506) - rollout_corr/log_ppl_diff_max:np.float64(0.012408077716827393) - rollout_corr/log_ppl_diff_min:np.float64(-0.010918736457824707) - rollout_corr/ppl_ratio:np.float64(1.000889461953193) - rollout_corr/chi2_token:np.float64(0.001883210614323616) - rollout_corr/chi2_seq:np.float64(1.0299227663781494) - actor/pg_loss:np.float64(0.00017131585627794266) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015263908690030803) - actor/ppo_kl:np.float64(6.009416163976766e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6240686252713203) - perf/mfu/actor:np.float64(0.06947042905322103) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.28501892089844) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:2 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014406857080757618 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014406857080757618 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:449.2421875 - response_length/max:6502.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:449.2421875 - response_length_non_aborted/max:6502.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:348.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.151758134365082e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.093883166089654) - timing_s/agent_loop/generate_sequences/max:np.float64(34.20732720755041) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.161816134750552) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(34.20732720755041) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:6502 - timing_s/gen:39.98209969326854 - timing_s/reward:0.0648152194917202 - timing_s/old_log_prob:2.563793510198593 - timing_s/adv:0.12993765622377396 - timing_s/update_actor:10.490677770227194 - timing_s/step:53.31818316318095 - timing_s/stop_profile:0.00011186115443706512 - timing_per_token_ms/gen:0.3476522937348359 - timing_per_token_ms/update_actor:0.0513971768665288 - timing_per_token_ms/adv:0.0006366060272587034 - perf/total_num_tokens:204110 - perf/time_per_step:53.31818316318095 - perf/throughput:478.5187432571522 (TaskRunner pid=2099163) Training Progress: 38%|███▊ | 38/100 [32:09<48:23, 46.84s/it] (TaskRunner pid=2099163) step:39 - global_seqlen/min:17054 - global_seqlen/max:28720 - global_seqlen/minmax_diff:11666 - global_seqlen/balanced_min:22500 - global_seqlen/balanced_max:22776 - global_seqlen/mean:22545.375 - actor/entropy:0.19491960108280182 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45386987924575806 - training/rollout_probs_diff_mean:0.004056902602314949 - training/rollout_probs_diff_std:0.012101990170776844 - training/rollout_actor_probs_pearson_corr:0.9984006881713867 - rollout_corr/rollout_is_mean:0.9999191164970398 - rollout_corr/rollout_is_ratio_fraction_high:1.0010110599978361e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012012132356176153 - rollout_corr/rollout_is_max:3.0525314807891846 - rollout_corr/rollout_is_min:0.011025836691260338 - rollout_corr/rollout_is_std:0.03583298996090889 - rollout_corr/rollout_is_eff_sample_size:0.9987174054849994 - rollout_corr/rollout_is_seq_mean:0.9999333024024963 - rollout_corr/rollout_is_seq_std:0.002865663729608059 - rollout_corr/rollout_is_seq_max:1.009408712387085 - rollout_corr/rollout_is_seq_min:0.9888485074043274 - rollout_corr/rollout_is_seq_max_deviation:0.011151492595672607 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2850506003014743) - rollout_corr/training_log_ppl:np.float64(0.24625673811533488) - rollout_corr/kl:np.float64(0.001440951160018411) - rollout_corr/k3_kl:np.float64(0.0010581204401205468) - rollout_corr/rollout_ppl:np.float64(1.2831103513017297) - rollout_corr/rollout_log_ppl:np.float64(0.24481578795530368) - rollout_corr/log_ppl_diff:np.float64(0.001440950160031207) - rollout_corr/log_ppl_abs_diff:np.float64(0.002506036587874405) - rollout_corr/log_ppl_diff_max:np.float64(0.015404284000396729) - rollout_corr/log_ppl_diff_min:np.float64(-0.008438989520072937) - rollout_corr/ppl_ratio:np.float64(1.0014470850583166) - rollout_corr/chi2_token:np.float64(0.0012923125177621841) - rollout_corr/chi2_seq:np.float64(0.49474142445251346) - actor/pg_loss:np.float64(8.401612285524607e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014041442402685789) - actor/ppo_kl:np.float64(0.00043948041806451954) - actor/pg_clipfrac_lower:np.float64(1.9405732018640265e-05) - actor/grad_norm:np.float64(0.6177720576524734) - perf/mfu/actor:np.float64(0.06559690994691629) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.28384017944336) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:2 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004189230967313051 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004189230967313051 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:390.23046875 - response_length/max:2201.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:390.23046875 - response_length_non_aborted/max:2201.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.3125 - prompt_length/max:498.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016164779663085938 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0495860055088997) - timing_s/agent_loop/generate_sequences/max:np.float64(13.142637204378843) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6693802871523076) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.142637204378843) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:2201 - timing_s/gen:18.87878078967333 - timing_s/reward:0.06111055053770542 - timing_s/old_log_prob:2.4288064669817686 - timing_s/adv:0.12556837499141693 - timing_s/update_actor:9.746203064918518 - timing_s/step:31.32407439313829 - timing_s/stop_profile:3.988295793533325e-05 - timing_per_token_ms/gen:0.18897867635985674 - timing_per_token_ms/update_actor:0.054036598775350365 - timing_per_token_ms/adv:0.0006961980838166194 - perf/total_num_tokens:180363 - perf/time_per_step:31.32407439313829 - perf/throughput:719.7459282288861 (TaskRunner pid=2099163) Training Progress: 39%|███▉ | 39/100 [32:41<42:53, 42.19s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score of a protein sequence cannot be determined without access to specific computational tools or databases that calculate such scores based on the sequence. The provided sequence is a string of amino acids, and without additional context or computational analysis, it is not possible to determine the folding stability score. Therefore, the question as posed cannot be answered with the given information. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] A (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2099163) step:40 - global_seqlen/min:15913 - global_seqlen/max:27702 - global_seqlen/minmax_diff:11789 - global_seqlen/balanced_min:21440 - global_seqlen/balanced_max:22074 - global_seqlen/mean:21638.125 - actor/entropy:0.2002803236246109 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28941571712493896 - training/rollout_probs_diff_mean:0.00437137670814991 - training/rollout_probs_diff_std:0.012427032925188541 - training/rollout_actor_probs_pearson_corr:0.998335063457489 - rollout_corr/rollout_is_mean:1.000082015991211 - rollout_corr/rollout_is_ratio_fraction_high:2.1701152945752256e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.425288327387534e-05 - rollout_corr/rollout_is_max:2.1118814945220947 - rollout_corr/rollout_is_min:0.3157401978969574 - rollout_corr/rollout_is_std:0.037414900958538055 - rollout_corr/rollout_is_eff_sample_size:0.9986024388513428 - rollout_corr/rollout_is_seq_mean:0.9999911785125732 - rollout_corr/rollout_is_seq_std:0.002646325621753931 - rollout_corr/rollout_is_seq_max:1.0076828002929688 - rollout_corr/rollout_is_seq_min:0.99074786901474 - rollout_corr/rollout_is_seq_max_deviation:0.00925213098526001 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2884388542734087) - rollout_corr/training_log_ppl:np.float64(0.24909779269364662) - rollout_corr/kl:np.float64(0.001254979046022564) - rollout_corr/k3_kl:np.float64(0.0012163868613583873) - rollout_corr/rollout_ppl:np.float64(1.2867698702029884) - rollout_corr/rollout_log_ppl:np.float64(0.2478428122412879) - rollout_corr/log_ppl_diff:np.float64(0.0012549804523587227) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025399263540748507) - rollout_corr/log_ppl_diff_max:np.float64(0.013394579291343689) - rollout_corr/log_ppl_diff_min:np.float64(-0.006626993417739868) - rollout_corr/ppl_ratio:np.float64(1.001261416822672) - rollout_corr/chi2_token:np.float64(0.002296414924785495) - rollout_corr/chi2_seq:np.float64(0.5827651822473854) - actor/pg_loss:np.float64(0.00018049252685159445) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001993459543427889) - actor/ppo_kl:np.float64(0.00036094990030921537) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7396467328071594) - perf/mfu/actor:np.float64(0.06226110744290928) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.27344131469727) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.35375) - val-aux/sciknoweval/reward/std@16:np.float64(0.3395231260295747) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5024000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.31451708561123753) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.20027999999999999) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.26304832865625566) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.34982000000000013) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.33924744416949965) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6401399999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.23702579096468043) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.10339999999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1561668404485443) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.3715800000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.30096314091120385) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7321000000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1335033733578523) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.05524) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07954253083945868) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.38284) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.25173507835666453) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.77508) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05977639622221202) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.02734) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.046689905811973874) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.39382) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.21182267262392962) - val-aux/sciknoweval/score/mean@16:np.float64(0.35375) - val-aux/sciknoweval/score/std@16:np.float64(0.3395231260295747) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5024000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.31451708561123753) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.20027999999999999) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.26304832865625566) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.34982000000000013) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.33924744416949965) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6401399999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.23702579096468043) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.10339999999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1561668404485443) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.3715800000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.30096314091120385) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7321000000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1335033733578523) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.05524) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07954253083945868) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.38284) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.25173507835666453) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.77508) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05977639622221202) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.02734) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.046689905811973874) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.39382) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.21182267262392962) - val-core/sciknoweval/acc/mean@16:np.float64(0.35375) - val-aux/sciknoweval/acc/std@16:np.float64(0.3395231260295747) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5024000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.31451708561123753) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.20027999999999999) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.26304832865625566) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.34982000000000013) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.33924744416949965) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6401399999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.23702579096468043) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.10339999999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1561668404485443) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.3715800000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.30096314091120385) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7321000000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1335033733578523) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.05524) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07954253083945868) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.38284) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.25173507835666453) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.77508) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05977639622221202) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.02734) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.046689905811973874) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.39382) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.21182267262392962) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.94375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.10821442299182672) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9848399999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.05425277298355868) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.89878) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.12778925051495435) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9417399999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.10929119954355648) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9982199999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.013301438756809114) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.84614) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.13137340985952187) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.97116) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.07631025659993602) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0012642784503423287) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.7881) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11433821446278315) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.98946) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.042506343696145) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7379399999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07740453538749054) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99836) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.014501222079305633) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:2 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008019932545721531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008019932545721531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.00390625 - response_length/max:2721.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.00390625 - response_length_non_aborted/max:2721.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:316.1875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.67449426651001e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9381169620901346) - timing_s/agent_loop/generate_sequences/max:np.float64(15.55611569993198) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4685358870920027) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.55611569993198) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:2721 - timing_s/gen:21.971412979066372 - timing_s/reward:0.06056964956223965 - timing_s/old_log_prob:2.4427013341337442 - timing_s/adv:0.12022142671048641 - timing_s/update_actor:9.820722814649343 - timing_s/step:34.50256798416376 - timing_s/testing:96.73482717573643 - timing_s/save_checkpoint:6.927775839343667 - timing_s/stop_profile:3.5040080547332764e-05 - timing_per_token_ms/gen:0.23840250191584697 - timing_per_token_ms/update_actor:0.05673275072730045 - timing_per_token_ms/adv:0.000694500024323309 - perf/total_num_tokens:173105 - perf/time_per_step:34.50256798416376 - perf/throughput:627.1453478457495 (TaskRunner pid=2099163) Training Progress: 40%|████ | 40/100 [34:59<1:10:59, 70.99s/it] (TaskRunner pid=2099163) step:41 - global_seqlen/min:13619 - global_seqlen/max:27244 - global_seqlen/minmax_diff:13625 - global_seqlen/balanced_min:22065 - global_seqlen/balanced_max:22100 - global_seqlen/mean:22093.0 - actor/entropy:0.1972004473209381 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7945776581764221 - training/rollout_probs_diff_mean:0.0044371881522238255 - training/rollout_probs_diff_std:0.013081462122499943 - training/rollout_actor_probs_pearson_corr:0.9981732368469238 - rollout_corr/rollout_is_mean:1.0000603199005127 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.645401299465448e-05 - rollout_corr/rollout_is_max:1.7906994819641113 - rollout_corr/rollout_is_min:0.14579379558563232 - rollout_corr/rollout_is_std:0.037738487124443054 - rollout_corr/rollout_is_eff_sample_size:0.9985780698008809 - rollout_corr/rollout_is_seq_mean:1.0000441074371338 - rollout_corr/rollout_is_seq_std:0.0030684443190693855 - rollout_corr/rollout_is_seq_max:1.007584571838379 - rollout_corr/rollout_is_seq_min:0.9881037473678589 - rollout_corr/rollout_is_seq_max_deviation:0.011896252632141113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2687957105226815) - rollout_corr/training_log_ppl:np.float64(0.23422167608805466) - rollout_corr/kl:np.float64(0.001997436562771071) - rollout_corr/k3_kl:np.float64(0.001895930330306328) - rollout_corr/rollout_ppl:np.float64(1.2661805939860642) - rollout_corr/rollout_log_ppl:np.float64(0.2322242384107085) - rollout_corr/log_ppl_diff:np.float64(0.00199743767734617) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037411078810691833) - rollout_corr/log_ppl_diff_max:np.float64(0.03718364238739014) - rollout_corr/log_ppl_diff_min:np.float64(-0.030967384576797485) - rollout_corr/ppl_ratio:np.float64(1.002018980216235) - rollout_corr/chi2_token:np.float64(0.0035980751272290945) - rollout_corr/chi2_seq:np.float64(2.2225923924706876) - actor/pg_loss:np.float64(0.0005501225823536515) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0022654881254311476) - actor/ppo_kl:np.float64(0.001161328320471) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7245654910802841) - perf/mfu/actor:np.float64(0.06373080366245223) - perf/max_memory_allocated_gb:np.float64(123.66978645324707) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.36378860473633) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:2 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027744551189243793 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027744551189243793 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.6875 - response_length/max:1485.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.6875 - response_length_non_aborted/max:1485.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:337.71875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8796494007110596e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.95299425534904) - timing_s/agent_loop/generate_sequences/max:np.float64(9.864370591938496) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4211922848553513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.864370591938496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:1485 - timing_s/gen:15.542456863448024 - timing_s/reward:0.05860104784369469 - timing_s/old_log_prob:2.535149410367012 - timing_s/adv:0.1277296431362629 - timing_s/update_actor:9.861975859850645 - timing_s/step:28.19834844954312 - timing_s/stop_profile:0.00011692754924297333 - timing_per_token_ms/gen:0.17214310720636214 - timing_per_token_ms/update_actor:0.05579808004713396 - timing_per_token_ms/adv:0.0007226816363568941 - perf/total_num_tokens:176744 - perf/time_per_step:28.19834844954312 - perf/throughput:783.4856016313239 (TaskRunner pid=2099163) Training Progress: 41%|████ | 41/100 [35:27<57:11, 58.17s/it] (TaskRunner pid=2099163) step:42 - global_seqlen/min:11327 - global_seqlen/max:31713 - global_seqlen/minmax_diff:20386 - global_seqlen/balanced_min:16962 - global_seqlen/balanced_max:23405 - global_seqlen/mean:17874.375 - actor/entropy:0.1793491095304489 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2369065284729004 - training/rollout_probs_diff_mean:0.003962434828281403 - training/rollout_probs_diff_std:0.011616604402661324 - training/rollout_actor_probs_pearson_corr:0.9984515309333801 - rollout_corr/rollout_is_mean:1.0001260042190552 - rollout_corr/rollout_is_ratio_fraction_high:1.2704059372481424e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.3973875045776367 - rollout_corr/rollout_is_min:0.5620211958885193 - rollout_corr/rollout_is_std:0.03498118743300438 - rollout_corr/rollout_is_eff_sample_size:0.9987781689679683 - rollout_corr/rollout_is_seq_mean:1.0004432201385498 - rollout_corr/rollout_is_seq_std:0.003099439898505807 - rollout_corr/rollout_is_seq_max:1.010513424873352 - rollout_corr/rollout_is_seq_min:0.9920833706855774 - rollout_corr/rollout_is_seq_max_deviation:0.01051342487335205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2783533367328346) - rollout_corr/training_log_ppl:np.float64(0.24186358234874206) - rollout_corr/kl:np.float64(0.0008186687481881449) - rollout_corr/k3_kl:np.float64(0.0014607259122385585) - rollout_corr/rollout_ppl:np.float64(1.2772352620959282) - rollout_corr/rollout_log_ppl:np.float64(0.24104491147591034) - rollout_corr/log_ppl_diff:np.float64(0.0008186708728317171) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031197732605505735) - rollout_corr/log_ppl_diff_max:np.float64(0.023270994424819946) - rollout_corr/log_ppl_diff_min:np.float64(-0.01981446146965027) - rollout_corr/ppl_ratio:np.float64(1.0008287110831589) - rollout_corr/chi2_token:np.float64(0.004162837052717805) - rollout_corr/chi2_seq:np.float64(0.8917187473271042) - actor/pg_loss:np.float64(0.000158449518494308) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011291635195220806) - actor/ppo_kl:np.float64(0.0004214342557844475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5698883533477783) - perf/mfu/actor:np.float64(0.0494323141668247) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.41558456420898) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:2 - critic/score/mean:0.328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03640824556350708 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03640824556350708 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:307.48046875 - response_length/max:8192.0 - response_length/min:79.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:307.48046875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:251.09375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000103706493973732 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9703596197068691) - timing_s/agent_loop/generate_sequences/max:np.float64(41.32044994644821) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.750317184647429) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(41.32044994644821) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:46.890859078615904 - timing_s/reward:0.05139863304793835 - timing_s/old_log_prob:2.586537519469857 - timing_s/adv:0.1239897795021534 - timing_s/update_actor:10.514672089368105 - timing_s/step:60.26146995835006 - timing_s/stop_profile:0.0001133400946855545 - timing_per_token_ms/gen:0.5957042378023999 - timing_per_token_ms/update_actor:0.07353174649021368 - timing_per_token_ms/adv:0.0008670917130120172 - perf/total_num_tokens:142995 - perf/time_per_step:60.26146995835006 - perf/throughput:296.6136573228954 (TaskRunner pid=2099163) Training Progress: 42%|████▏ | 42/100 [36:27<56:51, 58.81s/it] (TaskRunner pid=2099163) step:43 - global_seqlen/min:14105 - global_seqlen/max:31969 - global_seqlen/minmax_diff:17864 - global_seqlen/balanced_min:21216 - global_seqlen/balanced_max:21234 - global_seqlen/mean:21226.625 - actor/entropy:0.19266550242900848 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44984614849090576 - training/rollout_probs_diff_mean:0.004387952387332916 - training/rollout_probs_diff_std:0.01266084611415863 - training/rollout_actor_probs_pearson_corr:0.9982303977012634 - rollout_corr/rollout_is_mean:0.9999305605888367 - rollout_corr/rollout_is_ratio_fraction_high:3.408587508602068e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.544783223536797e-05 - rollout_corr/rollout_is_max:2.5337352752685547 - rollout_corr/rollout_is_min:0.32041898369789124 - rollout_corr/rollout_is_std:0.038009174168109894 - rollout_corr/rollout_is_eff_sample_size:0.9985570301628549 - rollout_corr/rollout_is_seq_mean:0.9998732805252075 - rollout_corr/rollout_is_seq_std:0.002512736711651087 - rollout_corr/rollout_is_seq_max:1.008451223373413 - rollout_corr/rollout_is_seq_min:0.9910997748374939 - rollout_corr/rollout_is_seq_max_deviation:0.008900225162506104 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2575704725459218) - rollout_corr/training_log_ppl:np.float64(0.22557678240991663) - rollout_corr/kl:np.float64(0.0013313335243312352) - rollout_corr/k3_kl:np.float64(0.0010828630935861838) - rollout_corr/rollout_ppl:np.float64(1.2558402898721397) - rollout_corr/rollout_log_ppl:np.float64(0.22424544939713087) - rollout_corr/log_ppl_diff:np.float64(0.0013313330127857625) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023614601814188063) - rollout_corr/log_ppl_diff_max:np.float64(0.011232882738113403) - rollout_corr/log_ppl_diff_min:np.float64(-0.007203608751296997) - rollout_corr/ppl_ratio:np.float64(1.0013366700150073) - rollout_corr/chi2_token:np.float64(0.0016005998477339745) - rollout_corr/chi2_seq:np.float64(0.9111351130995899) - actor/pg_loss:np.float64(-0.00032677757553756237) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019711919426299573) - actor/ppo_kl:np.float64(0.00035553486237560605) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6708932965993881) - perf/mfu/actor:np.float64(0.06152323141502927) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.48104095458984) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:3 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011432970641180873 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011432970641180873 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.80078125 - response_length/max:1356.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.80078125 - response_length_non_aborted/max:1356.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:319.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002473127096891403 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0578165259212255) - timing_s/agent_loop/generate_sequences/max:np.float64(9.275035005062819) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.464938813202025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.275035005062819) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1356 - timing_s/gen:15.193364929407835 - timing_s/reward:0.05014399252831936 - timing_s/old_log_prob:2.6355569884181023 - timing_s/adv:0.1556774377822876 - timing_s/update_actor:9.845597103238106 - timing_s/step:27.98137691617012 - timing_s/stop_profile:4.378892481327057e-05 - timing_per_token_ms/gen:0.17262637257459507 - timing_per_token_ms/update_actor:0.05797905403731225 - timing_per_token_ms/adv:0.0009167580678881334 - perf/total_num_tokens:169813 - perf/time_per_step:27.98137691617012 - perf/throughput:758.5983014200197 (TaskRunner pid=2099163) Training Progress: 43%|████▎ | 43/100 [36:56<47:12, 49.70s/it] (TaskRunner pid=2099163) step:44 - global_seqlen/min:14289 - global_seqlen/max:31868 - global_seqlen/minmax_diff:17579 - global_seqlen/balanced_min:23024 - global_seqlen/balanced_max:27562 - global_seqlen/mean:23664.625 - actor/entropy:0.16483263671398163 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5307222604751587 - training/rollout_probs_diff_mean:0.0037482562474906445 - training/rollout_probs_diff_std:0.011952490545809269 - training/rollout_actor_probs_pearson_corr:0.9982149600982666 - rollout_corr/rollout_is_mean:0.9997656941413879 - rollout_corr/rollout_is_ratio_fraction_high:1.877493559732102e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.754987119464204e-05 - rollout_corr/rollout_is_max:2.257253646850586 - rollout_corr/rollout_is_min:0.2560407221317291 - rollout_corr/rollout_is_std:0.03545422852039337 - rollout_corr/rollout_is_eff_sample_size:0.9987439217121525 - rollout_corr/rollout_is_seq_mean:0.9996024966239929 - rollout_corr/rollout_is_seq_std:0.002828868106007576 - rollout_corr/rollout_is_seq_max:1.0087915658950806 - rollout_corr/rollout_is_seq_min:0.9925878047943115 - rollout_corr/rollout_is_seq_max_deviation:0.008791565895080566 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2643110915087163) - rollout_corr/training_log_ppl:np.float64(0.23016513518814463) - rollout_corr/kl:np.float64(0.0011426463696224687) - rollout_corr/k3_kl:np.float64(0.0011375640360711259) - rollout_corr/rollout_ppl:np.float64(1.2628552531823516) - rollout_corr/rollout_log_ppl:np.float64(0.22902248779064394) - rollout_corr/log_ppl_diff:np.float64(0.0011426473975006957) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027303947317705024) - rollout_corr/log_ppl_diff_max:np.float64(0.012277275323867798) - rollout_corr/log_ppl_diff_min:np.float64(-0.010143131017684937) - rollout_corr/ppl_ratio:np.float64(1.0011494110804051) - rollout_corr/chi2_token:np.float64(0.0023959616664797068) - rollout_corr/chi2_seq:np.float64(1.1097935638390481) - actor/pg_loss:np.float64(0.0004005907103419304) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017527463992337289) - actor/ppo_kl:np.float64(-0.0001501098105372023) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6244964599609375) - perf/mfu/actor:np.float64(0.06486412678751474) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.46553421020508) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:3 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03883712738752365 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.03883712738752365 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:416.11328125 - response_length/max:7121.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:416.11328125 - response_length_non_aborted/max:7121.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:323.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.310292959213257e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.854373425245285) - timing_s/agent_loop/generate_sequences/max:np.float64(35.90070942789316) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.641139167644724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(35.90070942789316) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:7121 - timing_s/gen:41.78276102989912 - timing_s/reward:0.05910661816596985 - timing_s/old_log_prob:2.6271430384367704 - timing_s/adv:0.12738168239593506 - timing_s/update_actor:10.614613642916083 - timing_s/step:55.30339947156608 - timing_s/stop_profile:0.00013011880218982697 - timing_per_token_ms/gen:0.39223432086270005 - timing_per_token_ms/update_actor:0.0560679370733536 - timing_per_token_ms/adv:0.0006728486210743624 - perf/total_num_tokens:189317 - perf/time_per_step:55.30339947156608 - perf/throughput:427.9054312414742 (TaskRunner pid=2099163) Training Progress: 44%|████▍ | 44/100 [37:51<47:58, 51.40s/it] (TaskRunner pid=2099163) step:45 - global_seqlen/min:10341 - global_seqlen/max:43510 - global_seqlen/minmax_diff:33169 - global_seqlen/balanced_min:21899 - global_seqlen/balanced_max:27874 - global_seqlen/mean:22856.25 - actor/entropy:0.1471649408340454 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3509632647037506 - training/rollout_probs_diff_mean:0.0035214414820075035 - training/rollout_probs_diff_std:0.01150407176464796 - training/rollout_actor_probs_pearson_corr:0.9982120394706726 - rollout_corr/rollout_is_mean:0.9999986290931702 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.89342778059654e-05 - rollout_corr/rollout_is_max:1.867427945137024 - rollout_corr/rollout_is_min:0.030966857448220253 - rollout_corr/rollout_is_std:0.03498629853129387 - rollout_corr/rollout_is_eff_sample_size:0.9987774554595162 - rollout_corr/rollout_is_seq_mean:0.9998350739479065 - rollout_corr/rollout_is_seq_std:0.002756875241175294 - rollout_corr/rollout_is_seq_max:1.0082145929336548 - rollout_corr/rollout_is_seq_min:0.990932822227478 - rollout_corr/rollout_is_seq_max_deviation:0.009067177772521973 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2405082695186138) - rollout_corr/training_log_ppl:np.float64(0.21127635941229528) - rollout_corr/kl:np.float64(0.0012804620336837047) - rollout_corr/k3_kl:np.float64(0.0011797188574576012) - rollout_corr/rollout_ppl:np.float64(1.2388938385993242) - rollout_corr/rollout_log_ppl:np.float64(0.20999589761777315) - rollout_corr/log_ppl_diff:np.float64(0.0012804617945221253) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025750486165634356) - rollout_corr/log_ppl_diff_max:np.float64(0.01426708698272705) - rollout_corr/log_ppl_diff_min:np.float64(-0.016405731439590454) - rollout_corr/ppl_ratio:np.float64(1.001287407707423) - rollout_corr/chi2_token:np.float64(0.0022307620383799076) - rollout_corr/chi2_seq:np.float64(0.5359919595066458) - actor/pg_loss:np.float64(8.05326271802187e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001362902614914674) - actor/ppo_kl:np.float64(0.0002342196375835215) - actor/pg_clipfrac_lower:np.float64(7.356402875302592e-06) - actor/grad_norm:np.float64(0.6136828362941742) - perf/mfu/actor:np.float64(0.062255809404529946) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.4916729927063) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:3 - critic/score/mean:0.31640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.31640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0037901541218161583 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0037901541218161583 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:396.6640625 - response_length/max:8160.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:396.6640625 - response_length_non_aborted/max:8160.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.59375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.038004696369171e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9433151036500931) - timing_s/agent_loop/generate_sequences/max:np.float64(41.46591006591916) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.561986893051653) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(41.46591006591916) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8160 - timing_s/gen:47.15192976593971 - timing_s/reward:0.059990689158439636 - timing_s/old_log_prob:2.561908131465316 - timing_s/adv:0.12769671715795994 - timing_s/update_actor:10.679077072069049 - timing_s/step:60.668493922799826 - timing_s/stop_profile:0.00013370439410209656 - timing_per_token_ms/gen:0.4643405921054469 - timing_per_token_ms/update_actor:0.0584034841239762 - timing_per_token_ms/adv:0.0006983687019850147 - perf/total_num_tokens:182850 - perf/time_per_step:60.668493922799826 - perf/throughput:376.7400263649926 (TaskRunner pid=2099163) Training Progress: 45%|████▌ | 45/100 [38:52<49:40, 54.19s/it] (TaskRunner pid=2099163) step:46 - global_seqlen/min:11629 - global_seqlen/max:28235 - global_seqlen/minmax_diff:16606 - global_seqlen/balanced_min:17711 - global_seqlen/balanced_max:17919 - global_seqlen/mean:17760.75 - actor/entropy:0.20734380185604095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5599408149719238 - training/rollout_probs_diff_mean:0.004730684217065573 - training/rollout_probs_diff_std:0.013396089896559715 - training/rollout_actor_probs_pearson_corr:0.9980821013450623 - rollout_corr/rollout_is_mean:1.0000985860824585 - rollout_corr/rollout_is_ratio_fraction_high:2.7359032173990272e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3679516086995136e-05 - rollout_corr/rollout_is_max:5.697536468505859 - rollout_corr/rollout_is_min:0.4908820390701294 - rollout_corr/rollout_is_std:0.04028098285198212 - rollout_corr/rollout_is_eff_sample_size:0.9983800707884429 - rollout_corr/rollout_is_seq_mean:1.000237226486206 - rollout_corr/rollout_is_seq_std:0.00363980233669281 - rollout_corr/rollout_is_seq_max:1.021539568901062 - rollout_corr/rollout_is_seq_min:0.9916234016418457 - rollout_corr/rollout_is_seq_max_deviation:0.02153956890106201 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2810124992392957) - rollout_corr/training_log_ppl:np.float64(0.24381673078460153) - rollout_corr/kl:np.float64(0.0006558897876516312) - rollout_corr/k3_kl:np.float64(0.0010635649665289293) - rollout_corr/rollout_ppl:np.float64(1.2801448996178806) - rollout_corr/rollout_log_ppl:np.float64(0.24316083974554203) - rollout_corr/log_ppl_diff:np.float64(0.0006558910390594974) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025359629216836765) - rollout_corr/log_ppl_diff_max:np.float64(0.009192198514938354) - rollout_corr/log_ppl_diff_min:np.float64(-0.009751826524734497) - rollout_corr/ppl_ratio:np.float64(1.0006613184232265) - rollout_corr/chi2_token:np.float64(0.003016318893060088) - rollout_corr/chi2_seq:np.float64(0.5328007256612182) - actor/pg_loss:np.float64(-6.973580457270145e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015097170539775107) - actor/ppo_kl:np.float64(-0.00010064278021815198) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.563657782971859) - perf/mfu/actor:np.float64(0.051583663196279474) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.5073127746582) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:3 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023010656237602234 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023010656237602234 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:285.5546875 - response_length/max:1823.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:285.5546875 - response_length_non_aborted/max:1823.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.46875 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.830668568611145e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9158236924558878) - timing_s/agent_loop/generate_sequences/max:np.float64(10.958448996767402) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.764639292836364) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.958448996767402) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1823 - timing_s/gen:16.863597629591823 - timing_s/reward:0.053841013461351395 - timing_s/old_log_prob:2.380986912176013 - timing_s/adv:0.14236685261130333 - timing_s/update_actor:9.776626717299223 - timing_s/step:29.306406520307064 - timing_s/stop_profile:2.9392540454864502e-05 - timing_per_token_ms/gen:0.23068585852085885 - timing_per_token_ms/update_actor:0.06880781158804684 - timing_per_token_ms/adv:0.0010019766381719757 - perf/total_num_tokens:142086 - perf/time_per_step:29.306406520307064 - perf/throughput:606.036430556342 (TaskRunner pid=2099163) Training Progress: 46%|████▌ | 46/100 [39:21<42:03, 46.73s/it] (TaskRunner pid=2099163) step:47 - global_seqlen/min:18567 - global_seqlen/max:38675 - global_seqlen/minmax_diff:20108 - global_seqlen/balanced_min:25878 - global_seqlen/balanced_max:26514 - global_seqlen/mean:25983.75 - actor/entropy:0.14283302426338196 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7308436632156372 - training/rollout_probs_diff_mean:0.003343928838148713 - training/rollout_probs_diff_std:0.011697083711624146 - training/rollout_actor_probs_pearson_corr:0.9980596899986267 - rollout_corr/rollout_is_mean:1.0000288486480713 - rollout_corr/rollout_is_ratio_fraction_high:2.6085595891345292e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001130375821958296 - rollout_corr/rollout_is_max:2.327317714691162 - rollout_corr/rollout_is_min:0.23235759139060974 - rollout_corr/rollout_is_std:0.03357976675033569 - rollout_corr/rollout_is_eff_sample_size:0.9988737883222161 - rollout_corr/rollout_is_seq_mean:0.9999798536300659 - rollout_corr/rollout_is_seq_std:0.0024428016040474176 - rollout_corr/rollout_is_seq_max:1.0096776485443115 - rollout_corr/rollout_is_seq_min:0.9921210408210754 - rollout_corr/rollout_is_seq_max_deviation:0.009677648544311523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2100329943932593) - rollout_corr/training_log_ppl:np.float64(0.18659601080435095) - rollout_corr/kl:np.float64(0.0011015117767527727) - rollout_corr/k3_kl:np.float64(0.0009561012871870389) - rollout_corr/rollout_ppl:np.float64(1.208669331856072) - rollout_corr/rollout_log_ppl:np.float64(0.18549449724378064) - rollout_corr/log_ppl_diff:np.float64(0.0011015135605703108) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020468592338147573) - rollout_corr/log_ppl_diff_max:np.float64(0.013909026980400085) - rollout_corr/log_ppl_diff_min:np.float64(-0.007575169205665588) - rollout_corr/ppl_ratio:np.float64(1.001105838920921) - rollout_corr/chi2_token:np.float64(0.001610573148354888) - rollout_corr/chi2_seq:np.float64(0.26548866275697947) - actor/pg_loss:np.float64(4.338508006185293e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011870542070937518) - actor/ppo_kl:np.float64(0.0002975921167021056) - actor/pg_clipfrac_lower:np.float64(9.322792720922735e-06) - actor/grad_norm:np.float64(0.5477600991725922) - perf/mfu/actor:np.float64(0.07517353223146761) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.55639266967773) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:3 - critic/score/mean:0.39453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.39453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002075978700304404 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002075978700304404 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:449.2421875 - response_length/max:2732.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:449.2421875 - response_length_non_aborted/max:2732.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.532266080379486e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9549492709338665) - timing_s/agent_loop/generate_sequences/max:np.float64(15.872619692236185) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.19089348961279) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.872619692236185) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:2732 - timing_s/gen:21.63072121143341 - timing_s/reward:0.06235244311392307 - timing_s/old_log_prob:2.547062236815691 - timing_s/adv:0.1267538219690323 - timing_s/update_actor:9.819873118773103 - timing_s/step:34.27366493828595 - timing_s/stop_profile:0.00012672320008277893 - timing_per_token_ms/gen:0.18808341487777516 - timing_per_token_ms/update_actor:0.04724045373922693 - timing_per_token_ms/adv:0.0006097744839035565 - perf/total_num_tokens:207870 - perf/time_per_step:34.27366493828595 - perf/throughput:758.1258102040449 (TaskRunner pid=2099163) Training Progress: 47%|████▋ | 47/100 [39:55<37:59, 43.01s/it] (TaskRunner pid=2099163) step:48 - global_seqlen/min:16381 - global_seqlen/max:32235 - global_seqlen/minmax_diff:15854 - global_seqlen/balanced_min:21840 - global_seqlen/balanced_max:27905 - global_seqlen/mean:22746.0 - actor/entropy:0.16609838604927063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5618129372596741 - training/rollout_probs_diff_mean:0.0038480223156511784 - training/rollout_probs_diff_std:0.012091293931007385 - training/rollout_actor_probs_pearson_corr:0.9981785416603088 - rollout_corr/rollout_is_mean:1.0000433921813965 - rollout_corr/rollout_is_ratio_fraction_high:1.9581733795348555e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.8745201386045665e-05 - rollout_corr/rollout_is_max:2.1106479167938232 - rollout_corr/rollout_is_min:0.0005965463351458311 - rollout_corr/rollout_is_std:0.03607833385467529 - rollout_corr/rollout_is_eff_sample_size:0.9987001647473916 - rollout_corr/rollout_is_seq_mean:1.0000697374343872 - rollout_corr/rollout_is_seq_std:0.0026138562243431807 - rollout_corr/rollout_is_seq_max:1.010244369506836 - rollout_corr/rollout_is_seq_min:0.99269038438797 - rollout_corr/rollout_is_seq_max_deviation:0.010244369506835938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.260544263292104) - rollout_corr/training_log_ppl:np.float64(0.2275718234195665) - rollout_corr/kl:np.float64(0.000840849202081273) - rollout_corr/k3_kl:np.float64(0.0011277189190366244) - rollout_corr/rollout_ppl:np.float64(1.2594485664740205) - rollout_corr/rollout_log_ppl:np.float64(0.2267309735470917) - rollout_corr/log_ppl_diff:np.float64(0.0008408498724747915) - rollout_corr/log_ppl_abs_diff:np.float64(0.002264020313305082) - rollout_corr/log_ppl_diff_max:np.float64(0.02813515067100525) - rollout_corr/log_ppl_diff_min:np.float64(-0.008843004703521729) - rollout_corr/ppl_ratio:np.float64(1.0008466544095427) - rollout_corr/chi2_token:np.float64(0.0026712415274232626) - rollout_corr/chi2_seq:np.float64(0.9579408683348447) - actor/pg_loss:np.float64(1.6385456547141075e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00142442833248424) - actor/ppo_kl:np.float64(-7.805178177022754e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5536335557699203) - perf/mfu/actor:np.float64(0.06172036956944962) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.46680450439453) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:3 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0621769018471241 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0621769018471241 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:398.96875 - response_length/max:8192.0 - response_length/min:81.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:398.96875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:311.84375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.11615788936615e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5817424338310957) - timing_s/agent_loop/generate_sequences/max:np.float64(47.18641279824078) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3139364701564773) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(47.18641279824078) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.73312545567751 - timing_s/reward:0.06915519759058952 - timing_s/old_log_prob:2.6080319173634052 - timing_s/adv:0.13032598234713078 - timing_s/update_actor:10.7602223996073 - timing_s/step:66.35072775743902 - timing_s/stop_profile:0.0001078825443983078 - timing_per_token_ms/gen:0.5163030220067117 - timing_per_token_ms/update_actor:0.05913249801947211 - timing_per_token_ms/adv:0.0007162027518417017 - perf/total_num_tokens:181968 - perf/time_per_step:66.35072775743902 - perf/throughput:342.81462719072886 (TaskRunner pid=2099163) Training Progress: 48%|████▊ | 48/100 [41:02<43:21, 50.03s/it] (TaskRunner pid=2099163) step:49 - global_seqlen/min:13330 - global_seqlen/max:23649 - global_seqlen/minmax_diff:10319 - global_seqlen/balanced_min:19645 - global_seqlen/balanced_max:19656 - global_seqlen/mean:19652.75 - actor/entropy:0.21312448382377625 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29631492495536804 - training/rollout_probs_diff_mean:0.004599376115947962 - training/rollout_probs_diff_std:0.012834436260163784 - training/rollout_actor_probs_pearson_corr:0.9982872009277344 - rollout_corr/rollout_is_mean:0.9999762773513794 - rollout_corr/rollout_is_ratio_fraction_high:1.2280485861992929e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.684145849547349e-05 - rollout_corr/rollout_is_max:2.1531147956848145 - rollout_corr/rollout_is_min:0.16506630182266235 - rollout_corr/rollout_is_std:0.03860819712281227 - rollout_corr/rollout_is_eff_sample_size:0.9985116256038487 - rollout_corr/rollout_is_seq_mean:0.999916136264801 - rollout_corr/rollout_is_seq_std:0.002757440321147442 - rollout_corr/rollout_is_seq_max:1.011631965637207 - rollout_corr/rollout_is_seq_min:0.9919342398643494 - rollout_corr/rollout_is_seq_max_deviation:0.011631965637207031 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.290575869847089) - rollout_corr/training_log_ppl:np.float64(0.2508397557976423) - rollout_corr/kl:np.float64(0.0012681070296984842) - rollout_corr/k3_kl:np.float64(0.0010787070739013416) - rollout_corr/rollout_ppl:np.float64(1.2888763579539955) - rollout_corr/rollout_log_ppl:np.float64(0.24957164909574203) - rollout_corr/log_ppl_diff:np.float64(0.0012681067019002512) - rollout_corr/log_ppl_abs_diff:np.float64(0.002345262750168331) - rollout_corr/log_ppl_diff_max:np.float64(0.009302854537963867) - rollout_corr/log_ppl_diff_min:np.float64(-0.009885609149932861) - rollout_corr/ppl_ratio:np.float64(1.0012729486916214) - rollout_corr/chi2_token:np.float64(0.0018082375172525644) - rollout_corr/chi2_seq:np.float64(0.36839555972255766) - actor/pg_loss:np.float64(4.909001290798187e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016076876192983036) - actor/ppo_kl:np.float64(0.0002880832484706275) - actor/pg_clipfrac_lower:np.float64(4.828491910302546e-06) - actor/grad_norm:np.float64(0.629483163356781) - perf/mfu/actor:np.float64(0.05734176798977645) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.61180114746094) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:3 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0032481886446475983 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0032481886446475983 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:318.0859375 - response_length/max:1276.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:318.0859375 - response_length_non_aborted/max:1276.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.732825517654419e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8680097497999668) - timing_s/agent_loop/generate_sequences/max:np.float64(8.407540844753385) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0850135255313944) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.407540844753385) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1276 - timing_s/gen:14.08630795404315 - timing_s/reward:0.05711162090301514 - timing_s/old_log_prob:2.3311667516827583 - timing_s/adv:0.1224974524229765 - timing_s/update_actor:9.94140362367034 - timing_s/step:26.626789750531316 - timing_s/stop_profile:0.00010458938777446747 - timing_per_token_ms/gen:0.17298671194944307 - timing_per_token_ms/update_actor:0.06323163185604012 - timing_per_token_ms/adv:0.0007791368410462689 - perf/total_num_tokens:157222 - perf/time_per_step:26.626789750531316 - perf/throughput:738.0818410378535 (TaskRunner pid=2099163) Training Progress: 49%|████▉ | 49/100 [41:29<36:34, 43.02s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score of a protein sequence typically requires computational analysis using tools like I-TASSER, ROSETTA, or other protein structure prediction and stability analysis software. These tools calculate scores based on the predicted 3D structure of the protein, taking into account factors such as secondary structure, solvent accessibility, and other structural properties. Without access to specific computational tools or databases that provide folding stability scores for the given sequence, it is not possible to determine the exact folding stability score for the sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'. Therefore, the score cannot be determined from the information provided. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2099163) step:50 - global_seqlen/min:12100 - global_seqlen/max:26431 - global_seqlen/minmax_diff:14331 - global_seqlen/balanced_min:22098 - global_seqlen/balanced_max:22581 - global_seqlen/mean:22179.5 - actor/entropy:0.1872798055410385 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25002074241638184 - training/rollout_probs_diff_mean:0.0043448833748698235 - training/rollout_probs_diff_std:0.012745512649416924 - training/rollout_actor_probs_pearson_corr:0.9981958270072937 - rollout_corr/rollout_is_mean:0.9999893307685852 - rollout_corr/rollout_is_ratio_fraction_high:1.082672861230094e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.248018765589222e-05 - rollout_corr/rollout_is_max:2.2042269706726074 - rollout_corr/rollout_is_min:0.3166545629501343 - rollout_corr/rollout_is_std:0.03807654604315758 - rollout_corr/rollout_is_eff_sample_size:0.9985522755652161 - rollout_corr/rollout_is_seq_mean:0.9999541640281677 - rollout_corr/rollout_is_seq_std:0.0029842411167919636 - rollout_corr/rollout_is_seq_max:1.0160789489746094 - rollout_corr/rollout_is_seq_min:0.986756443977356 - rollout_corr/rollout_is_seq_max_deviation:0.016078948974609375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.264865230768919) - rollout_corr/training_log_ppl:np.float64(0.2308657732064603) - rollout_corr/kl:np.float64(0.001112062272518699) - rollout_corr/k3_kl:np.float64(0.0010981027096477192) - rollout_corr/rollout_ppl:np.float64(1.2634586868807673) - rollout_corr/rollout_log_ppl:np.float64(0.22975370922358707) - rollout_corr/log_ppl_diff:np.float64(0.0011120639828732237) - rollout_corr/log_ppl_abs_diff:np.float64(0.002492617248208262) - rollout_corr/log_ppl_diff_max:np.float64(0.01720789074897766) - rollout_corr/log_ppl_diff_min:np.float64(-0.013499855995178223) - rollout_corr/ppl_ratio:np.float64(1.001118621090427) - rollout_corr/chi2_token:np.float64(0.002215159824118018) - rollout_corr/chi2_seq:np.float64(0.9738334778230637) - actor/pg_loss:np.float64(0.00040720601100474596) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00143988847764831) - actor/ppo_kl:np.float64(0.00017129167446228166) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5727251768112183) - perf/mfu/actor:np.float64(0.06430968389502065) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.6324348449707) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.41875) - val-aux/sciknoweval/reward/std@16:np.float64(0.22869494240339183) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.51456) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.18468184508936822) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.31976) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20223553757769963) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.41681999999999997) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.22791020438572854) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.58694) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1261578566460394) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.23558) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15226752550720532) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4398) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18398851891396895) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.63446) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07544190394601284) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.17675999999999997) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09854951447854242) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.45147999999999994) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13395224221738825) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.66118) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03956012550827376) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.14170000000000002) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04890733027547041) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.45902) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09216818857180725) - val-aux/sciknoweval/score/mean@16:np.float64(0.41875) - val-aux/sciknoweval/score/std@16:np.float64(0.22869494240339183) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.51456) - val-aux/sciknoweval/score/best@2/std:np.float64(0.18468184508936822) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.31976) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.20223553757769963) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.41681999999999997) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.22791020438572854) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.58694) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1261578566460394) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.23558) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15226752550720532) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.4398) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18398851891396895) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.63446) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07544190394601284) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.17675999999999997) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09854951447854242) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.45147999999999994) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13395224221738825) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.66118) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03956012550827376) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.14170000000000002) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04890733027547041) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.45902) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09216818857180725) - val-core/sciknoweval/acc/mean@16:np.float64(0.41875) - val-aux/sciknoweval/acc/std@16:np.float64(0.22869494240339183) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.51456) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.18468184508936822) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.31976) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.20223553757769963) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.41681999999999997) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.22791020438572854) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.58694) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1261578566460394) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.23558) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15226752550720532) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.4398) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18398851891396895) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.63446) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07544190394601284) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.17675999999999997) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09854951447854242) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.45147999999999994) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13395224221738825) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.66118) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03956012550827376) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.14170000000000002) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04890733027547041) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.45902) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09216818857180725) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.05008237914987659) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99824) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.015941849190385885) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.96934) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.06589422003315963) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.98418) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04953093796525588) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.94512) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08018857855516771) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99496) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.02826029051906959) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9103999999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08613360783081132) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9989199999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.012363736848705086) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8677600000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07445805556218542) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0008935323161475471) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:3 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007015720475465059 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007015720475465059 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.796875 - response_length/max:2173.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.796875 - response_length_non_aborted/max:2173.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.383952081203461e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9896603114902973) - timing_s/agent_loop/generate_sequences/max:np.float64(12.962509341537952) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5228450542199425) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.962509341537952) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:2173 - timing_s/gen:18.663984855636954 - timing_s/reward:0.07062600553035736 - timing_s/old_log_prob:2.381566744297743 - timing_s/adv:0.1254611872136593 - timing_s/update_actor:9.75345753878355 - timing_s/step:31.04253143630922 - timing_s/testing:77.66123559139669 - timing_s/save_checkpoint:6.67768957093358 - timing_s/stop_profile:8.756667375564575e-05 - timing_per_token_ms/gen:0.20206990662635826 - timing_per_token_ms/update_actor:0.05496887632038341 - timing_per_token_ms/adv:0.0007070785365633766 - perf/total_num_tokens:177436 - perf/time_per_step:31.04253143630922 - perf/throughput:714.4874781074561 (TaskRunner pid=2099163) Training Progress: 50%|█████ | 50/100 [43:24<53:57, 64.75s/it] (TaskRunner pid=2099163) step:51 - global_seqlen/min:16904 - global_seqlen/max:25515 - global_seqlen/minmax_diff:8611 - global_seqlen/balanced_min:21289 - global_seqlen/balanced_max:21389 - global_seqlen/mean:21315.375 - actor/entropy:0.19681613147258759 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6418529748916626 - training/rollout_probs_diff_mean:0.004624536260962486 - training/rollout_probs_diff_std:0.013267040252685547 - training/rollout_actor_probs_pearson_corr:0.9980368614196777 - rollout_corr/rollout_is_mean:0.9997654557228088 - rollout_corr/rollout_is_ratio_fraction_high:2.3248514480656013e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.855645179748535 - rollout_corr/rollout_is_min:0.5350923538208008 - rollout_corr/rollout_is_std:0.03936876356601715 - rollout_corr/rollout_is_eff_sample_size:0.9984516075893486 - rollout_corr/rollout_is_seq_mean:0.9997847676277161 - rollout_corr/rollout_is_seq_std:0.0029975101351737976 - rollout_corr/rollout_is_seq_max:1.0126070976257324 - rollout_corr/rollout_is_seq_min:0.9907597303390503 - rollout_corr/rollout_is_seq_max_deviation:0.012607097625732422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2693150429986417) - rollout_corr/training_log_ppl:np.float64(0.2345841708010994) - rollout_corr/kl:np.float64(0.0011821451473537081) - rollout_corr/k3_kl:np.float64(0.0013152157603144587) - rollout_corr/rollout_ppl:np.float64(1.267772528808564) - rollout_corr/rollout_log_ppl:np.float64(0.23340202278632205) - rollout_corr/log_ppl_diff:np.float64(0.00118214801477734) - rollout_corr/log_ppl_abs_diff:np.float64(0.002665369029273279) - rollout_corr/log_ppl_diff_max:np.float64(0.010928303003311157) - rollout_corr/log_ppl_diff_min:np.float64(-0.00843799114227295) - rollout_corr/ppl_ratio:np.float64(1.0011883925180882) - rollout_corr/chi2_token:np.float64(0.0032832033466547728) - rollout_corr/chi2_seq:np.float64(2.628566562430933) - actor/pg_loss:np.float64(0.00014865747652947903) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017586877143003221) - actor/ppo_kl:np.float64(2.5402906045357554e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5996150523424149) - perf/mfu/actor:np.float64(0.061039498769682454) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.6258544921875) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:3 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019761237781494856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019761237781494856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:336.04296875 - response_length/max:1579.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:336.04296875 - response_length_non_aborted/max:1579.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.0625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010008178651332855 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1094308514147997) - timing_s/agent_loop/generate_sequences/max:np.float64(9.863438200205564) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3283677981307846) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.863438200205564) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1579 - timing_s/gen:15.612013388425112 - timing_s/reward:0.05625380948185921 - timing_s/old_log_prob:2.4509079605340958 - timing_s/adv:0.1235277634114027 - timing_s/update_actor:9.913907954469323 - timing_s/step:28.248313438147306 - timing_s/stop_profile:3.726966679096222e-05 - timing_per_token_ms/gen:0.18147806372912123 - timing_per_token_ms/update_actor:0.0581382450136892 - timing_per_token_ms/adv:0.0007244052908487576 - perf/total_num_tokens:170523 - perf/time_per_step:28.248313438147306 - perf/throughput:754.5715975812958 (TaskRunner pid=2099163) Training Progress: 51%|█████ | 51/100 [43:52<43:56, 53.81s/it] (TaskRunner pid=2099163) step:52 - global_seqlen/min:12379 - global_seqlen/max:22705 - global_seqlen/minmax_diff:10326 - global_seqlen/balanced_min:16786 - global_seqlen/balanced_max:16849 - global_seqlen/mean:16796.75 - actor/entropy:0.22803975641727448 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36542510986328125 - training/rollout_probs_diff_mean:0.0053499056957662106 - training/rollout_probs_diff_std:0.01391876582056284 - training/rollout_actor_probs_pearson_corr:0.9980417490005493 - rollout_corr/rollout_is_mean:1.0001552104949951 - rollout_corr/rollout_is_ratio_fraction_high:4.7668985644122586e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.355864752549678e-05 - rollout_corr/rollout_is_max:2.923143148422241 - rollout_corr/rollout_is_min:0.39950257539749146 - rollout_corr/rollout_is_std:0.04149097949266434 - rollout_corr/rollout_is_eff_sample_size:0.9982820511233786 - rollout_corr/rollout_is_seq_mean:1.0000829696655273 - rollout_corr/rollout_is_seq_std:0.003001436125487089 - rollout_corr/rollout_is_seq_max:1.011511206626892 - rollout_corr/rollout_is_seq_min:0.990175724029541 - rollout_corr/rollout_is_seq_max_deviation:0.01151120662689209 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2837047763168812) - rollout_corr/training_log_ppl:np.float64(0.24606362084159628) - rollout_corr/kl:np.float64(0.0014448560898543406) - rollout_corr/k3_kl:np.float64(0.0011887111170381104) - rollout_corr/rollout_ppl:np.float64(1.281900660134852) - rollout_corr/rollout_log_ppl:np.float64(0.24461876275017858) - rollout_corr/log_ppl_diff:np.float64(0.0014448580914177) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029126321896910667) - rollout_corr/log_ppl_diff_max:np.float64(0.013406842947006226) - rollout_corr/log_ppl_diff_min:np.float64(-0.008877873420715332) - rollout_corr/ppl_ratio:np.float64(1.001452179858461) - rollout_corr/chi2_token:np.float64(0.0019569701980799437) - rollout_corr/chi2_seq:np.float64(0.4788138067815453) - actor/pg_loss:np.float64(0.00014281971380114555) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018395553652226226) - actor/ppo_kl:np.float64(0.0005606412955785345) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6134368255734444) - perf/mfu/actor:np.float64(0.047977674968366274) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.7822380065918) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:3 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01040375605225563 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01040375605225563 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:245.8359375 - response_length/max:1193.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:245.8359375 - response_length_non_aborted/max:1193.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.0625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.267452120780945e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9509514290839434) - timing_s/agent_loop/generate_sequences/max:np.float64(7.854211928322911) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.525260150054237) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.854211928322911) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1193 - timing_s/gen:13.733885359019041 - timing_s/reward:0.05357689596712589 - timing_s/old_log_prob:2.40861296094954 - timing_s/adv:0.1382129043340683 - timing_s/update_actor:10.0369862485677 - timing_s/step:26.460144393146038 - timing_s/stop_profile:2.820044755935669e-05 - timing_per_token_ms/gen:0.21822679885306895 - timing_per_token_ms/update_actor:0.07469440701748627 - timing_per_token_ms/adv:0.001028568802998112 - perf/total_num_tokens:134374 - perf/time_per_step:26.460144393146038 - perf/throughput:634.7943439171427 (TaskRunner pid=2099163) Training Progress: 52%|█████▏ | 52/100 [44:19<36:29, 45.61s/it] (TaskRunner pid=2099163) step:53 - global_seqlen/min:15429 - global_seqlen/max:26564 - global_seqlen/minmax_diff:11135 - global_seqlen/balanced_min:20620 - global_seqlen/balanced_max:20653 - global_seqlen/mean:20644.25 - actor/entropy:0.20042356848716736 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535129427909851 - training/rollout_probs_diff_mean:0.0044761584140360355 - training/rollout_probs_diff_std:0.012780102901160717 - training/rollout_actor_probs_pearson_corr:0.9982099533081055 - rollout_corr/rollout_is_mean:0.9998370409011841 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012490943481680006 - rollout_corr/rollout_is_max:1.8165900707244873 - rollout_corr/rollout_is_min:0.2722632586956024 - rollout_corr/rollout_is_std:0.03822029009461403 - rollout_corr/rollout_is_eff_sample_size:0.9985411024389872 - rollout_corr/rollout_is_seq_mean:0.9997756481170654 - rollout_corr/rollout_is_seq_std:0.0028406018391251564 - rollout_corr/rollout_is_seq_max:1.0108007192611694 - rollout_corr/rollout_is_seq_min:0.9895545840263367 - rollout_corr/rollout_is_seq_max_deviation:0.010800719261169434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2743919994682074) - rollout_corr/training_log_ppl:np.float64(0.237915026533301) - rollout_corr/kl:np.float64(0.0014105199338541041) - rollout_corr/k3_kl:np.float64(0.0010296943997332164) - rollout_corr/rollout_ppl:np.float64(1.2725408389233053) - rollout_corr/rollout_log_ppl:np.float64(0.23650450706190895) - rollout_corr/log_ppl_diff:np.float64(0.0014105194713920355) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024846792221069336) - rollout_corr/log_ppl_diff_max:np.float64(0.014649003744125366) - rollout_corr/log_ppl_diff_min:np.float64(-0.0049605220556259155) - rollout_corr/ppl_ratio:np.float64(1.0014160515274853) - rollout_corr/chi2_token:np.float64(0.0013009211979806423) - rollout_corr/chi2_seq:np.float64(0.20528000919148326) - actor/pg_loss:np.float64(0.00033905566670000553) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011068140559018502) - actor/ppo_kl:np.float64(0.00029123531216690424) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.495539128780365) - perf/mfu/actor:np.float64(0.059122989206736365) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.70719528198242) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:3 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01337155606597662 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01337155606597662 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:312.7265625 - response_length/max:1431.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:312.7265625 - response_length_non_aborted/max:1431.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.40625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.622953176498413e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7730677053332329) - timing_s/agent_loop/generate_sequences/max:np.float64(9.259236015379429) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0541376218679943) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.259236015379429) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1431 - timing_s/gen:14.91242471896112 - timing_s/reward:0.059695588424801826 - timing_s/old_log_prob:2.3497717920690775 - timing_s/adv:0.12111933156847954 - timing_s/update_actor:9.819533618167043 - timing_s/step:27.35314425267279 - timing_s/stop_profile:0.00012644007802009583 - timing_per_token_ms/gen:0.18627026304630542 - timing_per_token_ms/update_actor:0.05945683191546704 - timing_per_token_ms/adv:0.0007333720743577482 - perf/total_num_tokens:165154 - perf/time_per_step:27.35314425267279 - perf/throughput:754.7304181669266 (TaskRunner pid=2099163) Training Progress: 53%|█████▎ | 53/100 [44:46<31:26, 40.15s/it] (TaskRunner pid=2099163) step:54 - global_seqlen/min:13022 - global_seqlen/max:20901 - global_seqlen/minmax_diff:7879 - global_seqlen/balanced_min:17654 - global_seqlen/balanced_max:19215 - global_seqlen/mean:17867.875 - actor/entropy:0.21646106243133545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5828086733818054 - training/rollout_probs_diff_mean:0.004788811318576336 - training/rollout_probs_diff_std:0.013526316732168198 - training/rollout_actor_probs_pearson_corr:0.9981729388237 - rollout_corr/rollout_is_mean:0.9998193979263306 - rollout_corr/rollout_is_ratio_fraction_high:1.4932727935956791e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010452909918967634 - rollout_corr/rollout_is_max:2.679741144180298 - rollout_corr/rollout_is_min:0.20122860372066498 - rollout_corr/rollout_is_std:0.040283203125 - rollout_corr/rollout_is_eff_sample_size:0.9983792390254678 - rollout_corr/rollout_is_seq_mean:0.9996543526649475 - rollout_corr/rollout_is_seq_std:0.0032862715888768435 - rollout_corr/rollout_is_seq_max:1.0084000825881958 - rollout_corr/rollout_is_seq_min:0.9903689622879028 - rollout_corr/rollout_is_seq_max_deviation:0.009631037712097168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2988018337637186) - rollout_corr/training_log_ppl:np.float64(0.257318063187995) - rollout_corr/kl:np.float64(0.0014489969444486661) - rollout_corr/k3_kl:np.float64(0.001245539906221893) - rollout_corr/rollout_ppl:np.float64(1.2968687978573143) - rollout_corr/rollout_log_ppl:np.float64(0.25586906356329564) - rollout_corr/log_ppl_diff:np.float64(0.0014489996246993542) - rollout_corr/log_ppl_abs_diff:np.float64(0.002985531697049737) - rollout_corr/log_ppl_diff_max:np.float64(0.013407379388809204) - rollout_corr/log_ppl_diff_min:np.float64(-0.011749684810638428) - rollout_corr/ppl_ratio:np.float64(1.0014570830389857) - rollout_corr/chi2_token:np.float64(0.002103503793478012) - rollout_corr/chi2_seq:np.float64(0.6353278271853924) - actor/pg_loss:np.float64(0.0002622948959469795) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012713276441900234) - actor/ppo_kl:np.float64(0.00012942606369392706) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5507104620337486) - perf/mfu/actor:np.float64(0.05088630331246074) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.86227798461914) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:3 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008711379952728748 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008711379952728748 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.58984375 - response_length/max:2899.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.58984375 - response_length_non_aborted/max:2899.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.422190487384796e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6486075595021248) - timing_s/agent_loop/generate_sequences/max:np.float64(15.47397425211966) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.486256126627268) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.47397425211966) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:2899 - timing_s/gen:21.19790812022984 - timing_s/reward:0.046014051884412766 - timing_s/old_log_prob:2.4986167792230844 - timing_s/adv:0.1188246849924326 - timing_s/update_actor:9.975375892594457 - timing_s/step:33.96630131267011 - timing_s/stop_profile:0.00012148916721343994 - timing_per_token_ms/gen:0.3165425974021509 - timing_per_token_ms/update_actor:0.06978569004844208 - timing_per_token_ms/adv:0.000831273199753976 - perf/total_num_tokens:142943 - perf/time_per_step:33.96630131267011 - perf/throughput:526.0471205127926 (TaskRunner pid=2099163) Training Progress: 54%|█████▍ | 54/100 [45:20<29:22, 38.31s/it] (TaskRunner pid=2099163) step:55 - global_seqlen/min:16785 - global_seqlen/max:28225 - global_seqlen/minmax_diff:11440 - global_seqlen/balanced_min:21265 - global_seqlen/balanced_max:22805 - global_seqlen/mean:21496.5 - actor/entropy:0.1851159632205963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7663038969039917 - training/rollout_probs_diff_mean:0.004267675802111626 - training/rollout_probs_diff_std:0.012877827510237694 - training/rollout_actor_probs_pearson_corr:0.9981168508529663 - rollout_corr/rollout_is_mean:0.9998764991760254 - rollout_corr/rollout_is_ratio_fraction_high:5.7237051805714145e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.578964217216708e-05 - rollout_corr/rollout_is_max:2.459521770477295 - rollout_corr/rollout_is_min:0.02907703071832657 - rollout_corr/rollout_is_std:0.03838213533163071 - rollout_corr/rollout_is_eff_sample_size:0.9985286221149012 - rollout_corr/rollout_is_seq_mean:0.9999008178710938 - rollout_corr/rollout_is_seq_std:0.002929779700934887 - rollout_corr/rollout_is_seq_max:1.008970022201538 - rollout_corr/rollout_is_seq_min:0.9923213124275208 - rollout_corr/rollout_is_seq_max_deviation:0.008970022201538086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2743566152639687) - rollout_corr/training_log_ppl:np.float64(0.23732106332317926) - rollout_corr/kl:np.float64(0.0011733787810150886) - rollout_corr/k3_kl:np.float64(0.0012601886370475768) - rollout_corr/rollout_ppl:np.float64(1.2727803294546902) - rollout_corr/rollout_log_ppl:np.float64(0.23614768433617428) - rollout_corr/log_ppl_diff:np.float64(0.0011733789870049804) - rollout_corr/log_ppl_abs_diff:np.float64(0.002733354893280193) - rollout_corr/log_ppl_diff_max:np.float64(0.014354616403579712) - rollout_corr/log_ppl_diff_min:np.float64(-0.008024752140045166) - rollout_corr/ppl_ratio:np.float64(1.0011803007218987) - rollout_corr/chi2_token:np.float64(0.0026173368096351624) - rollout_corr/chi2_seq:np.float64(0.7098474926315248) - actor/pg_loss:np.float64(0.00024044979363679886) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017808543852879666) - actor/ppo_kl:np.float64(0.00014058525295723712) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5874743461608887) - perf/mfu/actor:np.float64(0.061257645386096266) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.73693084716797) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:3 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010787753388285637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010787753388285637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:341.234375 - response_length/max:3184.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:341.234375 - response_length_non_aborted/max:3184.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:498.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.620934724807739e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9831281434744596) - timing_s/agent_loop/generate_sequences/max:np.float64(17.57840107381344) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.278305432460911) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.57840107381344) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:3184 - timing_s/gen:23.38557505980134 - timing_s/reward:0.05253286845982075 - timing_s/old_log_prob:2.4493800587952137 - timing_s/adv:0.12223737873136997 - timing_s/update_actor:10.099678717553616 - timing_s/step:36.21919436752796 - timing_s/stop_profile:0.00011761859059333801 - timing_per_token_ms/gen:0.2677042797266512 - timing_per_token_ms/update_actor:0.05872862278483483 - timing_per_token_ms/adv:0.0007107981458107713 - perf/total_num_tokens:171972 - perf/time_per_step:36.21919436752796 - perf/throughput:593.5112686899662 (TaskRunner pid=2099163) Training Progress: 55%|█████▌ | 55/100 [45:56<28:16, 37.69s/it] (TaskRunner pid=2099163) step:56 - global_seqlen/min:11908 - global_seqlen/max:34677 - global_seqlen/minmax_diff:22769 - global_seqlen/balanced_min:19857 - global_seqlen/balanced_max:21973 - global_seqlen/mean:20175.375 - actor/entropy:0.19302956759929657 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5312970876693726 - training/rollout_probs_diff_mean:0.0045180413872003555 - training/rollout_probs_diff_std:0.01335164438933134 - training/rollout_actor_probs_pearson_corr:0.9980792999267578 - rollout_corr/rollout_is_mean:0.9997986555099487 - rollout_corr/rollout_is_ratio_fraction_high:1.2620685083675198e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.834479376673698e-05 - rollout_corr/rollout_is_max:2.0223238468170166 - rollout_corr/rollout_is_min:0.36936911940574646 - rollout_corr/rollout_is_std:0.03903120756149292 - rollout_corr/rollout_is_eff_sample_size:0.9984784662444588 - rollout_corr/rollout_is_seq_mean:0.9998159408569336 - rollout_corr/rollout_is_seq_std:0.0029684919863939285 - rollout_corr/rollout_is_seq_max:1.0116934776306152 - rollout_corr/rollout_is_seq_min:0.9913424253463745 - rollout_corr/rollout_is_seq_max_deviation:0.011693477630615234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2815037313848734) - rollout_corr/training_log_ppl:np.float64(0.24245586230244953) - rollout_corr/kl:np.float64(0.0015198735749812897) - rollout_corr/k3_kl:np.float64(0.0011902167615289727) - rollout_corr/rollout_ppl:np.float64(1.2794606960378587) - rollout_corr/rollout_log_ppl:np.float64(0.24093598501349334) - rollout_corr/log_ppl_diff:np.float64(0.0015198772889561951) - rollout_corr/log_ppl_abs_diff:np.float64(0.002925883134594187) - rollout_corr/log_ppl_diff_max:np.float64(0.015707731246948242) - rollout_corr/log_ppl_diff_min:np.float64(-0.008237093687057495) - rollout_corr/ppl_ratio:np.float64(1.0015279354993254) - rollout_corr/chi2_token:np.float64(0.001645366894081235) - rollout_corr/chi2_seq:np.float64(0.5515149484854192) - actor/pg_loss:np.float64(-7.30195315554738e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013563435145442782) - actor/ppo_kl:np.float64(0.0003750655004007841) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6355954259634018) - perf/mfu/actor:np.float64(0.057120220065657104) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.86117172241211) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:3 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028096800670027733 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0028096800670027733 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:309.51171875 - response_length/max:3693.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:309.51171875 - response_length_non_aborted/max:3693.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:320.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010253302752971649 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9995839446783066) - timing_s/agent_loop/generate_sequences/max:np.float64(19.118556136265397) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.989160280507349) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.118556136265397) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:3693 - timing_s/gen:25.32910566404462 - timing_s/reward:0.05742966942489147 - timing_s/old_log_prob:2.481884676963091 - timing_s/adv:0.12075792998075485 - timing_s/update_actor:10.213837813585997 - timing_s/step:38.28866018541157 - timing_s/stop_profile:0.00011267699301242828 - timing_per_token_ms/gen:0.31967067159771084 - timing_per_token_ms/update_actor:0.06328158592830367 - timing_per_token_ms/adv:0.0007481764897849163 - perf/total_num_tokens:161403 - perf/time_per_step:38.28866018541157 - perf/throughput:526.9282054347531 (TaskRunner pid=2099163) Training Progress: 56%|█████▌ | 56/100 [46:35<27:47, 37.89s/it] (TaskRunner pid=2099163) step:57 - global_seqlen/min:18300 - global_seqlen/max:23742 - global_seqlen/minmax_diff:5442 - global_seqlen/balanced_min:20262 - global_seqlen/balanced_max:20589 - global_seqlen/mean:20356.625 - actor/entropy:0.1873641461133957 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9385040998458862 - training/rollout_probs_diff_mean:0.004480709787458181 - training/rollout_probs_diff_std:0.013861753977835178 - training/rollout_actor_probs_pearson_corr:0.9978734850883484 - rollout_corr/rollout_is_mean:1.0002038478851318 - rollout_corr/rollout_is_ratio_fraction_high:2.4369737730012275e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.529407932655886e-05 - rollout_corr/rollout_is_max:2.595475912094116 - rollout_corr/rollout_is_min:0.0481024831533432 - rollout_corr/rollout_is_std:0.03987497463822365 - rollout_corr/rollout_is_eff_sample_size:0.9984132236126418 - rollout_corr/rollout_is_seq_mean:1.000412940979004 - rollout_corr/rollout_is_seq_std:0.0028782689478248358 - rollout_corr/rollout_is_seq_max:1.0121479034423828 - rollout_corr/rollout_is_seq_min:0.992722749710083 - rollout_corr/rollout_is_seq_max_deviation:0.012147903442382812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2728352225385606) - rollout_corr/training_log_ppl:np.float64(0.23661376311792992) - rollout_corr/kl:np.float64(0.0012007082295639737) - rollout_corr/k3_kl:np.float64(0.0011471591212455223) - rollout_corr/rollout_ppl:np.float64(1.2712574996985495) - rollout_corr/rollout_log_ppl:np.float64(0.23541305179242045) - rollout_corr/log_ppl_diff:np.float64(0.0012007113255094737) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025620612141210586) - rollout_corr/log_ppl_diff_max:np.float64(0.013712316751480103) - rollout_corr/log_ppl_diff_min:np.float64(-0.00932343304157257) - rollout_corr/ppl_ratio:np.float64(1.001206706976518) - rollout_corr/chi2_token:np.float64(0.002160461153835058) - rollout_corr/chi2_seq:np.float64(0.438829617574811) - actor/pg_loss:np.float64(0.00015424867160618305) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00141477917463817) - actor/ppo_kl:np.float64(0.0006155347940577371) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5178738236427307) - perf/mfu/actor:np.float64(0.058537288240303825) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.82574844360352) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:4 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.013144426979124546 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.013144426979124546 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:320.58203125 - response_length/max:2049.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:320.58203125 - response_length_non_aborted/max:2049.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.5625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000209735706448555 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0480106547474861) - timing_s/agent_loop/generate_sequences/max:np.float64(12.178743667900562) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1250689009248163) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.178743667900562) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:2049 - timing_s/gen:18.099144043400884 - timing_s/reward:0.04921534284949303 - timing_s/old_log_prob:2.4653547890484333 - timing_s/adv:0.13351497240364552 - timing_s/update_actor:9.881367065012455 - timing_s/step:30.751196237280965 - timing_s/stop_profile:2.9180198907852173e-05 - timing_per_token_ms/gen:0.22053569610207122 - timing_per_token_ms/update_actor:0.06067660445317222 - timing_per_token_ms/adv:0.0008198496337411378 - perf/total_num_tokens:162853 - perf/time_per_step:30.751196237280965 - perf/throughput:661.9783127435156 (TaskRunner pid=2099163) Training Progress: 57%|█████▋ | 57/100 [47:06<25:43, 35.88s/it] (TaskRunner pid=2099163) step:58 - global_seqlen/min:13787 - global_seqlen/max:26334 - global_seqlen/minmax_diff:12547 - global_seqlen/balanced_min:19233 - global_seqlen/balanced_max:19250 - global_seqlen/mean:19242.625 - actor/entropy:0.21358489990234375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141245484352112 - training/rollout_probs_diff_mean:0.00496778916567564 - training/rollout_probs_diff_std:0.013776712119579315 - training/rollout_actor_probs_pearson_corr:0.9979900121688843 - rollout_corr/rollout_is_mean:1.0003639459609985 - rollout_corr/rollout_is_ratio_fraction_high:2.7048592528444715e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.762148404959589e-05 - rollout_corr/rollout_is_max:2.116508960723877 - rollout_corr/rollout_is_min:0.3293769061565399 - rollout_corr/rollout_is_std:0.04176446422934532 - rollout_corr/rollout_is_eff_sample_size:0.9982600735978991 - rollout_corr/rollout_is_seq_mean:1.000337839126587 - rollout_corr/rollout_is_seq_std:0.0029638041742146015 - rollout_corr/rollout_is_seq_max:1.0089969635009766 - rollout_corr/rollout_is_seq_min:0.9919787049293518 - rollout_corr/rollout_is_seq_max_deviation:0.008996963500976562 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2812516475096345) - rollout_corr/training_log_ppl:np.float64(0.2435778763901908) - rollout_corr/kl:np.float64(0.0010009155997465768) - rollout_corr/k3_kl:np.float64(0.0012113901442489805) - rollout_corr/rollout_ppl:np.float64(1.2799442182295024) - rollout_corr/rollout_log_ppl:np.float64(0.24257695997948758) - rollout_corr/log_ppl_diff:np.float64(0.001000916410703212) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027250831481069326) - rollout_corr/log_ppl_diff_max:np.float64(0.01267920434474945) - rollout_corr/log_ppl_diff_min:np.float64(-0.010362982749938965) - rollout_corr/ppl_ratio:np.float64(1.0010073960293084) - rollout_corr/chi2_token:np.float64(0.002911286661401391) - rollout_corr/chi2_seq:np.float64(0.7661588522605598) - actor/pg_loss:np.float64(0.0003306339494884014) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00199477257979197) - actor/ppo_kl:np.float64(0.0003630212834728752) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5990487411618233) - perf/mfu/actor:np.float64(0.05602923257711591) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.89770126342773) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:4 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001112373429350555 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001112373429350555 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.83203125 - response_length/max:1110.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:288.83203125 - response_length_non_aborted/max:1110.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.475431680679321e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0492791645228863) - timing_s/agent_loop/generate_sequences/max:np.float64(7.552903410047293) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9448991375684272) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.552903410047293) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1110 - timing_s/gen:13.404031103476882 - timing_s/reward:0.057840101420879364 - timing_s/old_log_prob:2.417235178872943 - timing_s/adv:0.11892589367926121 - timing_s/update_actor:9.727152958512306 - timing_s/step:25.811423553153872 - timing_s/stop_profile:0.00012199766933917999 - timing_per_token_ms/gen:0.18128008957786454 - timing_per_token_ms/update_actor:0.06318753911246715 - timing_per_token_ms/adv:0.0007725420367495418 - perf/total_num_tokens:153941 - perf/time_per_step:25.811423553153872 - perf/throughput:745.5080871604528 (TaskRunner pid=2099163) Training Progress: 58%|█████▊ | 58/100 [47:32<23:00, 32.88s/it] (TaskRunner pid=2099163) step:59 - global_seqlen/min:14785 - global_seqlen/max:28128 - global_seqlen/minmax_diff:13343 - global_seqlen/balanced_min:20039 - global_seqlen/balanced_max:23101 - global_seqlen/mean:20570.0 - actor/entropy:0.19363823533058167 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3583551049232483 - training/rollout_probs_diff_mean:0.004458777606487274 - training/rollout_probs_diff_std:0.012819879688322544 - training/rollout_actor_probs_pearson_corr:0.9981608390808105 - rollout_corr/rollout_is_mean:1.0000730752944946 - rollout_corr/rollout_is_ratio_fraction_high:4.982560858479701e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.719482138985768e-05 - rollout_corr/rollout_is_max:2.615692138671875 - rollout_corr/rollout_is_min:0.29617154598236084 - rollout_corr/rollout_is_std:0.038851357996463776 - rollout_corr/rollout_is_eff_sample_size:0.9984930846153224 - rollout_corr/rollout_is_seq_mean:0.9999926686286926 - rollout_corr/rollout_is_seq_std:0.0030627932865172625 - rollout_corr/rollout_is_seq_max:1.009785532951355 - rollout_corr/rollout_is_seq_min:0.9908705949783325 - rollout_corr/rollout_is_seq_max_deviation:0.00978553295135498 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2848408012650907) - rollout_corr/training_log_ppl:np.float64(0.24571586469028261) - rollout_corr/kl:np.float64(0.001712628750690115) - rollout_corr/k3_kl:np.float64(0.0012789307371718905) - rollout_corr/rollout_ppl:np.float64(1.2825767188332975) - rollout_corr/rollout_log_ppl:np.float64(0.24400323344525532) - rollout_corr/log_ppl_diff:np.float64(0.0017126312450272962) - rollout_corr/log_ppl_abs_diff:np.float64(0.002915053817559965) - rollout_corr/log_ppl_diff_max:np.float64(0.011791497468948364) - rollout_corr/log_ppl_diff_min:np.float64(-0.008301064372062683) - rollout_corr/ppl_ratio:np.float64(1.0017199639696628) - rollout_corr/chi2_token:np.float64(0.0019151861779391766) - rollout_corr/chi2_seq:np.float64(0.34929756191559136) - actor/pg_loss:np.float64(2.256268635392189e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014333776648527419) - actor/ppo_kl:np.float64(0.0007561920642880082) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5470746606588364) - perf/mfu/actor:np.float64(0.05836452796553154) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.88081741333008) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:4 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03495578095316887 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03495578095316887 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:313.59375 - response_length/max:4644.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:313.59375 - response_length_non_aborted/max:4644.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:329.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001398380845785141 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8809522464871407) - timing_s/agent_loop/generate_sequences/max:np.float64(23.754080295562744) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0718314567784546) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.754080295562744) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:4644 - timing_s/gen:29.604787712916732 - timing_s/reward:0.04805128648877144 - timing_s/old_log_prob:2.5187726486474276 - timing_s/adv:0.11889026686549187 - timing_s/update_actor:10.145638149231672 - timing_s/step:42.470872562378645 - timing_s/stop_profile:0.00012224912643432617 - timing_per_token_ms/gen:0.3687691543711601 - timing_per_token_ms/update_actor:0.06165312438764993 - timing_per_token_ms/adv:0.0007224736683610347 - perf/total_num_tokens:164560 - perf/time_per_step:42.470872562378645 - perf/throughput:484.33193760707485 (TaskRunner pid=2099163) Training Progress: 59%|█████▉ | 59/100 [48:14<24:26, 35.77s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a protein sequence is typically determined using computational methods such as those implemented in tools like DeepFold, Rosetta, or other protein structure prediction software. The provided sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR' is a specific amino acid sequence, and without access to specific computational tools or databases that calculate folding stability scores, it is not possible to determine the exact folding stability score for this sequence. Therefore, the score cannot be determined from the given information. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2099163) step:60 - global_seqlen/min:11482 - global_seqlen/max:23421 - global_seqlen/minmax_diff:11939 - global_seqlen/balanced_min:16763 - global_seqlen/balanced_max:16948 - global_seqlen/mean:16792.25 - actor/entropy:0.24822477996349335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30099523067474365 - training/rollout_probs_diff_mean:0.005300878547132015 - training/rollout_probs_diff_std:0.01358079444617033 - training/rollout_actor_probs_pearson_corr:0.9982614517211914 - rollout_corr/rollout_is_mean:1.000307321548462 - rollout_corr/rollout_is_ratio_fraction_high:3.160056803608313e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.160056803608313e-05 - rollout_corr/rollout_is_max:2.7097694873809814 - rollout_corr/rollout_is_min:0.2010701298713684 - rollout_corr/rollout_is_std:0.041654426604509354 - rollout_corr/rollout_is_eff_sample_size:0.9982688644919676 - rollout_corr/rollout_is_seq_mean:1.000412106513977 - rollout_corr/rollout_is_seq_std:0.003237298224121332 - rollout_corr/rollout_is_seq_max:1.0087321996688843 - rollout_corr/rollout_is_seq_min:0.9887025952339172 - rollout_corr/rollout_is_seq_max_deviation:0.011297404766082764 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3292968641035259) - rollout_corr/training_log_ppl:np.float64(0.2800096425926313) - rollout_corr/kl:np.float64(0.0010625352792885678) - rollout_corr/k3_kl:np.float64(0.0012574967236105294) - rollout_corr/rollout_ppl:np.float64(1.3278319356031716) - rollout_corr/rollout_log_ppl:np.float64(0.2789471071882872) - rollout_corr/log_ppl_diff:np.float64(0.0010625354043440893) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028606804116861895) - rollout_corr/log_ppl_diff_max:np.float64(0.01503533124923706) - rollout_corr/log_ppl_diff_min:np.float64(-0.010933548212051392) - rollout_corr/ppl_ratio:np.float64(1.0010700710117817) - rollout_corr/chi2_token:np.float64(0.0029621769208461046) - rollout_corr/chi2_seq:np.float64(1.0398543775081635) - actor/pg_loss:np.float64(0.0007010122062638402) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012734923266179976) - actor/ppo_kl:np.float64(0.0005196989214417691) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6372415572404861) - perf/mfu/actor:np.float64(0.049047718363570796) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.9422492980957) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3825) - val-aux/sciknoweval/reward/std@16:np.float64(0.2596834756971686) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.49317999999999995) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2221376792011573) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.27333999999999997) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.22152763913536153) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.38454) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2597535466020497) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5833200000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15799078019111001) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.18596000000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15643648114578526) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.39120000000000005) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.21626416144931568) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6409199999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09541131973399612) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.12792) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09909837524937166) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.39025999999999994) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16916725228527277) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.67412) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05028767227181808) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.08998) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.058669001814606235) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.38689999999999997) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13557260155312797) - val-aux/sciknoweval/score/mean@16:np.float64(0.3825) - val-aux/sciknoweval/score/std@16:np.float64(0.2596834756971686) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.49317999999999995) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2221376792011573) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.27333999999999997) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.22152763913536153) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.38454) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2597535466020497) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5833200000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15799078019111001) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.18596000000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15643648114578526) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.39120000000000005) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.21626416144931568) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6409199999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09541131973399612) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.12792) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09909837524937166) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.39025999999999994) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.16916725228527277) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.67412) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05028767227181808) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.08998) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.058669001814606235) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.38689999999999997) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13557260155312797) - val-core/sciknoweval/acc/mean@16:np.float64(0.3825) - val-aux/sciknoweval/acc/std@16:np.float64(0.2596834756971686) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.49317999999999995) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2221376792011573) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.27333999999999997) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.22152763913536153) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.38454) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2597535466020497) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5833200000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15799078019111001) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.18596000000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15643648114578526) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.39120000000000005) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.21626416144931568) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6409199999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09541131973399612) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.12792) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09909837524937166) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.39025999999999994) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.16916725228527277) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.67412) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05028767227181808) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.08998) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.058669001814606235) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.38689999999999997) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13557260155312797) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.02113806582593929) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99956) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.00561792069220043) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9884999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0273368559769371) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99374) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.020753176673987798) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.97858) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.03448809285937782) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9984999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.010504955437387428) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.96292) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.038690345327705815) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9998600000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0028808653454051197) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9443) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.03543438166339959) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:4 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01805972494184971 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01805972494184971 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:247.2265625 - response_length/max:1516.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:247.2265625 - response_length_non_aborted/max:1516.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013054721057415009 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6093897819519043) - timing_s/agent_loop/generate_sequences/max:np.float64(8.942918440327048) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4281441207785974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.942918440327048) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:1516 - timing_s/gen:14.714311951771379 - timing_s/reward:0.05381259322166443 - timing_s/old_log_prob:2.322748025879264 - timing_s/adv:0.12639613077044487 - timing_s/update_actor:9.668666616082191 - timing_s/step:26.97286426089704 - timing_s/testing:71.51457652077079 - timing_s/save_checkpoint:6.8972090892493725 - timing_s/stop_profile:0.00016043893992900848 - timing_per_token_ms/gen:0.23249031366363374 - timing_per_token_ms/update_actor:0.07197268543585725 - timing_per_token_ms/adv:0.0009408814391344584 - perf/total_num_tokens:134338 - perf/time_per_step:26.97286426089704 - perf/throughput:622.5608759075681 (TaskRunner pid=2099163) Training Progress: 60%|██████ | 60/100 [50:00<37:46, 56.67s/it] (TaskRunner pid=2099163) step:61 - global_seqlen/min:11756 - global_seqlen/max:28743 - global_seqlen/minmax_diff:16987 - global_seqlen/balanced_min:17559 - global_seqlen/balanced_max:20581 - global_seqlen/mean:18308.5 - actor/entropy:0.22188033163547516 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7668329477310181 - training/rollout_probs_diff_mean:0.004735196474939585 - training/rollout_probs_diff_std:0.01345571968704462 - training/rollout_actor_probs_pearson_corr:0.9982025623321533 - rollout_corr/rollout_is_mean:1.000156283378601 - rollout_corr/rollout_is_ratio_fraction_high:2.8573062081704848e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.285959221306257e-05 - rollout_corr/rollout_is_max:10.01133918762207 - rollout_corr/rollout_is_min:0.23175029456615448 - rollout_corr/rollout_is_std:0.03982262313365936 - rollout_corr/rollout_is_eff_sample_size:0.9984169073956883 - rollout_corr/rollout_is_seq_mean:1.000264286994934 - rollout_corr/rollout_is_seq_std:0.0033271322026848793 - rollout_corr/rollout_is_seq_max:1.0181152820587158 - rollout_corr/rollout_is_seq_min:0.9914125800132751 - rollout_corr/rollout_is_seq_max_deviation:0.01811528205871582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3211746821179986) - rollout_corr/training_log_ppl:np.float64(0.2735010301985312) - rollout_corr/kl:np.float64(0.0008562939778675727) - rollout_corr/k3_kl:np.float64(0.0012789235939010268) - rollout_corr/rollout_ppl:np.float64(1.3200623537413776) - rollout_corr/rollout_log_ppl:np.float64(0.27264473447576165) - rollout_corr/log_ppl_diff:np.float64(0.0008562957227695733) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028399873117450625) - rollout_corr/log_ppl_diff_max:np.float64(0.014549434185028076) - rollout_corr/log_ppl_diff_min:np.float64(-0.008745625615119934) - rollout_corr/ppl_ratio:np.float64(1.0008633281104267) - rollout_corr/chi2_token:np.float64(0.004113542148843408) - rollout_corr/chi2_seq:np.float64(0.8988656422588974) - actor/pg_loss:np.float64(0.0002667094813659787) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015468199947576977) - actor/ppo_kl:np.float64(9.708452350309926e-05) - actor/pg_clipfrac_lower:np.float64(1.4575559362128843e-05) - actor/grad_norm:np.float64(0.5922280102968216) - perf/mfu/actor:np.float64(0.050955802459431765) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.23041534423828) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:4 - critic/score/mean:0.28125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.020483313128352165 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.020483313128352165 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:273.421875 - response_length/max:4322.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:273.421875 - response_length_non_aborted/max:4322.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.061714470386505e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9486130196601152) - timing_s/agent_loop/generate_sequences/max:np.float64(22.567378267645836) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5980923264069133) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.567378267645836) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:4322 - timing_s/gen:28.375739235430956 - timing_s/reward:0.053623124957084656 - timing_s/old_log_prob:2.4645230006426573 - timing_s/adv:0.11367073655128479 - timing_s/update_actor:10.312344249337912 - timing_s/step:41.40952676907182 - timing_s/stop_profile:0.00012328848242759705 - timing_per_token_ms/gen:0.40539086855578826 - timing_per_token_ms/update_actor:0.07040680728444378 - timing_per_token_ms/adv:0.0007760789834727367 - perf/total_num_tokens:146468 - perf/time_per_step:41.40952676907182 - perf/throughput:442.13255809709847 (TaskRunner pid=2099163) Training Progress: 61%|██████ | 61/100 [50:41<33:52, 52.11s/it] (TaskRunner pid=2099163) step:62 - global_seqlen/min:11720 - global_seqlen/max:23390 - global_seqlen/minmax_diff:11670 - global_seqlen/balanced_min:17845 - global_seqlen/balanced_max:18228 - global_seqlen/mean:17903.375 - actor/entropy:0.23856085538864136 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7297425270080566 - training/rollout_probs_diff_mean:0.004906587768346071 - training/rollout_probs_diff_std:0.013448494486510754 - training/rollout_actor_probs_pearson_corr:0.9983062148094177 - rollout_corr/rollout_is_mean:0.9999780654907227 - rollout_corr/rollout_is_ratio_fraction_high:1.470869392505847e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.825216355035082e-05 - rollout_corr/rollout_is_max:2.1296794414520264 - rollout_corr/rollout_is_min:0.06101953238248825 - rollout_corr/rollout_is_std:0.0396200455725193 - rollout_corr/rollout_is_eff_sample_size:0.9984327123217758 - rollout_corr/rollout_is_seq_mean:1.0001113414764404 - rollout_corr/rollout_is_seq_std:0.0030740199144929647 - rollout_corr/rollout_is_seq_max:1.014378547668457 - rollout_corr/rollout_is_seq_min:0.9906818866729736 - rollout_corr/rollout_is_seq_max_deviation:0.014378547668457031 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3296263343654573) - rollout_corr/training_log_ppl:np.float64(0.28007101351977326) - rollout_corr/kl:np.float64(0.0011143518837499755) - rollout_corr/k3_kl:np.float64(0.0012758465617253023) - rollout_corr/rollout_ppl:np.float64(1.3281300985254347) - rollout_corr/rollout_log_ppl:np.float64(0.27895666120457463) - rollout_corr/log_ppl_diff:np.float64(0.00111435231519863) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026538760284893215) - rollout_corr/log_ppl_diff_max:np.float64(0.03442579507827759) - rollout_corr/log_ppl_diff_min:np.float64(-0.012630432844161987) - rollout_corr/ppl_ratio:np.float64(1.0011220904998481) - rollout_corr/chi2_token:np.float64(0.0027193170972168446) - rollout_corr/chi2_seq:np.float64(0.22215013159438968) - actor/pg_loss:np.float64(0.0003498832229524851) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012161810750512814) - actor/ppo_kl:np.float64(0.0002681101446029288) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5286872908473015) - perf/mfu/actor:np.float64(0.051028239589326306) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.716552734375) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:4 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004962345585227013 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004962345585227013 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:265.57421875 - response_length/max:1654.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:265.57421875 - response_length_non_aborted/max:1654.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.90625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.049180567264557e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.962077796459198) - timing_s/agent_loop/generate_sequences/max:np.float64(9.180661825463176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6732208340472425) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.180661825463176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:1654 - timing_s/gen:14.704399591311812 - timing_s/reward:0.054863808676600456 - timing_s/old_log_prob:2.536904999986291 - timing_s/adv:0.11875544860959053 - timing_s/update_actor:9.82179962284863 - timing_s/step:27.321818605065346 - timing_s/stop_profile:0.0001788027584552765 - timing_per_token_ms/gen:0.21628251858902162 - timing_per_token_ms/update_actor:0.06857505653856208 - timing_per_token_ms/adv:0.0008291414929419072 - perf/total_num_tokens:143227 - perf/time_per_step:27.321818605065346 - perf/throughput:655.2775735316826 (TaskRunner pid=2099163) Training Progress: 62%|██████▏ | 62/100 [51:09<28:18, 44.69s/it] (TaskRunner pid=2099163) step:63 - global_seqlen/min:15327 - global_seqlen/max:23927 - global_seqlen/minmax_diff:8600 - global_seqlen/balanced_min:19595 - global_seqlen/balanced_max:26173 - global_seqlen/mean:20418.0 - actor/entropy:0.23164616525173187 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.777930498123169 - training/rollout_probs_diff_mean:0.004647471942007542 - training/rollout_probs_diff_std:0.01323886401951313 - training/rollout_actor_probs_pearson_corr:0.998352587223053 - rollout_corr/rollout_is_mean:0.9999091625213623 - rollout_corr/rollout_is_ratio_fraction_high:2.5786488549783826e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012893244274891913 - rollout_corr/rollout_is_max:3.2854862213134766 - rollout_corr/rollout_is_min:0.04563618823885918 - rollout_corr/rollout_is_std:0.03892645984888077 - rollout_corr/rollout_is_eff_sample_size:0.9984866667412581 - rollout_corr/rollout_is_seq_mean:0.9998888373374939 - rollout_corr/rollout_is_seq_std:0.0031720264814794064 - rollout_corr/rollout_is_seq_max:1.0129690170288086 - rollout_corr/rollout_is_seq_min:0.9918916821479797 - rollout_corr/rollout_is_seq_max_deviation:0.012969017028808594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3354551973752677) - rollout_corr/training_log_ppl:np.float64(0.2843584521106095) - rollout_corr/kl:np.float64(0.0012951181959266478) - rollout_corr/k3_kl:np.float64(0.0012268340739751693) - rollout_corr/rollout_ppl:np.float64(1.3336740545928478) - rollout_corr/rollout_log_ppl:np.float64(0.283063332484744) - rollout_corr/log_ppl_diff:np.float64(0.0012951196258654818) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027128312940476462) - rollout_corr/log_ppl_diff_max:np.float64(0.016135364770889282) - rollout_corr/log_ppl_diff_min:np.float64(-0.010828465223312378) - rollout_corr/ppl_ratio:np.float64(1.0013016369193792) - rollout_corr/chi2_token:np.float64(0.0022587431594729424) - rollout_corr/chi2_seq:np.float64(1.104415416251868) - actor/pg_loss:np.float64(-3.0042603611946106e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019501109186421672) - actor/ppo_kl:np.float64(0.00024567433211086787) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7029889523983002) - perf/mfu/actor:np.float64(0.0549835847859291) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.80621337890625) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:4 - critic/score/mean:0.39453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.39453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.020585674792528152 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.020585674792528152 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:302.96875 - response_length/max:7755.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:302.96875 - response_length_non_aborted/max:7755.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:335.09375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001079123467206955 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0766320154070854) - timing_s/agent_loop/generate_sequences/max:np.float64(38.67040351592004) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.959399871499045) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(38.67040351592004) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:7755 - timing_s/gen:44.32738490961492 - timing_s/reward:0.05780384503304958 - timing_s/old_log_prob:2.531794674694538 - timing_s/adv:0.12005887180566788 - timing_s/update_actor:10.524201342836022 - timing_s/step:57.64935720153153 - timing_s/stop_profile:0.00011799484491348267 - timing_per_token_ms/gen:0.5715237868697127 - timing_per_token_ms/update_actor:0.06442967812001679 - timing_per_token_ms/adv:0.000735006316765035 - perf/total_num_tokens:163344 - perf/time_per_step:57.64935720153153 - perf/throughput:354.17567499707644 (TaskRunner pid=2099163) Training Progress: 63%|██████▎ | 63/100 [52:06<29:57, 48.59s/it] (TaskRunner pid=2099163) step:64 - global_seqlen/min:16789 - global_seqlen/max:29163 - global_seqlen/minmax_diff:12374 - global_seqlen/balanced_min:21715 - global_seqlen/balanced_max:21735 - global_seqlen/mean:21731.25 - actor/entropy:0.2434695065021515 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3808082342147827 - training/rollout_probs_diff_mean:0.004839340224862099 - training/rollout_probs_diff_std:0.012994172982871532 - training/rollout_actor_probs_pearson_corr:0.998428463935852 - rollout_corr/rollout_is_mean:1.0002532005310059 - rollout_corr/rollout_is_ratio_fraction_high:3.917164940503426e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.528608355438337e-05 - rollout_corr/rollout_is_max:3.4824414253234863 - rollout_corr/rollout_is_min:0.37827831506729126 - rollout_corr/rollout_is_std:0.03948592394590378 - rollout_corr/rollout_is_eff_sample_size:0.9984440018610561 - rollout_corr/rollout_is_seq_mean:1.000284194946289 - rollout_corr/rollout_is_seq_std:0.002908356487751007 - rollout_corr/rollout_is_seq_max:1.011995553970337 - rollout_corr/rollout_is_seq_min:0.9921226501464844 - rollout_corr/rollout_is_seq_max_deviation:0.011995553970336914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3239814243279397) - rollout_corr/training_log_ppl:np.float64(0.27563152175571304) - rollout_corr/kl:np.float64(0.0009577943357328422) - rollout_corr/k3_kl:np.float64(0.001276045324914321) - rollout_corr/rollout_ppl:np.float64(1.3226902424357831) - rollout_corr/rollout_log_ppl:np.float64(0.27467372607497964) - rollout_corr/log_ppl_diff:np.float64(0.0009577956807333976) - rollout_corr/log_ppl_abs_diff:np.float64(0.002473829168593511) - rollout_corr/log_ppl_diff_max:np.float64(0.010276317596435547) - rollout_corr/log_ppl_diff_min:np.float64(-0.009252876043319702) - rollout_corr/ppl_ratio:np.float64(1.0009632629808038) - rollout_corr/chi2_token:np.float64(0.0031634781043976545) - rollout_corr/chi2_seq:np.float64(1.2617831665556878) - actor/pg_loss:np.float64(0.0002826264826580882) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002043358704668208) - actor/ppo_kl:np.float64(0.000321553345604908) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6392286270856857) - perf/mfu/actor:np.float64(0.06297104741900578) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.78218460083008) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:4 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009126994758844376 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009126994758844376 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.1640625 - response_length/max:1171.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.1640625 - response_length_non_aborted/max:1171.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.9375 - prompt_length/max:499.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013485923409461975 - timing_s/agent_loop/generate_sequences/min:np.float64(1.014983855187893) - timing_s/agent_loop/generate_sequences/max:np.float64(7.733801892027259) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1143013205582974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.733801892027259) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1171 - timing_s/gen:13.659647185355425 - timing_s/reward:0.06105978973209858 - timing_s/old_log_prob:2.402337344363332 - timing_s/adv:0.11804847605526447 - timing_s/update_actor:9.979404045268893 - timing_s/step:26.307940509170294 - timing_s/stop_profile:0.00010655634105205536 - timing_per_token_ms/gen:0.17835697366823472 - timing_per_token_ms/update_actor:0.0574023816236347 - timing_per_token_ms/adv:0.000679024883838162 - perf/total_num_tokens:173850 - perf/time_per_step:26.307940509170294 - perf/throughput:826.0338734012655 (TaskRunner pid=2099163) Training Progress: 64%|██████▍ | 64/100 [52:33<25:09, 41.92s/it] (TaskRunner pid=2099163) step:65 - global_seqlen/min:14100 - global_seqlen/max:21693 - global_seqlen/minmax_diff:7593 - global_seqlen/balanced_min:18213 - global_seqlen/balanced_max:18230 - global_seqlen/mean:18224.5 - actor/entropy:0.25228142738342285 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.299439013004303 - training/rollout_probs_diff_mean:0.004961303435266018 - training/rollout_probs_diff_std:0.012695282697677612 - training/rollout_actor_probs_pearson_corr:0.9985328316688538 - rollout_corr/rollout_is_mean:1.0000190734863281 - rollout_corr/rollout_is_ratio_fraction_high:1.530081317469012e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.060162634938024e-05 - rollout_corr/rollout_is_max:2.15134859085083 - rollout_corr/rollout_is_min:0.4842594861984253 - rollout_corr/rollout_is_std:0.03996157646179199 - rollout_corr/rollout_is_eff_sample_size:0.9984056184691176 - rollout_corr/rollout_is_seq_mean:1.0000884532928467 - rollout_corr/rollout_is_seq_std:0.003020690055564046 - rollout_corr/rollout_is_seq_max:1.0103381872177124 - rollout_corr/rollout_is_seq_min:0.99142986536026 - rollout_corr/rollout_is_seq_max_deviation:0.010338187217712402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.337971543893218) - rollout_corr/training_log_ppl:np.float64(0.285963032671134) - rollout_corr/kl:np.float64(0.0009719172556970079) - rollout_corr/k3_kl:np.float64(0.0011187582267666585) - rollout_corr/rollout_ppl:np.float64(1.3366436455398798) - rollout_corr/rollout_log_ppl:np.float64(0.28499111408018507) - rollout_corr/log_ppl_diff:np.float64(0.000971918590948917) - rollout_corr/log_ppl_abs_diff:np.float64(0.002662504601175897) - rollout_corr/log_ppl_diff_max:np.float64(0.010240867733955383) - rollout_corr/log_ppl_diff_min:np.float64(-0.011126220226287842) - rollout_corr/ppl_ratio:np.float64(1.0009777853265405) - rollout_corr/chi2_token:np.float64(0.0026106988079845905) - rollout_corr/chi2_seq:np.float64(0.5336100137792528) - actor/pg_loss:np.float64(-0.00011365010868757963) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015210017868412251) - actor/ppo_kl:np.float64(0.0001487335989569516) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5744654163718224) - perf/mfu/actor:np.float64(0.05192652203466894) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.78650283813477) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:4 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003016172908246517 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003016172908246517 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:255.296875 - response_length/max:1081.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:255.296875 - response_length_non_aborted/max:1081.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.21875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000135716050863266 - timing_s/agent_loop/generate_sequences/min:np.float64(0.946566354483366) - timing_s/agent_loop/generate_sequences/max:np.float64(7.011935571208596) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.608640009377268) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.011935571208596) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1081 - timing_s/gen:12.771047074347734 - timing_s/reward:0.055413682013750076 - timing_s/old_log_prob:2.3575429376214743 - timing_s/adv:0.11941352300345898 - timing_s/update_actor:9.974061073735356 - timing_s/step:25.366581419482827 - timing_s/stop_profile:0.00011139549314975739 - timing_per_token_ms/gen:0.19540741591204686 - timing_per_token_ms/update_actor:0.06841107488364123 - timing_per_token_ms/adv:0.0008190452618964785 - perf/total_num_tokens:145796 - perf/time_per_step:25.366581419482827 - perf/throughput:718.4452527766574 (TaskRunner pid=2099163) Training Progress: 65%|██████▌ | 65/100 [52:58<21:34, 36.98s/it] (TaskRunner pid=2099163) step:66 - global_seqlen/min:13080 - global_seqlen/max:22615 - global_seqlen/minmax_diff:9535 - global_seqlen/balanced_min:17453 - global_seqlen/balanced_max:17514 - global_seqlen/mean:17466.5 - actor/entropy:0.27698102593421936 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2931666672229767 - training/rollout_probs_diff_mean:0.005428791046142578 - training/rollout_probs_diff_std:0.013127138838171959 - training/rollout_actor_probs_pearson_corr:0.998500406742096 - rollout_corr/rollout_is_mean:0.9999055862426758 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.560062446515076e-05 - rollout_corr/rollout_is_max:1.9062473773956299 - rollout_corr/rollout_is_min:0.4490605890750885 - rollout_corr/rollout_is_std:0.04169447347521782 - rollout_corr/rollout_is_eff_sample_size:0.9982642314339047 - rollout_corr/rollout_is_seq_mean:0.999854326248169 - rollout_corr/rollout_is_seq_std:0.003016769653186202 - rollout_corr/rollout_is_seq_max:1.0081275701522827 - rollout_corr/rollout_is_seq_min:0.9917121529579163 - rollout_corr/rollout_is_seq_max_deviation:0.00828784704208374 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.361108566634357) - rollout_corr/training_log_ppl:np.float64(0.3038711922999937) - rollout_corr/kl:np.float64(0.0010244684705327245) - rollout_corr/k3_kl:np.float64(0.001206662257914104) - rollout_corr/rollout_ppl:np.float64(1.3596958024427295) - rollout_corr/rollout_log_ppl:np.float64(0.3028467216063291) - rollout_corr/log_ppl_diff:np.float64(0.0010244706936646253) - rollout_corr/log_ppl_abs_diff:np.float64(0.002828612778102979) - rollout_corr/log_ppl_diff_max:np.float64(0.010544925928115845) - rollout_corr/log_ppl_diff_min:np.float64(-0.013002634048461914) - rollout_corr/ppl_ratio:np.float64(1.001031328458339) - rollout_corr/chi2_token:np.float64(0.002860499545931816) - rollout_corr/chi2_seq:np.float64(0.9732876946218312) - actor/pg_loss:np.float64(2.8914306312799454e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017755173166733584) - actor/ppo_kl:np.float64(-8.335111564150566e-05) - actor/pg_clipfrac_lower:np.float64(1.0643732821336016e-05) - actor/grad_norm:np.float64(0.679240420460701) - perf/mfu/actor:np.float64(0.05005040663095298) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.89555358886719) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:4 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004099064040929079 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004099064040929079 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:250.390625 - response_length/max:1240.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:250.390625 - response_length_non_aborted/max:1240.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.4375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001604091376066208 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0681240409612656) - timing_s/agent_loop/generate_sequences/max:np.float64(7.348783219233155) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5614106698412797) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.348783219233155) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:1240 - timing_s/gen:13.083483047783375 - timing_s/reward:0.05359955504536629 - timing_s/old_log_prob:2.406119966879487 - timing_s/adv:0.12794314324855804 - timing_s/update_actor:9.988153088837862 - timing_s/step:25.744537329301238 - timing_s/stop_profile:0.00010885298252105713 - timing_per_token_ms/gen:0.20411049996541927 - timing_per_token_ms/update_actor:0.07148078528066486 - timing_per_token_ms/adv:0.0009156323766106407 - perf/total_num_tokens:139732 - perf/time_per_step:25.744537329301238 - perf/throughput:678.4546087033555 (TaskRunner pid=2099163) Training Progress: 66%|██████▌ | 66/100 [53:24<19:03, 33.62s/it] (TaskRunner pid=2099163) step:67 - global_seqlen/min:16776 - global_seqlen/max:26215 - global_seqlen/minmax_diff:9439 - global_seqlen/balanced_min:19749 - global_seqlen/balanced_max:25482 - global_seqlen/mean:20468.5 - actor/entropy:0.2276402860879898 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21511590480804443 - training/rollout_probs_diff_mean:0.004441514611244202 - training/rollout_probs_diff_std:0.012101520784199238 - training/rollout_actor_probs_pearson_corr:0.998544454574585 - rollout_corr/rollout_is_mean:0.9999508261680603 - rollout_corr/rollout_is_ratio_fraction_high:3.911342719220556e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.911342719220556e-05 - rollout_corr/rollout_is_max:2.2540721893310547 - rollout_corr/rollout_is_min:0.3879011869430542 - rollout_corr/rollout_is_std:0.03753895312547684 - rollout_corr/rollout_is_eff_sample_size:0.9985924533404948 - rollout_corr/rollout_is_seq_mean:0.9999358654022217 - rollout_corr/rollout_is_seq_std:0.002823594259098172 - rollout_corr/rollout_is_seq_max:1.0075790882110596 - rollout_corr/rollout_is_seq_min:0.9925720691680908 - rollout_corr/rollout_is_seq_max_deviation:0.00757908821105957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.324487020727247) - rollout_corr/training_log_ppl:np.float64(0.2760363431443693) - rollout_corr/kl:np.float64(0.0011812180125367888) - rollout_corr/k3_kl:np.float64(0.001190797103646446) - rollout_corr/rollout_ppl:np.float64(1.3228898402303457) - rollout_corr/rollout_log_ppl:np.float64(0.27485512448402005) - rollout_corr/log_ppl_diff:np.float64(0.0011812186603492592) - rollout_corr/log_ppl_abs_diff:np.float64(0.002600522890134016) - rollout_corr/log_ppl_diff_max:np.float64(0.013327181339263916) - rollout_corr/log_ppl_diff_min:np.float64(-0.00664597749710083) - rollout_corr/ppl_ratio:np.float64(1.0011871331371367) - rollout_corr/chi2_token:np.float64(0.002502224873751402) - rollout_corr/chi2_seq:np.float64(0.4073182353749871) - actor/pg_loss:np.float64(0.0001626985613256693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002055950665180717) - actor/ppo_kl:np.float64(0.00024570608130236593) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6885803639888763) - perf/mfu/actor:np.float64(0.055952627926162825) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.84812545776367) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:4 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.08921447396278381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.08921447396278381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.609375 - response_length/max:7123.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.609375 - response_length_non_aborted/max:7123.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:340.03125 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.429827332496643e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4895208682864904) - timing_s/agent_loop/generate_sequences/max:np.float64(35.69096236862242) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9385357215287513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(35.69096236862242) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:7123 - timing_s/gen:41.56289967894554 - timing_s/reward:0.058504676446318626 - timing_s/old_log_prob:2.5877962671220303 - timing_s/adv:0.14371454156935215 - timing_s/update_actor:10.467117831110954 - timing_s/step:54.904828580096364 - timing_s/stop_profile:5.9584155678749084e-05 - timing_per_token_ms/gen:0.5418891744321452 - timing_per_token_ms/update_actor:0.06392211099439965 - timing_per_token_ms/adv:0.000877656774857416 - perf/total_num_tokens:163748 - perf/time_per_step:54.904828580096364 - perf/throughput:372.79963400924026 (TaskRunner pid=2099163) Training Progress: 67%|██████▋ | 67/100 [54:19<22:00, 40.01s/it] (TaskRunner pid=2099163) step:68 - global_seqlen/min:12125 - global_seqlen/max:24360 - global_seqlen/minmax_diff:12235 - global_seqlen/balanced_min:16093 - global_seqlen/balanced_max:18598 - global_seqlen/mean:16412.875 - actor/entropy:0.2802455723285675 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2584646940231323 - training/rollout_probs_diff_mean:0.005321393255144358 - training/rollout_probs_diff_std:0.012953302823007107 - training/rollout_actor_probs_pearson_corr:0.9985373020172119 - rollout_corr/rollout_is_mean:0.9998360276222229 - rollout_corr/rollout_is_ratio_fraction_high:3.26120643876493e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.6202330589294434 - rollout_corr/rollout_is_min:0.5360647439956665 - rollout_corr/rollout_is_std:0.0403851680457592 - rollout_corr/rollout_is_eff_sample_size:0.998371040293158 - rollout_corr/rollout_is_seq_mean:0.9997125864028931 - rollout_corr/rollout_is_seq_std:0.003077426226809621 - rollout_corr/rollout_is_seq_max:1.0127679109573364 - rollout_corr/rollout_is_seq_min:0.9891027808189392 - rollout_corr/rollout_is_seq_max_deviation:0.012767910957336426 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.368599426932633) - rollout_corr/training_log_ppl:np.float64(0.30979057060176274) - rollout_corr/kl:np.float64(0.0015277589002717917) - rollout_corr/k3_kl:np.float64(0.0013763565973192726) - rollout_corr/rollout_ppl:np.float64(1.3664680924266577) - rollout_corr/rollout_log_ppl:np.float64(0.3082628087795456) - rollout_corr/log_ppl_diff:np.float64(0.001527761822217144) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030798379593761638) - rollout_corr/log_ppl_diff_max:np.float64(0.01730947196483612) - rollout_corr/log_ppl_diff_min:np.float64(-0.008380800485610962) - rollout_corr/ppl_ratio:np.float64(1.0015360116958618) - rollout_corr/chi2_token:np.float64(0.002355032367631793) - rollout_corr/chi2_seq:np.float64(0.5932061832863837) - actor/pg_loss:np.float64(0.00043787446338683367) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020217109231452923) - actor/ppo_kl:np.float64(0.00030286650112287816) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6202314719557762) - perf/mfu/actor:np.float64(0.04649614156988041) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.8221549987793) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:4 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.05651262775063515 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.05651262775063515 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:239.55859375 - response_length/max:3687.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:239.55859375 - response_length_non_aborted/max:3687.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.34375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.628541111946106e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.119384104385972) - timing_s/agent_loop/generate_sequences/max:np.float64(18.886547598987818) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.397038237679226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.886547598987818) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:3687 - timing_s/gen:24.744802374392748 - timing_s/reward:0.056385381147265434 - timing_s/old_log_prob:2.4465242717415094 - timing_s/adv:0.11799081787467003 - timing_s/update_actor:10.066473811864853 - timing_s/step:37.518324200063944 - timing_s/stop_profile:5.867891013622284e-05 - timing_per_token_ms/gen:0.4034895294795563 - timing_per_token_ms/update_actor:0.0766659848736499 - timing_per_token_ms/adv:0.0008986147907867301 - perf/total_num_tokens:131303 - perf/time_per_step:37.518324200063944 - perf/throughput:437.4629024601271 (TaskRunner pid=2099163) Training Progress: 68%|██████▊ | 68/100 [54:56<20:56, 39.27s/it] (TaskRunner pid=2099163) step:69 - global_seqlen/min:14077 - global_seqlen/max:22439 - global_seqlen/minmax_diff:8362 - global_seqlen/balanced_min:18403 - global_seqlen/balanced_max:18420 - global_seqlen/mean:18416.125 - actor/entropy:0.2733437716960907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3037760257720947 - training/rollout_probs_diff_mean:0.005474385805428028 - training/rollout_probs_diff_std:0.013468709774315357 - training/rollout_actor_probs_pearson_corr:0.9984379410743713 - rollout_corr/rollout_is_mean:0.9997246861457825 - rollout_corr/rollout_is_ratio_fraction_high:1.5642353901057504e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.692705988418311e-05 - rollout_corr/rollout_is_max:2.0041916370391846 - rollout_corr/rollout_is_min:0.16430728137493134 - rollout_corr/rollout_is_std:0.04152041673660278 - rollout_corr/rollout_is_eff_sample_size:0.9982780119380065 - rollout_corr/rollout_is_seq_mean:0.9996422529220581 - rollout_corr/rollout_is_seq_std:0.0031316319946199656 - rollout_corr/rollout_is_seq_max:1.0090826749801636 - rollout_corr/rollout_is_seq_min:0.9910612106323242 - rollout_corr/rollout_is_seq_max_deviation:0.009082674980163574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3545742672868073) - rollout_corr/training_log_ppl:np.float64(0.2993480234290473) - rollout_corr/kl:np.float64(0.0016278082214427059) - rollout_corr/k3_kl:np.float64(0.0013303804649922313) - rollout_corr/rollout_ppl:np.float64(1.3523246650584042) - rollout_corr/rollout_log_ppl:np.float64(0.29772021461394615) - rollout_corr/log_ppl_diff:np.float64(0.0016278088151011616) - rollout_corr/log_ppl_abs_diff:np.float64(0.003158511739457026) - rollout_corr/log_ppl_diff_max:np.float64(0.016750067472457886) - rollout_corr/log_ppl_diff_min:np.float64(-0.006825670599937439) - rollout_corr/ppl_ratio:np.float64(1.0016362892929465) - rollout_corr/chi2_token:np.float64(0.002009368734434247) - rollout_corr/chi2_seq:np.float64(1.0319822325836867) - actor/pg_loss:np.float64(0.0003217571647837758) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0027308197345519147) - actor/ppo_kl:np.float64(0.00029400911517996065) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7714866697788239) - perf/mfu/actor:np.float64(0.053972941498170224) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.87010192871094) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:4 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012910807505249977 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012910807505249977 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:249.72265625 - response_length/max:1039.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:249.72265625 - response_length_non_aborted/max:1039.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:325.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.7713518142700195e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6453578509390354) - timing_s/agent_loop/generate_sequences/max:np.float64(6.574762849137187) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3507946640675073) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.574762849137187) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1039 - timing_s/gen:12.447308329865336 - timing_s/reward:0.05588654242455959 - timing_s/old_log_prob:2.361416006460786 - timing_s/adv:0.11917807720601559 - timing_s/update_actor:9.651425044983625 - timing_s/step:24.725356981158257 - timing_s/stop_profile:0.00011798366904258728 - timing_per_token_ms/gen:0.19470519372843836 - timing_per_token_ms/update_actor:0.0655093365527739 - timing_per_token_ms/adv:0.0008089247684163715 - perf/total_num_tokens:147329 - perf/time_per_step:24.725356981158257 - perf/throughput:744.8274665572613 (TaskRunner pid=2099163) Training Progress: 69%|██████▉ | 69/100 [55:21<18:02, 34.92s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a protein sequence can be determined using computational tools like I-TASSER, which predicts the structure and calculates stability scores. The provided sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR' would need to be input into such a tool to obtain the folding stability score. Based on known scores from similar sequences and computational predictions, the folding stability score for this specific sequence is approximately 1.69. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) B (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 1.0 (TaskRunner pid=2099163) [acc] 1.0 (TaskRunner pid=2099163) [pred] B (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2099163) step:70 - global_seqlen/min:12908 - global_seqlen/max:23884 - global_seqlen/minmax_diff:10976 - global_seqlen/balanced_min:18177 - global_seqlen/balanced_max:18200 - global_seqlen/mean:18194.5 - actor/entropy:0.27147170901298523 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2765524983406067 - training/rollout_probs_diff_mean:0.005476649384945631 - training/rollout_probs_diff_std:0.013665086589753628 - training/rollout_actor_probs_pearson_corr:0.9983801245689392 - rollout_corr/rollout_is_mean:0.9999741315841675 - rollout_corr/rollout_is_ratio_fraction_high:1.597342088643927e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.584052168065682e-05 - rollout_corr/rollout_is_max:2.8377625942230225 - rollout_corr/rollout_is_min:0.39244869351387024 - rollout_corr/rollout_is_std:0.042557571083307266 - rollout_corr/rollout_is_eff_sample_size:0.9981917712502576 - rollout_corr/rollout_is_seq_mean:0.9999601244926453 - rollout_corr/rollout_is_seq_std:0.003185849403962493 - rollout_corr/rollout_is_seq_max:1.0091570615768433 - rollout_corr/rollout_is_seq_min:0.9889717698097229 - rollout_corr/rollout_is_seq_max_deviation:0.0110282301902771 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3532036440446973) - rollout_corr/training_log_ppl:np.float64(0.297806289701839) - rollout_corr/kl:np.float64(0.001792206864962509) - rollout_corr/k3_kl:np.float64(0.0015404767690370136) - rollout_corr/rollout_ppl:np.float64(1.3506689360365272) - rollout_corr/rollout_log_ppl:np.float64(0.2960140811483143) - rollout_corr/log_ppl_diff:np.float64(0.0017922085535246879) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032309792877640575) - rollout_corr/log_ppl_diff_max:np.float64(0.019662469625473022) - rollout_corr/log_ppl_diff_min:np.float64(-0.009732484817504883) - rollout_corr/ppl_ratio:np.float64(1.0018020996358246) - rollout_corr/chi2_token:np.float64(0.0025641866959631443) - rollout_corr/chi2_seq:np.float64(0.30121213220991194) - actor/pg_loss:np.float64(0.00043956295121461153) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002650087731126405) - actor/ppo_kl:np.float64(0.0007622978668138103) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8055207878351212) - perf/mfu/actor:np.float64(0.05315399906233151) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.96969223022461) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.395) - val-aux/sciknoweval/reward/std@16:np.float64(0.25608552674596197) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.50316) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2167468179495711) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.28751999999999994) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21605136415229434) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.39537999999999995) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2547211208942672) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.58856) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.15622607807957983) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.19984) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1565882935853783) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.40007999999999994) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.209600774094245) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6486799999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10171288109893938) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.1405) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09636893444194905) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4068) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.15999762458359668) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6872) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.060830405323077257) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.10672000000000001) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.050487971198454974) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.41072) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11352260117902001) - val-aux/sciknoweval/score/mean@16:np.float64(0.395) - val-aux/sciknoweval/score/std@16:np.float64(0.25608552674596197) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.50316) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2167468179495711) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.28751999999999994) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21605136415229434) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.39537999999999995) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2547211208942672) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.58856) - val-aux/sciknoweval/score/best@4/std:np.float64(0.15622607807957983) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.19984) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1565882935853783) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.40007999999999994) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.209600774094245) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6486799999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10171288109893938) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.1405) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09636893444194905) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4068) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.15999762458359668) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6872) - val-aux/sciknoweval/score/best@16/std:np.float64(0.060830405323077257) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.10672000000000001) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.050487971198454974) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.41072) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11352260117902001) - val-core/sciknoweval/acc/mean@16:np.float64(0.395) - val-aux/sciknoweval/acc/std@16:np.float64(0.25608552674596197) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.50316) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2167468179495711) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.28751999999999994) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21605136415229434) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.39537999999999995) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2547211208942672) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.58856) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.15622607807957983) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.19984) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1565882935853783) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.40007999999999994) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.209600774094245) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6486799999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10171288109893938) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.1405) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09636893444194905) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4068) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.15999762458359668) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6872) - val-core/sciknoweval/acc/best@16/std:np.float64(0.060830405323077257) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.10672000000000001) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.050487971198454974) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.41072) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11352260117902001) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.004841229182759271) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99994) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0010938007131100255) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.99752) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.006591631057636644) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9986799999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.004965641952456903) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9954599999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008377851753283773) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9996600000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0025854206620973697) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9919) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.009817840903172144) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0012623787070447602) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.98744) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.00966676781556276) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711644) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:4 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.019984744489192963 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.019984744489192963 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:244.546875 - response_length/max:1104.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:244.546875 - response_length_non_aborted/max:1104.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:324.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.822472929954529e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0265482608228922) - timing_s/agent_loop/generate_sequences/max:np.float64(6.8141403291374445) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4398168486659415) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.8141403291374445) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1104 - timing_s/gen:12.706574181094766 - timing_s/reward:0.055348560214042664 - timing_s/old_log_prob:2.4000668544322252 - timing_s/adv:0.11832345649600029 - timing_s/update_actor:9.746209550648928 - timing_s/step:25.11487785167992 - timing_s/testing:66.92163533717394 - timing_s/save_checkpoint:6.685545779764652 - timing_s/stop_profile:0.000119037926197052 - timing_per_token_ms/gen:0.20296744906227662 - timing_per_token_ms/update_actor:0.066958487115948 - timing_per_token_ms/adv:0.0008129067609442433 - perf/total_num_tokens:145556 - perf/time_per_step:25.11487785167992 - perf/throughput:724.4510647215025 (TaskRunner pid=2099163) Training Progress: 70%|███████ | 70/100 [57:00<27:02, 54.08s/it] (TaskRunner pid=2099163) step:71 - global_seqlen/min:11983 - global_seqlen/max:23722 - global_seqlen/minmax_diff:11739 - global_seqlen/balanced_min:18369 - global_seqlen/balanced_max:18382 - global_seqlen/mean:18378.375 - actor/entropy:0.26438087224960327 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49391263723373413 - training/rollout_probs_diff_mean:0.0053352005779743195 - training/rollout_probs_diff_std:0.013760312460362911 - training/rollout_actor_probs_pearson_corr:0.9983255863189697 - rollout_corr/rollout_is_mean:1.0000841617584229 - rollout_corr/rollout_is_ratio_fraction_high:1.5517589417868294e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.655276643461548e-05 - rollout_corr/rollout_is_max:2.3463339805603027 - rollout_corr/rollout_is_min:0.04733983799815178 - rollout_corr/rollout_is_std:0.04175732657313347 - rollout_corr/rollout_is_eff_sample_size:0.9982597172135673 - rollout_corr/rollout_is_seq_mean:1.000182032585144 - rollout_corr/rollout_is_seq_std:0.0031579334754496813 - rollout_corr/rollout_is_seq_max:1.0100078582763672 - rollout_corr/rollout_is_seq_min:0.9906702637672424 - rollout_corr/rollout_is_seq_max_deviation:0.010007858276367188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3460410116240382) - rollout_corr/training_log_ppl:np.float64(0.2919461800775025) - rollout_corr/kl:np.float64(0.0015395511177251908) - rollout_corr/k3_kl:np.float64(0.0017241420433720123) - rollout_corr/rollout_ppl:np.float64(1.3439419087953866) - rollout_corr/rollout_log_ppl:np.float64(0.29040662810439244) - rollout_corr/log_ppl_diff:np.float64(0.0015395519731100649) - rollout_corr/log_ppl_abs_diff:np.float64(0.003382161114132032) - rollout_corr/log_ppl_diff_max:np.float64(0.01569351553916931) - rollout_corr/log_ppl_diff_min:np.float64(-0.013844609260559082) - rollout_corr/ppl_ratio:np.float64(1.0015493920072913) - rollout_corr/chi2_token:np.float64(0.004469351610168815) - rollout_corr/chi2_seq:np.float64(0.9466164640616626) - actor/pg_loss:np.float64(0.0009243421955034137) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002824900551331666) - actor/ppo_kl:np.float64(0.0007459541272538672) - actor/pg_clipfrac_lower:np.float64(6.143417522253003e-05) - actor/grad_norm:np.float64(0.7537118345499039) - perf/mfu/actor:np.float64(0.05423491984814467) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.76616668701172) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:5 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003462362103164196 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003462362103164196 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:251.73046875 - response_length/max:1004.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:251.73046875 - response_length_non_aborted/max:1004.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.59375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002311021089553833 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0665221828967333) - timing_s/agent_loop/generate_sequences/max:np.float64(6.538059942424297) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5836986561625963) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.538059942424297) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1004 - timing_s/gen:12.774788860231638 - timing_s/reward:0.05202013999223709 - timing_s/old_log_prob:2.479814240708947 - timing_s/adv:0.12360932119190693 - timing_s/update_actor:9.744195263832808 - timing_s/step:25.277720926329494 - timing_s/stop_profile:0.00012053363025188446 - timing_per_token_ms/gen:0.19823392548813118 - timing_per_token_ms/update_actor:0.06627486967586095 - timing_per_token_ms/adv:0.0008407253170635797 - perf/total_num_tokens:147027 - perf/time_per_step:25.277720926329494 - perf/throughput:727.0582286101958 (TaskRunner pid=2099163) Training Progress: 71%|███████ | 71/100 [57:26<22:02, 45.59s/it] (TaskRunner pid=2099163) step:72 - global_seqlen/min:13664 - global_seqlen/max:22468 - global_seqlen/minmax_diff:8804 - global_seqlen/balanced_min:18001 - global_seqlen/balanced_max:18006 - global_seqlen/mean:18004.25 - actor/entropy:0.2748372554779053 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519516587257385 - training/rollout_probs_diff_mean:0.005624288693070412 - training/rollout_probs_diff_std:0.013938440941274166 - training/rollout_actor_probs_pearson_corr:0.9983092546463013 - rollout_corr/rollout_is_mean:1.0002027750015259 - rollout_corr/rollout_is_ratio_fraction_high:1.6573303582845256e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.314660716569051e-05 - rollout_corr/rollout_is_max:3.3772945404052734 - rollout_corr/rollout_is_min:0.4679282307624817 - rollout_corr/rollout_is_std:0.04276713728904724 - rollout_corr/rollout_is_eff_sample_size:0.9981750237537251 - rollout_corr/rollout_is_seq_mean:1.0002542734146118 - rollout_corr/rollout_is_seq_std:0.0032979086972773075 - rollout_corr/rollout_is_seq_max:1.013193964958191 - rollout_corr/rollout_is_seq_min:0.9926396012306213 - rollout_corr/rollout_is_seq_max_deviation:0.013193964958190918 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3490465520881116) - rollout_corr/training_log_ppl:np.float64(0.29478535568341613) - rollout_corr/kl:np.float64(0.001085285884805387) - rollout_corr/k3_kl:np.float64(0.0015339718757445553) - rollout_corr/rollout_ppl:np.float64(1.3474991344846785) - rollout_corr/rollout_log_ppl:np.float64(0.29370006851968355) - rollout_corr/log_ppl_diff:np.float64(0.0010852871637325734) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028866918582934886) - rollout_corr/log_ppl_diff_max:np.float64(0.020227819681167603) - rollout_corr/log_ppl_diff_min:np.float64(-0.012600928544998169) - rollout_corr/ppl_ratio:np.float64(1.0010935172904283) - rollout_corr/chi2_token:np.float64(0.003948581404983997) - rollout_corr/chi2_seq:np.float64(0.5234741037711501) - actor/pg_loss:np.float64(0.00017087801825255156) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0027429519377619727) - actor/ppo_kl:np.float64(0.0003700984995536061) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.679483950138092) - perf/mfu/actor:np.float64(0.05314325335515786) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(105.71037673950195) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:5 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009772034361958504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009772034361958504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:235.6953125 - response_length/max:723.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:235.6953125 - response_length_non_aborted/max:723.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:326.9375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.920448064804077e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9485215358436108) - timing_s/agent_loop/generate_sequences/max:np.float64(5.383510269224644) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.490466148345149) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.383510269224644) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:723 - timing_s/gen:11.147572057321668 - timing_s/reward:0.054647378623485565 - timing_s/old_log_prob:2.295195208862424 - timing_s/adv:0.11946884915232658 - timing_s/update_actor:9.803522318601608 - timing_s/step:23.509083898738027 - timing_s/stop_profile:0.00010769441723823547 - timing_per_token_ms/gen:0.18475209747293028 - timing_per_token_ms/update_actor:0.06806394544761382 - timing_per_token_ms/adv:0.0008294489436683463 - perf/total_num_tokens:144034 - perf/time_per_step:23.509083898738027 - perf/throughput:765.8422624016614 (TaskRunner pid=2099163) Training Progress: 72%|███████▏ | 72/100 [57:49<18:11, 38.98s/it] (TaskRunner pid=2099163) step:73 - global_seqlen/min:12492 - global_seqlen/max:19109 - global_seqlen/minmax_diff:6617 - global_seqlen/balanced_min:15295 - global_seqlen/balanced_max:16032 - global_seqlen/mean:15388.75 - actor/entropy:0.2661946415901184 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37658703327178955 - training/rollout_probs_diff_mean:0.005457909312099218 - training/rollout_probs_diff_std:0.013575312681496143 - training/rollout_actor_probs_pearson_corr:0.9983363151550293 - rollout_corr/rollout_is_mean:1.0002198219299316 - rollout_corr/rollout_is_ratio_fraction_high:1.8412814824841917e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1717958450317383 - rollout_corr/rollout_is_min:0.5105080008506775 - rollout_corr/rollout_is_std:0.041652996093034744 - rollout_corr/rollout_is_eff_sample_size:0.9982688644919676 - rollout_corr/rollout_is_seq_mean:1.000148057937622 - rollout_corr/rollout_is_seq_std:0.0032269360963255167 - rollout_corr/rollout_is_seq_max:1.0114909410476685 - rollout_corr/rollout_is_seq_min:0.9871060252189636 - rollout_corr/rollout_is_seq_max_deviation:0.012893974781036377 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3228384959511459) - rollout_corr/training_log_ppl:np.float64(0.2764760266873054) - rollout_corr/kl:np.float64(0.0013183017936526653) - rollout_corr/k3_kl:np.float64(0.001363425177373756) - rollout_corr/rollout_ppl:np.float64(1.321010212879628) - rollout_corr/rollout_log_ppl:np.float64(0.2751577235030709) - rollout_corr/log_ppl_diff:np.float64(0.0013183031842345372) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031554124579997733) - rollout_corr/log_ppl_diff_max:np.float64(0.014645874500274658) - rollout_corr/log_ppl_diff_min:np.float64(-0.012716293334960938) - rollout_corr/ppl_ratio:np.float64(1.001326964236796) - rollout_corr/chi2_token:np.float64(0.002702318597584963) - rollout_corr/chi2_seq:np.float64(0.5265928467269987) - actor/pg_loss:np.float64(0.00030526239424943924) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001643506417167373) - actor/ppo_kl:np.float64(0.0005654427488943625) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5535854026675224) - perf/mfu/actor:np.float64(0.044940075380136224) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.07952976226807) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:5 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012893574312329292 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.012893574312329292 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:212.1484375 - response_length/max:1646.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:212.1484375 - response_length_non_aborted/max:1646.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001221541315317154 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7583393473178148) - timing_s/agent_loop/generate_sequences/max:np.float64(9.147600388154387) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.0318894187221304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.147600388154387) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1646 - timing_s/gen:15.125937445089221 - timing_s/reward:0.051669539883732796 - timing_s/old_log_prob:2.325926223769784 - timing_s/adv:0.12074156478047371 - timing_s/update_actor:9.70467483624816 - timing_s/step:27.41567281074822 - timing_s/stop_profile:5.909241735935211e-05 - timing_per_token_ms/gen:0.27851109271016794 - timing_per_token_ms/update_actor:0.07882929767076728 - timing_per_token_ms/adv:0.0009807616341521706 - perf/total_num_tokens:123110 - perf/time_per_step:27.41567281074822 - perf/throughput:561.312140914043 (TaskRunner pid=2099163) Training Progress: 73%|███████▎ | 73/100 [58:17<15:59, 35.52s/it] (TaskRunner pid=2099163) step:74 - global_seqlen/min:13372 - global_seqlen/max:23732 - global_seqlen/minmax_diff:10360 - global_seqlen/balanced_min:17453 - global_seqlen/balanced_max:17463 - global_seqlen/mean:17459.25 - actor/entropy:0.2593246102333069 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3182520568370819 - training/rollout_probs_diff_mean:0.005462802015244961 - training/rollout_probs_diff_std:0.013881351798772812 - training/rollout_actor_probs_pearson_corr:0.9982698559761047 - rollout_corr/rollout_is_mean:0.9997365474700928 - rollout_corr/rollout_is_ratio_fraction_high:3.281162935309112e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.281162935309112e-05 - rollout_corr/rollout_is_max:3.5407092571258545 - rollout_corr/rollout_is_min:0.41278213262557983 - rollout_corr/rollout_is_std:0.04266597703099251 - rollout_corr/rollout_is_eff_sample_size:0.9981817939506096 - rollout_corr/rollout_is_seq_mean:0.9995617866516113 - rollout_corr/rollout_is_seq_std:0.0030475708190351725 - rollout_corr/rollout_is_seq_max:1.0164185762405396 - rollout_corr/rollout_is_seq_min:0.9908937811851501 - rollout_corr/rollout_is_seq_max_deviation:0.01641857624053955 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3252906440757215) - rollout_corr/training_log_ppl:np.float64(0.27802602780866437) - rollout_corr/kl:np.float64(0.0018082092363584934) - rollout_corr/k3_kl:np.float64(0.001329023798803064) - rollout_corr/rollout_ppl:np.float64(1.3228671373799443) - rollout_corr/rollout_log_ppl:np.float64(0.27621781785273924) - rollout_corr/log_ppl_diff:np.float64(0.001808209955925122) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030505233735311776) - rollout_corr/log_ppl_diff_max:np.float64(0.0164526104927063) - rollout_corr/log_ppl_diff_min:np.float64(-0.006425350904464722) - rollout_corr/ppl_ratio:np.float64(1.0018170916009694) - rollout_corr/chi2_token:np.float64(0.001663966104388237) - rollout_corr/chi2_seq:np.float64(0.45517092291265726) - actor/pg_loss:np.float64(0.00037874444387853146) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015740662133794103) - actor/ppo_kl:np.float64(0.00035873608290160064) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5778901502490044) - perf/mfu/actor:np.float64(0.05122353731438829) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.05221939086914) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:5 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000875660334713757 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000875660334713757 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:238.1015625 - response_length/max:964.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:238.1015625 - response_length_non_aborted/max:964.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.5 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.1486695408821106e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1809429358690977) - timing_s/agent_loop/generate_sequences/max:np.float64(6.445059455931187) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5263324798143003) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.445059455931187) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:964 - timing_s/gen:12.528105229139328 - timing_s/reward:0.05381600931286812 - timing_s/old_log_prob:2.4174425285309553 - timing_s/adv:0.12314742244780064 - timing_s/update_actor:9.780229168012738 - timing_s/step:24.991026105359197 - timing_s/stop_profile:0.00012196414172649384 - timing_per_token_ms/gen:0.20553376692488315 - timing_per_token_ms/update_actor:0.07002183060564413 - timing_per_token_ms/adv:0.0008816774950799765 - perf/total_num_tokens:139674 - perf/time_per_step:24.991026105359197 - perf/throughput:698.6207739687788 (TaskRunner pid=2099163) Training Progress: 74%|███████▍ | 74/100 [58:42<14:01, 32.37s/it] (TaskRunner pid=2099163) step:75 - global_seqlen/min:13280 - global_seqlen/max:25094 - global_seqlen/minmax_diff:11814 - global_seqlen/balanced_min:19571 - global_seqlen/balanced_max:19676 - global_seqlen/mean:19596.625 - actor/entropy:0.24914591014385223 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3969437777996063 - training/rollout_probs_diff_mean:0.005310408771038055 - training/rollout_probs_diff_std:0.013804785907268524 - training/rollout_actor_probs_pearson_corr:0.998227596282959 - rollout_corr/rollout_is_mean:1.0003116130828857 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.204922439996153e-05 - rollout_corr/rollout_is_max:1.7981332540512085 - rollout_corr/rollout_is_min:0.2928912937641144 - rollout_corr/rollout_is_std:0.042179133743047714 - rollout_corr/rollout_is_eff_sample_size:0.9982251491428059 - rollout_corr/rollout_is_seq_mean:1.000361680984497 - rollout_corr/rollout_is_seq_std:0.0032836582977324724 - rollout_corr/rollout_is_seq_max:1.011388897895813 - rollout_corr/rollout_is_seq_min:0.9914733171463013 - rollout_corr/rollout_is_seq_max_deviation:0.011388897895812988 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3274900810793042) - rollout_corr/training_log_ppl:np.float64(0.2792092165327631) - rollout_corr/kl:np.float64(0.0009840833473546695) - rollout_corr/k3_kl:np.float64(0.0012072064104131641) - rollout_corr/rollout_ppl:np.float64(1.3261237805709243) - rollout_corr/rollout_log_ppl:np.float64(0.27822513245337177) - rollout_corr/log_ppl_diff:np.float64(0.000984084079391323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027362913970137015) - rollout_corr/log_ppl_diff_max:np.float64(0.01530107855796814) - rollout_corr/log_ppl_diff_min:np.float64(-0.010391294956207275) - rollout_corr/ppl_ratio:np.float64(1.0009908275678754) - rollout_corr/chi2_token:np.float64(0.0028621042147278786) - rollout_corr/chi2_seq:np.float64(0.6469766469672322) - actor/pg_loss:np.float64(-0.00014654418919235468) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010958636398754606) - actor/ppo_kl:np.float64(0.0003283484302762929) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5892626643180847) - perf/mfu/actor:np.float64(0.05761350722790884) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.95429992675781) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:5 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006997780874371529 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.006997780874371529 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:271.08203125 - response_length/max:1332.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.08203125 - response_length_non_aborted/max:1332.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:341.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.216368198394775e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2000188957899809) - timing_s/agent_loop/generate_sequences/max:np.float64(8.659884678199887) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.824555200713803) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.659884678199887) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1332 - timing_s/gen:14.400074571371078 - timing_s/reward:0.05767091363668442 - timing_s/old_log_prob:2.4206353165209293 - timing_s/adv:0.12469611130654812 - timing_s/update_actor:9.697892174124718 - timing_s/step:26.788869658485055 - timing_s/stop_profile:0.00010457262396812439 - timing_per_token_ms/gen:0.20750283976787295 - timing_per_token_ms/update_actor:0.0618594539501363 - timing_per_token_ms/adv:0.0007953927736698801 - perf/total_num_tokens:156773 - perf/time_per_step:26.788869658485055 - perf/throughput:731.5211597139188 (TaskRunner pid=2099163) Training Progress: 75%|███████▌ | 75/100 [59:09<12:47, 30.71s/it] (TaskRunner pid=2099163) step:76 - global_seqlen/min:11824 - global_seqlen/max:21898 - global_seqlen/minmax_diff:10074 - global_seqlen/balanced_min:16652 - global_seqlen/balanced_max:16677 - global_seqlen/mean:16667.125 - actor/entropy:0.25044116377830505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49410587549209595 - training/rollout_probs_diff_mean:0.005233317147940397 - training/rollout_probs_diff_std:0.013545987196266651 - training/rollout_actor_probs_pearson_corr:0.9983153343200684 - rollout_corr/rollout_is_mean:1.000156044960022 - rollout_corr/rollout_is_ratio_fraction_high:1.6923624571063556e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.461812103632838e-05 - rollout_corr/rollout_is_max:2.701498508453369 - rollout_corr/rollout_is_min:0.2689496576786041 - rollout_corr/rollout_is_std:0.0415240079164505 - rollout_corr/rollout_is_eff_sample_size:0.9982794375291114 - rollout_corr/rollout_is_seq_mean:1.000076413154602 - rollout_corr/rollout_is_seq_std:0.003434262704104185 - rollout_corr/rollout_is_seq_max:1.0094366073608398 - rollout_corr/rollout_is_seq_min:0.9889258742332458 - rollout_corr/rollout_is_seq_max_deviation:0.01107412576675415 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3120325771160424) - rollout_corr/training_log_ppl:np.float64(0.26778477303741965) - rollout_corr/kl:np.float64(0.001165033413258243) - rollout_corr/k3_kl:np.float64(0.0011289213275631482) - rollout_corr/rollout_ppl:np.float64(1.3104448192752898) - rollout_corr/rollout_log_ppl:np.float64(0.26661973919544835) - rollout_corr/log_ppl_diff:np.float64(0.001165033841971308) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029716190765611827) - rollout_corr/log_ppl_diff_max:np.float64(0.01145792007446289) - rollout_corr/log_ppl_diff_min:np.float64(-0.009343594312667847) - rollout_corr/ppl_ratio:np.float64(1.001172634307295) - rollout_corr/chi2_token:np.float64(0.002230182522907853) - rollout_corr/chi2_seq:np.float64(0.7144021939020604) - actor/pg_loss:np.float64(-2.578599378466606e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011448644404481456) - actor/ppo_kl:np.float64(0.00031501110731468884) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.568739764392376) - perf/mfu/actor:np.float64(0.04933886842194326) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.97966003417969) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:5 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007423124276101589 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.007423124276101589 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:230.81640625 - response_length/max:1078.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:230.81640625 - response_length_non_aborted/max:1078.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.03125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.485520422458649e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2769716065376997) - timing_s/agent_loop/generate_sequences/max:np.float64(6.788432452827692) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.423513734356675) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.788432452827692) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:496 - timing_s/agent_loop/slowest/response_length:1078 - timing_s/gen:12.49279616959393 - timing_s/reward:0.053980570286512375 - timing_s/old_log_prob:2.3466072604060173 - timing_s/adv:0.11904684640467167 - timing_s/update_actor:9.500159479677677 - timing_s/step:24.59877328388393 - timing_s/stop_profile:0.00014404766261577606 - timing_per_token_ms/gen:0.2114233811639041 - timing_per_token_ms/update_actor:0.07124923674357213 - timing_per_token_ms/adv:0.0008928267952981668 - perf/total_num_tokens:133337 - perf/time_per_step:24.59877328388393 - perf/throughput:677.5591940155647 (TaskRunner pid=2099163) Training Progress: 76%|███████▌ | 76/100 [59:33<11:33, 28.89s/it] (TaskRunner pid=2099163) step:77 - global_seqlen/min:14979 - global_seqlen/max:24145 - global_seqlen/minmax_diff:9166 - global_seqlen/balanced_min:17784 - global_seqlen/balanced_max:17827 - global_seqlen/mean:17795.375 - actor/entropy:0.2697010934352875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3591962456703186 - training/rollout_probs_diff_mean:0.005411113146692514 - training/rollout_probs_diff_std:0.013911079615354538 - training/rollout_actor_probs_pearson_corr:0.9983001947402954 - rollout_corr/rollout_is_mean:0.9998016357421875 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00011526238813530654 - rollout_corr/rollout_is_max:1.7653801441192627 - rollout_corr/rollout_is_min:0.3611298203468323 - rollout_corr/rollout_is_std:0.04165085032582283 - rollout_corr/rollout_is_eff_sample_size:0.9982674389310602 - rollout_corr/rollout_is_seq_mean:0.999787449836731 - rollout_corr/rollout_is_seq_std:0.0032496904022991657 - rollout_corr/rollout_is_seq_max:1.0096925497055054 - rollout_corr/rollout_is_seq_min:0.989059329032898 - rollout_corr/rollout_is_seq_max_deviation:0.01094067096710205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3272224520333111) - rollout_corr/training_log_ppl:np.float64(0.2796602731687017) - rollout_corr/kl:np.float64(0.0015744009766365252) - rollout_corr/k3_kl:np.float64(0.00129540834745967) - rollout_corr/rollout_ppl:np.float64(1.325115962419659) - rollout_corr/rollout_log_ppl:np.float64(0.2780858727055602) - rollout_corr/log_ppl_diff:np.float64(0.001574400463141501) - rollout_corr/log_ppl_abs_diff:np.float64(0.003008296713232994) - rollout_corr/log_ppl_diff_max:np.float64(0.012766122817993164) - rollout_corr/log_ppl_diff_min:np.float64(-0.008344709873199463) - rollout_corr/ppl_ratio:np.float64(1.001582418801263) - rollout_corr/chi2_token:np.float64(0.002242328831925988) - rollout_corr/chi2_seq:np.float64(0.23463519662618637) - actor/pg_loss:np.float64(0.00029490748420357704) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014422528074646834) - actor/ppo_kl:np.float64(0.0004096870738550251) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5797202438116074) - perf/mfu/actor:np.float64(0.05305596361026153) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.87368392944336) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:5 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008615863509476185 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008615863509476185 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:237.23046875 - response_length/max:1068.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:237.23046875 - response_length_non_aborted/max:1068.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.313225746154785e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0067482870072126) - timing_s/agent_loop/generate_sequences/max:np.float64(6.926735129207373) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.472584339346213) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.926735129207373) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1068 - timing_s/gen:12.724126374348998 - timing_s/reward:0.05565854348242283 - timing_s/old_log_prob:2.3462998792529106 - timing_s/adv:0.12156063131988049 - timing_s/update_actor:9.614609019830823 - timing_s/step:24.949520375579596 - timing_s/stop_profile:0.00014005228877067566 - timing_per_token_ms/gen:0.20951616759725672 - timing_per_token_ms/update_actor:0.06753586971215009 - timing_per_token_ms/adv:0.0008538779831829934 - perf/total_num_tokens:142363 - perf/time_per_step:24.949520375579596 - perf/throughput:713.2551941727096 (TaskRunner pid=2099163) Training Progress: 77%|███████▋ | 77/100 [59:58<10:37, 27.72s/it] (TaskRunner pid=2099163) step:78 - global_seqlen/min:15123 - global_seqlen/max:23777 - global_seqlen/minmax_diff:8654 - global_seqlen/balanced_min:18759 - global_seqlen/balanced_max:18771 - global_seqlen/mean:18768.625 - actor/entropy:0.25392431020736694 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6629616618156433 - training/rollout_probs_diff_mean:0.00486655393615365 - training/rollout_probs_diff_std:0.012811429798603058 - training/rollout_actor_probs_pearson_corr:0.9985421299934387 - rollout_corr/rollout_is_mean:1.0001100301742554 - rollout_corr/rollout_is_ratio_fraction_high:1.5132027328945696e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.0528109315782785e-05 - rollout_corr/rollout_is_max:2.284006357192993 - rollout_corr/rollout_is_min:0.19336040318012238 - rollout_corr/rollout_is_std:0.03998841717839241 - rollout_corr/rollout_is_eff_sample_size:0.9984037172013397 - rollout_corr/rollout_is_seq_mean:1.000152587890625 - rollout_corr/rollout_is_seq_std:0.003139234147965908 - rollout_corr/rollout_is_seq_max:1.0102607011795044 - rollout_corr/rollout_is_seq_min:0.9885665774345398 - rollout_corr/rollout_is_seq_max_deviation:0.011433422565460205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3306259801611304) - rollout_corr/training_log_ppl:np.float64(0.2809669480775483) - rollout_corr/kl:np.float64(0.0013480030079620064) - rollout_corr/k3_kl:np.float64(0.0014645395496017954) - rollout_corr/rollout_ppl:np.float64(1.328828570432961) - rollout_corr/rollout_log_ppl:np.float64(0.2796189447399229) - rollout_corr/log_ppl_diff:np.float64(0.0013480033376254141) - rollout_corr/log_ppl_abs_diff:np.float64(0.003176851896569133) - rollout_corr/log_ppl_diff_max:np.float64(0.02776774764060974) - rollout_corr/log_ppl_diff_min:np.float64(-0.009948790073394775) - rollout_corr/ppl_ratio:np.float64(1.0013591742608696) - rollout_corr/chi2_token:np.float64(0.0036836275830864906) - rollout_corr/chi2_seq:np.float64(2.4141830562148243) - actor/pg_loss:np.float64(0.0004103210521861911) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0027973866708634887) - actor/ppo_kl:np.float64(0.0005619901072861921) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7025328427553177) - perf/mfu/actor:np.float64(0.054799120913851654) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.03145599365234) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:5 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018179239705204964 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018179239705204964 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:258.14453125 - response_length/max:1046.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:258.14453125 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010188482701778412 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7525202222168446) - timing_s/agent_loop/generate_sequences/max:np.float64(6.806002235040069) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3579090451748925) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.806002235040069) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:12.32015965692699 - timing_s/reward:0.056415656581521034 - timing_s/old_log_prob:2.315779550001025 - timing_s/adv:0.11760320328176022 - timing_s/update_actor:9.819602735340595 - timing_s/step:24.71673825569451 - timing_s/stop_profile:0.00011189281940460205 - timing_per_token_ms/gen:0.18642898777221745 - timing_per_token_ms/update_actor:0.065399055174131 - timing_per_token_ms/adv:0.0007832433335004577 - perf/total_num_tokens:150149 - perf/time_per_step:24.71673825569451 - perf/throughput:759.3487783800066 (TaskRunner pid=2099163) Training Progress: 78%|███████▊ | 78/100 [1:00:23<09:50, 26.84s/it] (TaskRunner pid=2099163) step:79 - global_seqlen/min:12334 - global_seqlen/max:22428 - global_seqlen/minmax_diff:10094 - global_seqlen/balanced_min:17635 - global_seqlen/balanced_max:17782 - global_seqlen/mean:17654.5 - actor/entropy:0.27384743094444275 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2592231035232544 - training/rollout_probs_diff_mean:0.005185640417039394 - training/rollout_probs_diff_std:0.012722447514533997 - training/rollout_actor_probs_pearson_corr:0.9985835552215576 - rollout_corr/rollout_is_mean:0.9998155236244202 - rollout_corr/rollout_is_ratio_fraction_high:1.6117593986564316e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.223518797312863e-05 - rollout_corr/rollout_is_max:2.173496961593628 - rollout_corr/rollout_is_min:0.37063977122306824 - rollout_corr/rollout_is_std:0.039584673941135406 - rollout_corr/rollout_is_eff_sample_size:0.9984348513727722 - rollout_corr/rollout_is_seq_mean:0.9998263120651245 - rollout_corr/rollout_is_seq_std:0.003011947264894843 - rollout_corr/rollout_is_seq_max:1.014046549797058 - rollout_corr/rollout_is_seq_min:0.9915293455123901 - rollout_corr/rollout_is_seq_max_deviation:0.014046549797058105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.340323363430798) - rollout_corr/training_log_ppl:np.float64(0.2888445410062559) - rollout_corr/kl:np.float64(0.0010857870700355932) - rollout_corr/k3_kl:np.float64(0.0011207945151454624) - rollout_corr/rollout_ppl:np.float64(1.3388538309372962) - rollout_corr/rollout_log_ppl:np.float64(0.2877587534894701) - rollout_corr/log_ppl_diff:np.float64(0.0010857875167857856) - rollout_corr/log_ppl_abs_diff:np.float64(0.002984179329359904) - rollout_corr/log_ppl_diff_max:np.float64(0.013294041156768799) - rollout_corr/log_ppl_diff_min:np.float64(-0.009193763136863708) - rollout_corr/ppl_ratio:np.float64(1.001093138474971) - rollout_corr/chi2_token:np.float64(0.0023503343109041452) - rollout_corr/chi2_seq:np.float64(0.57042295509018) - actor/pg_loss:np.float64(-0.00013655168004333973) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013187846618620824) - actor/ppo_kl:np.float64(3.9207486743464415e-05) - actor/pg_clipfrac_lower:np.float64(2.0559211407089606e-05) - actor/grad_norm:np.float64(0.6367911696434021) - perf/mfu/actor:np.float64(0.05222786833305854) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.97899627685547) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:5 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013035103678703308 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013035103678703308 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:242.359375 - response_length/max:1251.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:242.359375 - response_length_non_aborted/max:1251.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.34375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014719553291797638 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1327050235122442) - timing_s/agent_loop/generate_sequences/max:np.float64(7.875947218388319) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5553116282462724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.875947218388319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1251 - timing_s/gen:13.897762881591916 - timing_s/reward:0.05496951378881931 - timing_s/old_log_prob:2.362206546589732 - timing_s/adv:0.12380427122116089 - timing_s/update_actor:9.482831593602896 - timing_s/step:26.00793389044702 - timing_s/stop_profile:0.00010822899639606476 - timing_per_token_ms/gen:0.22399849915530778 - timing_per_token_ms/update_actor:0.06714174568525656 - timing_per_token_ms/adv:0.0008765772977226832 - perf/total_num_tokens:141236 - perf/time_per_step:26.00793389044702 - perf/throughput:678.8120915089175 (TaskRunner pid=2099163) Training Progress: 79%|███████▉ | 79/100 [1:00:49<09:18, 26.60s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a specific protein sequence can be determined using computational methods such as those implemented in tools like I-TASSER, DSSP, or other protein structure prediction software. These tools calculate various scores based on the predicted 3D structure of the protein. The score provided (e.g., -0.01, 1.69, 2.49, 0.45) typically reflects the stability of the predicted structure relative to a reference structure. (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given the sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR', it is likely that this sequence has been analyzed using a specific computational tool. Based on known results for similar sequences and the range of scores provided, the folding stability score for this sequence is most likely 0.45. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2099163) step:80 - global_seqlen/min:12878 - global_seqlen/max:22644 - global_seqlen/minmax_diff:9766 - global_seqlen/balanced_min:16682 - global_seqlen/balanced_max:16689 - global_seqlen/mean:16687.125 - actor/entropy:0.31366831064224243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3867309093475342 - training/rollout_probs_diff_mean:0.005574310198426247 - training/rollout_probs_diff_std:0.012939517386257648 - training/rollout_actor_probs_pearson_corr:0.9986551403999329 - rollout_corr/rollout_is_mean:0.9999090433120728 - rollout_corr/rollout_is_ratio_fraction_high:1.7949132598005235e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.589826519601047e-05 - rollout_corr/rollout_is_max:2.8355820178985596 - rollout_corr/rollout_is_min:0.44648870825767517 - rollout_corr/rollout_is_std:0.041322559118270874 - rollout_corr/rollout_is_eff_sample_size:0.9982948816937318 - rollout_corr/rollout_is_seq_mean:0.9998655319213867 - rollout_corr/rollout_is_seq_std:0.003264273749664426 - rollout_corr/rollout_is_seq_max:1.0161584615707397 - rollout_corr/rollout_is_seq_min:0.9922093152999878 - rollout_corr/rollout_is_seq_max_deviation:0.016158461570739746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3791068121790886) - rollout_corr/training_log_ppl:np.float64(0.31735638427198865) - rollout_corr/kl:np.float64(0.0011907178431336263) - rollout_corr/k3_kl:np.float64(0.0012447628572545) - rollout_corr/rollout_ppl:np.float64(1.3774029430933297) - rollout_corr/rollout_log_ppl:np.float64(0.3161656655429397) - rollout_corr/log_ppl_diff:np.float64(0.0011907187290489674) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031865338096395135) - rollout_corr/log_ppl_diff_max:np.float64(0.015649110078811646) - rollout_corr/log_ppl_diff_min:np.float64(-0.012567013502120972) - rollout_corr/ppl_ratio:np.float64(1.0011996023822576) - rollout_corr/chi2_token:np.float64(0.0025439781602472067) - rollout_corr/chi2_seq:np.float64(0.696196470875293) - actor/pg_loss:np.float64(0.0002089452464133501) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019131674780510366) - actor/ppo_kl:np.float64(0.0001621537604137302) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6633199006319046) - perf/mfu/actor:np.float64(0.049463591662902) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.01802444458008) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.41625) - val-aux/sciknoweval/reward/std@16:np.float64(0.3144664633085353) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.55652) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.28145093411213773) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.27288) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2558732573095932) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4132) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.3137503776392298) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6777599999999998) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.19852437956426017) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.17295999999999995) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1598024868585598) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.41869999999999996) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2850227986027261) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.74804) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.10224935564981508) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.12108) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07521408052792339) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.41981999999999997) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.2487908426641159) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7791199999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.046372263261744234) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.09526) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.035559683891627565) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.42042) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.20911651768131542) - val-aux/sciknoweval/score/mean@16:np.float64(0.41625) - val-aux/sciknoweval/score/std@16:np.float64(0.3144664633085353) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.55652) - val-aux/sciknoweval/score/best@2/std:np.float64(0.28145093411213773) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.27288) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2558732573095932) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4132) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.3137503776392298) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6777599999999998) - val-aux/sciknoweval/score/best@4/std:np.float64(0.19852437956426017) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.17295999999999995) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1598024868585598) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.41869999999999996) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2850227986027261) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.74804) - val-aux/sciknoweval/score/best@8/std:np.float64(0.10224935564981508) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.12108) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07521408052792339) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.41981999999999997) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.2487908426641159) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7791199999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.046372263261744234) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.09526) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.035559683891627565) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.42042) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.20911651768131542) - val-core/sciknoweval/acc/mean@16:np.float64(0.41625) - val-aux/sciknoweval/acc/std@16:np.float64(0.3144664633085353) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.55652) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.28145093411213773) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.27288) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2558732573095932) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4132) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.3137503776392298) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6777599999999998) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.19852437956426017) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.17295999999999995) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1598024868585598) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.41869999999999996) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2850227986027261) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.74804) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.10224935564981508) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.12108) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07521408052792339) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.41981999999999997) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.2487908426641159) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7791199999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.046372263261744234) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.09526) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.035559683891627565) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.42042) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.20911651768131542) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:5 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010260173119604588 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010260173119604588 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:217.62890625 - response_length/max:808.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:217.62890625 - response_length_non_aborted/max:808.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.84375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013283640146255493 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0369690340012312) - timing_s/agent_loop/generate_sequences/max:np.float64(5.508786328136921) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4494186551164603) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.508786328136921) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:808 - timing_s/gen:11.310583300888538 - timing_s/reward:0.05467207543551922 - timing_s/old_log_prob:2.336861990392208 - timing_s/adv:0.1195262223482132 - timing_s/update_actor:9.54221854172647 - timing_s/step:23.44945344515145 - timing_s/testing:31.679148972034454 - timing_s/save_checkpoint:7.0816802475601435 - timing_s/stop_profile:0.0001507047563791275 - timing_per_token_ms/gen:0.20301515446823074 - timing_per_token_ms/update_actor:0.07147889871477614 - timing_per_token_ms/adv:0.0008953476283977407 - perf/total_num_tokens:133497 - perf/time_per_step:23.44945344515145 - perf/throughput:711.6210635369982 (TaskRunner pid=2099163) Training Progress: 80%|████████ | 80/100 [1:01:51<12:26, 37.30s/it] (TaskRunner pid=2099163) step:81 - global_seqlen/min:15559 - global_seqlen/max:22877 - global_seqlen/minmax_diff:7318 - global_seqlen/balanced_min:19228 - global_seqlen/balanced_max:19441 - global_seqlen/mean:19257.0 - actor/entropy:0.2643565535545349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2313688099384308 - training/rollout_probs_diff_mean:0.0049353111535310745 - training/rollout_probs_diff_std:0.012572631239891052 - training/rollout_actor_probs_pearson_corr:0.9985886812210083 - rollout_corr/rollout_is_mean:1.0000520944595337 - rollout_corr/rollout_is_ratio_fraction_high:1.5707464626757428e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.8178207874298096 - rollout_corr/rollout_is_min:0.5387734174728394 - rollout_corr/rollout_is_std:0.03955530375242233 - rollout_corr/rollout_is_eff_sample_size:0.9984379411292814 - rollout_corr/rollout_is_seq_mean:1.000102162361145 - rollout_corr/rollout_is_seq_std:0.0028133587911725044 - rollout_corr/rollout_is_seq_max:1.0072697401046753 - rollout_corr/rollout_is_seq_min:0.9939631819725037 - rollout_corr/rollout_is_seq_max_deviation:0.007269740104675293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.344506647437811) - rollout_corr/training_log_ppl:np.float64(0.29159667881322093) - rollout_corr/kl:np.float64(0.0010141099861371572) - rollout_corr/k3_kl:np.float64(0.0013735205002944895) - rollout_corr/rollout_ppl:np.float64(1.3431055014953017) - rollout_corr/rollout_log_ppl:np.float64(0.2905825663037831) - rollout_corr/log_ppl_diff:np.float64(0.0010141125094378367) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028742786817019805) - rollout_corr/log_ppl_diff_max:np.float64(0.01934957504272461) - rollout_corr/log_ppl_diff_min:np.float64(-0.010468631982803345) - rollout_corr/ppl_ratio:np.float64(1.0010220303665847) - rollout_corr/chi2_token:np.float64(0.003484314540401101) - rollout_corr/chi2_seq:np.float64(0.5838833905290812) - actor/pg_loss:np.float64(0.00048545573372393847) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0023849409749345796) - actor/ppo_kl:np.float64(0.00024174882969418832) - actor/pg_clipfrac_lower:np.float64(3.98596930608619e-05) - actor/grad_norm:np.float64(0.7806772142648697) - perf/mfu/actor:np.float64(0.05648842110660632) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.96889114379883) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:5 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:3.9268659747904167e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:3.9268659747904167e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:248.6875 - response_length/max:1421.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:248.6875 - response_length_non_aborted/max:1421.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:353.09375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015094690024852753 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7632148675620556) - timing_s/agent_loop/generate_sequences/max:np.float64(8.421508487313986) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.538855082115333) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.421508487313986) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1421 - timing_s/gen:13.950085857883096 - timing_s/reward:0.053744787350296974 - timing_s/old_log_prob:2.4113160967826843 - timing_s/adv:0.11996482498943806 - timing_s/update_actor:9.743194399401546 - timing_s/step:26.376118445768952 - timing_s/stop_profile:4.84175980091095e-05 - timing_per_token_ms/gen:0.21912047401801796 - timing_per_token_ms/update_actor:0.0632444981007007 - timing_per_token_ms/adv:0.0007787092030783485 - perf/total_num_tokens:154056 - perf/time_per_step:26.376118445768952 - perf/throughput:730.0922628018095 (TaskRunner pid=2099163) Training Progress: 81%|████████ | 81/100 [1:02:18<10:46, 34.04s/it] (TaskRunner pid=2099163) step:82 - global_seqlen/min:13327 - global_seqlen/max:21973 - global_seqlen/minmax_diff:8646 - global_seqlen/balanced_min:17016 - global_seqlen/balanced_max:17032 - global_seqlen/mean:17027.5 - actor/entropy:0.2781171500682831 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25190842151641846 - training/rollout_probs_diff_mean:0.005356854293495417 - training/rollout_probs_diff_std:0.013084305450320244 - training/rollout_actor_probs_pearson_corr:0.9985019564628601 - rollout_corr/rollout_is_mean:0.9998621344566345 - rollout_corr/rollout_is_ratio_fraction_high:1.7285487047047354e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.642743341624737e-05 - rollout_corr/rollout_is_max:2.251864433288574 - rollout_corr/rollout_is_min:0.29135599732398987 - rollout_corr/rollout_is_std:0.0413888581097126 - rollout_corr/rollout_is_eff_sample_size:0.99828953558267 - rollout_corr/rollout_is_seq_mean:0.999849796295166 - rollout_corr/rollout_is_seq_std:0.003499638754874468 - rollout_corr/rollout_is_seq_max:1.008987307548523 - rollout_corr/rollout_is_seq_min:0.9893508553504944 - rollout_corr/rollout_is_seq_max_deviation:0.010649144649505615 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3479181840084493) - rollout_corr/training_log_ppl:np.float64(0.293532015173696) - rollout_corr/kl:np.float64(0.0012917956987497092) - rollout_corr/k3_kl:np.float64(0.0012352343883321737) - rollout_corr/rollout_ppl:np.float64(1.346091446466744) - rollout_corr/rollout_log_ppl:np.float64(0.29224021729896776) - rollout_corr/log_ppl_diff:np.float64(0.0012917978747282177) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030594941636081785) - rollout_corr/log_ppl_diff_max:np.float64(0.012573987245559692) - rollout_corr/log_ppl_diff_min:np.float64(-0.011092990636825562) - rollout_corr/ppl_ratio:np.float64(1.0012995367869735) - rollout_corr/chi2_token:np.float64(0.0022806792985647917) - rollout_corr/chi2_seq:np.float64(1.2513767953496426) - actor/pg_loss:np.float64(-1.996220089495182e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011945247019866656) - actor/ppo_kl:np.float64(0.00018004048818909268) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6068511605262756) - perf/mfu/actor:np.float64(0.04981626783172659) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.0265121459961) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:5 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005667479243129492 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005667479243129492 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:225.984375 - response_length/max:982.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:225.984375 - response_length_non_aborted/max:982.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.120729863643646e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0436728745698929) - timing_s/agent_loop/generate_sequences/max:np.float64(6.355573097243905) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3314835890123504) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.355573097243905) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:982 - timing_s/gen:12.609378339722753 - timing_s/reward:0.05375535041093826 - timing_s/old_log_prob:2.3262925799936056 - timing_s/adv:0.12487000413239002 - timing_s/update_actor:9.59987291507423 - timing_s/step:24.80237107910216 - timing_s/stop_profile:0.00011940300464630127 - timing_per_token_ms/gen:0.2179592466936796 - timing_per_token_ms/update_actor:0.0704732999197932 - timing_per_token_ms/adv:0.0009166789321126855 - perf/total_num_tokens:136220 - perf/time_per_step:24.80237107910216 - perf/throughput:686.5271044326457 (TaskRunner pid=2099163) Training Progress: 82%|████████▏ | 82/100 [1:02:43<09:23, 31.28s/it] (TaskRunner pid=2099163) step:83 - global_seqlen/min:12925 - global_seqlen/max:26465 - global_seqlen/minmax_diff:13540 - global_seqlen/balanced_min:17539 - global_seqlen/balanced_max:17863 - global_seqlen/mean:17587.5 - actor/entropy:0.2657328248023987 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2070435881614685 - training/rollout_probs_diff_mean:0.005034568719565868 - training/rollout_probs_diff_std:0.012614690698683262 - training/rollout_actor_probs_pearson_corr:0.9985789060592651 - rollout_corr/rollout_is_mean:1.000186562538147 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.767050268128514e-05 - rollout_corr/rollout_is_max:1.6093506813049316 - rollout_corr/rollout_is_min:0.31631869077682495 - rollout_corr/rollout_is_std:0.03947233781218529 - rollout_corr/rollout_is_eff_sample_size:0.9984449525707558 - rollout_corr/rollout_is_seq_mean:1.0001424551010132 - rollout_corr/rollout_is_seq_std:0.0030042307917028666 - rollout_corr/rollout_is_seq_max:1.0098767280578613 - rollout_corr/rollout_is_seq_min:0.9898255467414856 - rollout_corr/rollout_is_seq_max_deviation:0.010174453258514404 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3396281930617988) - rollout_corr/training_log_ppl:np.float64(0.28824585146503523) - rollout_corr/kl:np.float64(0.0013105916907552029) - rollout_corr/k3_kl:np.float64(0.0012636836211754598) - rollout_corr/rollout_ppl:np.float64(1.3378654848784208) - rollout_corr/rollout_log_ppl:np.float64(0.2869352596462704) - rollout_corr/log_ppl_diff:np.float64(0.0013105918187648058) - rollout_corr/log_ppl_abs_diff:np.float64(0.002559717104304582) - rollout_corr/log_ppl_diff_max:np.float64(0.012187853455543518) - rollout_corr/log_ppl_diff_min:np.float64(-0.008456051349639893) - rollout_corr/ppl_ratio:np.float64(1.0013164200354367) - rollout_corr/chi2_token:np.float64(0.0023376578465104103) - rollout_corr/chi2_seq:np.float64(0.27040194789879024) - actor/pg_loss:np.float64(0.00010856811422854662) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011537321538526157) - actor/ppo_kl:np.float64(0.0005853857873887591) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6243516802787781) - perf/mfu/actor:np.float64(0.05163460793549127) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.9116096496582) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:5 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0019763396121561527 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0019763396121561527 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:245.828125 - response_length/max:1487.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:245.828125 - response_length_non_aborted/max:1487.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.453855454921722e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1134148389101028) - timing_s/agent_loop/generate_sequences/max:np.float64(8.951366694644094) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.531380183805595) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.951366694644094) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:1487 - timing_s/gen:14.601487068459392 - timing_s/reward:0.0553099662065506 - timing_s/old_log_prob:2.3057028613984585 - timing_s/adv:0.11853219196200371 - timing_s/update_actor:9.658092549070716 - timing_s/step:26.824403259903193 - timing_s/stop_profile:4.142150282859802e-05 - timing_per_token_ms/gen:0.23202007036896002 - timing_per_token_ms/update_actor:0.0686431595527414 - timing_per_token_ms/adv:0.0008424462826013057 - perf/total_num_tokens:140700 - perf/time_per_step:26.824403259903193 - perf/throughput:655.6529824575666 (TaskRunner pid=2099163) Training Progress: 83%|████████▎ | 83/100 [1:03:09<08:29, 29.95s/it] (TaskRunner pid=2099163) step:84 - global_seqlen/min:14425 - global_seqlen/max:25507 - global_seqlen/minmax_diff:11082 - global_seqlen/balanced_min:19094 - global_seqlen/balanced_max:19099 - global_seqlen/mean:19098.0 - actor/entropy:0.29119229316711426 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3042685389518738 - training/rollout_probs_diff_mean:0.005367741920053959 - training/rollout_probs_diff_std:0.012942570261657238 - training/rollout_actor_probs_pearson_corr:0.9985758066177368 - rollout_corr/rollout_is_mean:1.0001381635665894 - rollout_corr/rollout_is_ratio_fraction_high:1.4721469597134273e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.9442939194268547e-05 - rollout_corr/rollout_is_max:2.259265661239624 - rollout_corr/rollout_is_min:0.32297244668006897 - rollout_corr/rollout_is_std:0.04044489935040474 - rollout_corr/rollout_is_eff_sample_size:0.9983673568485679 - rollout_corr/rollout_is_seq_mean:1.0002832412719727 - rollout_corr/rollout_is_seq_std:0.0029685506597161293 - rollout_corr/rollout_is_seq_max:1.0090413093566895 - rollout_corr/rollout_is_seq_min:0.9883840680122375 - rollout_corr/rollout_is_seq_max_deviation:0.011615931987762451 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3692081668414176) - rollout_corr/training_log_ppl:np.float64(0.31009084297693335) - rollout_corr/kl:np.float64(0.0009806591620620253) - rollout_corr/k3_kl:np.float64(0.0011354790972291084) - rollout_corr/rollout_ppl:np.float64(1.367792998906225) - rollout_corr/rollout_log_ppl:np.float64(0.30911018152255565) - rollout_corr/log_ppl_diff:np.float64(0.000980661454377696) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027504815079737455) - rollout_corr/log_ppl_diff_max:np.float64(0.018712282180786133) - rollout_corr/log_ppl_diff_min:np.float64(-0.009203359484672546) - rollout_corr/ppl_ratio:np.float64(1.0009879535064101) - rollout_corr/chi2_token:np.float64(0.0026025581173598766) - rollout_corr/chi2_seq:np.float64(1.0249379503075033) - actor/pg_loss:np.float64(0.0003060917370021343) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015142915781325428) - actor/ppo_kl:np.float64(0.00034652300968218697) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5972811579704285) - perf/mfu/actor:np.float64(0.05583602661339712) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.96282958984375) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:5 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036454040091484785 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036454040091484785 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:265.34375 - response_length/max:926.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:265.34375 - response_length_non_aborted/max:926.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:331.46875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8295442461967468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1363158952444792) - timing_s/agent_loop/generate_sequences/max:np.float64(6.200030520558357) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6375616744408035) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.200030520558357) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:926 - timing_s/gen:12.048710951581597 - timing_s/reward:0.057196732610464096 - timing_s/old_log_prob:2.329210080206394 - timing_s/adv:0.11962089873850346 - timing_s/update_actor:9.724592242389917 - timing_s/step:24.364509677514434 - timing_s/stop_profile:3.784149885177612e-05 - timing_per_token_ms/gen:0.1773747343007537 - timing_per_token_ms/update_actor:0.0636492842338852 - timing_per_token_ms/adv:0.0007829412683167311 - perf/total_num_tokens:152784 - perf/time_per_step:24.364509677514434 - perf/throughput:783.8450374244633 (TaskRunner pid=2099163) Training Progress: 84%|████████▍ | 84/100 [1:03:34<07:32, 28.28s/it] (TaskRunner pid=2099163) step:85 - global_seqlen/min:13201 - global_seqlen/max:23013 - global_seqlen/minmax_diff:9812 - global_seqlen/balanced_min:17098 - global_seqlen/balanced_max:17114 - global_seqlen/mean:17108.75 - actor/entropy:0.3020118772983551 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4086613655090332 - training/rollout_probs_diff_mean:0.005688806995749474 - training/rollout_probs_diff_std:0.01348093617707491 - training/rollout_actor_probs_pearson_corr:0.9984976053237915 - rollout_corr/rollout_is_mean:0.9999252557754517 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.976279888069257e-05 - rollout_corr/rollout_is_max:1.660659909248352 - rollout_corr/rollout_is_min:0.12563924491405487 - rollout_corr/rollout_is_std:0.04169733077287674 - rollout_corr/rollout_is_eff_sample_size:0.9982643502297281 - rollout_corr/rollout_is_seq_mean:0.9999344944953918 - rollout_corr/rollout_is_seq_std:0.0032068761065602303 - rollout_corr/rollout_is_seq_max:1.0129774808883667 - rollout_corr/rollout_is_seq_min:0.99162358045578 - rollout_corr/rollout_is_seq_max_deviation:0.0129774808883667 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3707473212853074) - rollout_corr/training_log_ppl:np.float64(0.3106351788446773) - rollout_corr/kl:np.float64(0.0010191974550188831) - rollout_corr/k3_kl:np.float64(0.0012126699965619991) - rollout_corr/rollout_ppl:np.float64(1.3693385659717023) - rollout_corr/rollout_log_ppl:np.float64(0.3096159797569271) - rollout_corr/log_ppl_diff:np.float64(0.0010191990877501667) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027312866295687854) - rollout_corr/log_ppl_diff_max:np.float64(0.01148831844329834) - rollout_corr/log_ppl_diff_min:np.float64(-0.009180232882499695) - rollout_corr/ppl_ratio:np.float64(1.001025541452691) - rollout_corr/chi2_token:np.float64(0.002835497260093689) - rollout_corr/chi2_seq:np.float64(0.9057672906201333) - actor/pg_loss:np.float64(-5.922233685851097e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001044121142513177) - actor/ppo_kl:np.float64(4.945502578124206e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4638519212603569) - perf/mfu/actor:np.float64(0.050201444400562734) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.93396759033203) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:6 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006879706867039204 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006879706867039204 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:235.4921875 - response_length/max:1005.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:235.4921875 - response_length_non_aborted/max:1005.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.15625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0004599113017320633 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2714347038418055) - timing_s/agent_loop/generate_sequences/max:np.float64(6.474359966814518) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.499616449684254) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.474359966814518) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1005 - timing_s/gen:12.510914074257016 - timing_s/reward:0.04957201145589352 - timing_s/old_log_prob:2.413747824728489 - timing_s/adv:0.13123666122555733 - timing_s/update_actor:9.761451695114374 - timing_s/step:24.98761263117194 - timing_s/stop_profile:0.0001248195767402649 - timing_per_token_ms/gen:0.20752602717475063 - timing_per_token_ms/update_actor:0.07131914733041846 - timing_per_token_ms/adv:0.0009588416835358904 - perf/total_num_tokens:136870 - perf/time_per_step:24.98761263117194 - perf/throughput:684.6892599358174 (TaskRunner pid=2099163) Training Progress: 85%|████████▌ | 85/100 [1:03:59<06:51, 27.44s/it] (TaskRunner pid=2099163) step:86 - global_seqlen/min:12509 - global_seqlen/max:25462 - global_seqlen/minmax_diff:12953 - global_seqlen/balanced_min:18292 - global_seqlen/balanced_max:18296 - global_seqlen/mean:18294.125 - actor/entropy:0.2849932610988617 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35150495171546936 - training/rollout_probs_diff_mean:0.0054772403091192245 - training/rollout_probs_diff_std:0.013425457291305065 - training/rollout_actor_probs_pearson_corr:0.9984551668167114 - rollout_corr/rollout_is_mean:1.0001648664474487 - rollout_corr/rollout_is_ratio_fraction_high:4.6187244151951745e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0791496101301163e-05 - rollout_corr/rollout_is_max:2.974616289138794 - rollout_corr/rollout_is_min:0.459360271692276 - rollout_corr/rollout_is_std:0.04225114360451698 - rollout_corr/rollout_is_eff_sample_size:0.9982186159276442 - rollout_corr/rollout_is_seq_mean:1.0002799034118652 - rollout_corr/rollout_is_seq_std:0.003540112404152751 - rollout_corr/rollout_is_seq_max:1.0164246559143066 - rollout_corr/rollout_is_seq_min:0.9909144639968872 - rollout_corr/rollout_is_seq_max_deviation:0.01642465591430664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3678325335495174) - rollout_corr/training_log_ppl:np.float64(0.30802994256373495) - rollout_corr/kl:np.float64(0.0011918487436304304) - rollout_corr/k3_kl:np.float64(0.0014298067977733808) - rollout_corr/rollout_ppl:np.float64(1.366152125876397) - rollout_corr/rollout_log_ppl:np.float64(0.30683809251058847) - rollout_corr/log_ppl_diff:np.float64(0.0011918500531464815) - rollout_corr/log_ppl_abs_diff:np.float64(0.003217801859136671) - rollout_corr/log_ppl_diff_max:np.float64(0.010454237461090088) - rollout_corr/log_ppl_diff_min:np.float64(-0.01674443483352661) - rollout_corr/ppl_ratio:np.float64(1.0012002452276647) - rollout_corr/chi2_token:np.float64(0.003339178394526243) - rollout_corr/chi2_seq:np.float64(1.9388157108332962) - actor/pg_loss:np.float64(-0.0001976813655346632) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001634979141272197) - actor/ppo_kl:np.float64(0.0004851831923424754) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5677638277411461) - perf/mfu/actor:np.float64(0.05395100018655371) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.07014083862305) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:6 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019225439755246043 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019225439755246043 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:253.72265625 - response_length/max:792.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:253.72265625 - response_length_non_aborted/max:792.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.96875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015383213758468628 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4205127842724323) - timing_s/agent_loop/generate_sequences/max:np.float64(5.750876966863871) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6700041760486783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.750876966863871) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:787 - timing_s/gen:11.532334372401237 - timing_s/reward:0.0561569519340992 - timing_s/old_log_prob:2.331783862784505 - timing_s/adv:0.12503085657954216 - timing_s/update_actor:9.721932409331203 - timing_s/step:23.852430075407028 - timing_s/stop_profile:0.00010659173130989075 - timing_per_token_ms/gen:0.17754891032594702 - timing_per_token_ms/update_actor:0.0664279680589479 - timing_per_token_ms/adv:0.0008543101718416579 - perf/total_num_tokens:146353 - perf/time_per_step:23.852430075407028 - perf/throughput:766.9711195951518 (TaskRunner pid=2099163) Training Progress: 86%|████████▌ | 86/100 [1:04:23<06:09, 26.38s/it] (TaskRunner pid=2099163) step:87 - global_seqlen/min:13933 - global_seqlen/max:20602 - global_seqlen/minmax_diff:6669 - global_seqlen/balanced_min:16687 - global_seqlen/balanced_max:16691 - global_seqlen/mean:16689.75 - actor/entropy:0.28263986110687256 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105812072753906 - training/rollout_probs_diff_mean:0.0056616635993123055 - training/rollout_probs_diff_std:0.013550084084272385 - training/rollout_actor_probs_pearson_corr:0.9984059929847717 - rollout_corr/rollout_is_mean:1.0001320838928223 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.4052106861490756e-05 - rollout_corr/rollout_is_max:1.8633671998977661 - rollout_corr/rollout_is_min:0.20320795476436615 - rollout_corr/rollout_is_std:0.04268762469291687 - rollout_corr/rollout_is_eff_sample_size:0.998181556398287 - rollout_corr/rollout_is_seq_mean:1.000098466873169 - rollout_corr/rollout_is_seq_std:0.00287253619171679 - rollout_corr/rollout_is_seq_max:1.0077333450317383 - rollout_corr/rollout_is_seq_min:0.9905746579170227 - rollout_corr/rollout_is_seq_max_deviation:0.009425342082977295 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3366769887506962) - rollout_corr/training_log_ppl:np.float64(0.2858038642152678) - rollout_corr/kl:np.float64(0.0010864866192696887) - rollout_corr/k3_kl:np.float64(0.001194243301597453) - rollout_corr/rollout_ppl:np.float64(1.3351997244171798) - rollout_corr/rollout_log_ppl:np.float64(0.28471737628569826) - rollout_corr/log_ppl_diff:np.float64(0.0010864879295695573) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029874266765546054) - rollout_corr/log_ppl_diff_max:np.float64(0.014816895127296448) - rollout_corr/log_ppl_diff_min:np.float64(-0.01041361689567566) - rollout_corr/ppl_ratio:np.float64(1.0010941252112389) - rollout_corr/chi2_token:np.float64(0.002591905649751425) - rollout_corr/chi2_seq:np.float64(0.8676178343594074) - actor/pg_loss:np.float64(2.4948501959443092e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008350495745617081) - actor/ppo_kl:np.float64(0.00021582131192943166) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.45856113731861115) - perf/mfu/actor:np.float64(0.04922829341564032) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.0233154296875) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:6 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.013735991902649403 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.013735991902649403 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:216.8046875 - response_length/max:808.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:216.8046875 - response_length_non_aborted/max:808.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.75 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.88928771018982e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.328505938872695) - timing_s/agent_loop/generate_sequences/max:np.float64(5.489037185907364) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3260599871064187) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.489037185907364) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:808 - timing_s/gen:11.125011002644897 - timing_s/reward:0.04664120823144913 - timing_s/old_log_prob:2.281039910390973 - timing_s/adv:0.12088093161582947 - timing_s/update_actor:9.694726234301925 - timing_s/step:23.32410356402397 - timing_s/stop_profile:0.00012058950960636139 - timing_per_token_ms/gen:0.20044342550979957 - timing_per_token_ms/update_actor:0.07260988207059665 - timing_per_token_ms/adv:0.0009053530731124602 - perf/total_num_tokens:133518 - perf/time_per_step:23.32410356402397 - perf/throughput:715.5580472444367 (TaskRunner pid=2099163) Training Progress: 87%|████████▋ | 87/100 [1:04:47<05:31, 25.48s/it] (TaskRunner pid=2099163) step:88 - global_seqlen/min:13016 - global_seqlen/max:21866 - global_seqlen/minmax_diff:8850 - global_seqlen/balanced_min:18267 - global_seqlen/balanced_max:18275 - global_seqlen/mean:18273.375 - actor/entropy:0.2799515724182129 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5366853475570679 - training/rollout_probs_diff_mean:0.005581203382462263 - training/rollout_probs_diff_std:0.013881382532417774 - training/rollout_actor_probs_pearson_corr:0.9983412027359009 - rollout_corr/rollout_is_mean:1.0000699758529663 - rollout_corr/rollout_is_ratio_fraction_high:1.5643821825506166e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.128764365101233e-05 - rollout_corr/rollout_is_max:2.0318520069122314 - rollout_corr/rollout_is_min:0.3905743956565857 - rollout_corr/rollout_is_std:0.042155105620622635 - rollout_corr/rollout_is_eff_sample_size:0.998226337009296 - rollout_corr/rollout_is_seq_mean:1.0000312328338623 - rollout_corr/rollout_is_seq_std:0.0031614850740879774 - rollout_corr/rollout_is_seq_max:1.0118353366851807 - rollout_corr/rollout_is_seq_min:0.9903686046600342 - rollout_corr/rollout_is_seq_max_deviation:0.011835336685180664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.347297566011548) - rollout_corr/training_log_ppl:np.float64(0.29373508717981167) - rollout_corr/kl:np.float64(0.0011751465276006456) - rollout_corr/k3_kl:np.float64(0.001120794104167544) - rollout_corr/rollout_ppl:np.float64(1.3456657594069839) - rollout_corr/rollout_log_ppl:np.float64(0.29255994062987156) - rollout_corr/log_ppl_diff:np.float64(0.0011751465499401093) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026922697434201837) - rollout_corr/log_ppl_diff_max:np.float64(0.011649519205093384) - rollout_corr/log_ppl_diff_min:np.float64(-0.010374203324317932) - rollout_corr/ppl_ratio:np.float64(1.001181423664093) - rollout_corr/chi2_token:np.float64(0.002159958705306053) - rollout_corr/chi2_seq:np.float64(1.1241237167268991) - actor/pg_loss:np.float64(0.00016208365559577942) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000806990730779944) - actor/ppo_kl:np.float64(0.00024305590492046392) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4049631729722023) - perf/mfu/actor:np.float64(0.05369724660460473) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.09536361694336) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:6 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01436884980648756 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01436884980648756 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:249.69921875 - response_length/max:838.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:249.69921875 - response_length_non_aborted/max:838.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:321.34375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014184601604938507 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1800287831574678) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7370752189308405) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6392609360191273) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7370752189308405) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:838 - timing_s/gen:11.481335692107677 - timing_s/reward:0.0557560957968235 - timing_s/old_log_prob:2.273548835888505 - timing_s/adv:0.11617436446249485 - timing_s/update_actor:9.607393434271216 - timing_s/step:23.618814580142498 - timing_s/stop_profile:5.055032670497894e-05 - timing_per_token_ms/gen:0.1796119658355784 - timing_per_token_ms/update_actor:0.06571988914384463 - timing_per_token_ms/adv:0.0007946969598014519 - perf/total_num_tokens:146187 - perf/time_per_step:23.618814580142498 - perf/throughput:773.6787525045108 (TaskRunner pid=2099163) Training Progress: 88%|████████▊ | 88/100 [1:05:10<04:59, 24.94s/it] (TaskRunner pid=2099163) step:89 - global_seqlen/min:15884 - global_seqlen/max:25708 - global_seqlen/minmax_diff:9824 - global_seqlen/balanced_min:19388 - global_seqlen/balanced_max:19400 - global_seqlen/mean:19395.0 - actor/entropy:0.2626587748527527 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9052478075027466 - training/rollout_probs_diff_mean:0.00535238441079855 - training/rollout_probs_diff_std:0.013822822831571102 - training/rollout_actor_probs_pearson_corr:0.9982951879501343 - rollout_corr/rollout_is_mean:0.9997128248214722 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.095330900279805e-05 - rollout_corr/rollout_is_max:1.9251736402511597 - rollout_corr/rollout_is_min:0.07634969055652618 - rollout_corr/rollout_is_std:0.04105702042579651 - rollout_corr/rollout_is_eff_sample_size:0.9983161479024087 - rollout_corr/rollout_is_seq_mean:0.9995173215866089 - rollout_corr/rollout_is_seq_std:0.0029820753261446953 - rollout_corr/rollout_is_seq_max:1.0083783864974976 - rollout_corr/rollout_is_seq_min:0.9902425408363342 - rollout_corr/rollout_is_seq_max_deviation:0.009757459163665771 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3277254402637482) - rollout_corr/training_log_ppl:np.float64(0.2786243225564249) - rollout_corr/kl:np.float64(0.0015448006739369191) - rollout_corr/k3_kl:np.float64(0.0011937761133822278) - rollout_corr/rollout_ppl:np.float64(1.3255531983450055) - rollout_corr/rollout_log_ppl:np.float64(0.2770795207761694) - rollout_corr/log_ppl_diff:np.float64(0.0015448017802555114) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028917556919623166) - rollout_corr/log_ppl_diff_max:np.float64(0.015002906322479248) - rollout_corr/log_ppl_diff_min:np.float64(-0.009606748819351196) - rollout_corr/ppl_ratio:np.float64(1.0015520232263952) - rollout_corr/chi2_token:np.float64(0.001597132533788681) - rollout_corr/chi2_seq:np.float64(0.5050114756450057) - actor/pg_loss:np.float64(0.0002184249460697174) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010780754209918086) - actor/ppo_kl:np.float64(0.00011659164282207257) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.43044082075357437) - perf/mfu/actor:np.float64(0.056737380618191155) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.99333953857422) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:6 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013847829774022102 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013847829774022102 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:256.34375 - response_length/max:944.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.34375 - response_length_non_aborted/max:944.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.797747194766998e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5112595465034246) - timing_s/agent_loop/generate_sequences/max:np.float64(6.177097799256444) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6719028287552646) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.177097799256444) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:944 - timing_s/gen:11.939295399934053 - timing_s/reward:0.054529689252376556 - timing_s/old_log_prob:2.3607167918235064 - timing_s/adv:0.12182499840855598 - timing_s/update_actor:9.700234856456518 - timing_s/step:24.254260797053576 - timing_s/stop_profile:0.00011785514652729034 - timing_per_token_ms/gen:0.1819348927211699 - timing_per_token_ms/update_actor:0.06251762604058081 - timing_per_token_ms/adv:0.0007851572467682134 - perf/total_num_tokens:155160 - perf/time_per_step:24.254260797053576 - perf/throughput:799.6533129699058 (TaskRunner pid=2099163) Training Progress: 89%|████████▉ | 89/100 [1:05:35<04:32, 24.74s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a protein sequence can be determined using computational tools such as I-TAFOR, DSSP, or other protein structure prediction software. These tools calculate the stability of a protein's folded structure based on the sequence and predicted three-dimensional structure. The score provided in the question, "-0.01", is likely derived from such computational analysis. Scores close to zero indicate that the protein sequence has a stable fold, while positive scores may indicate higher stability. Based on the given sequence and typical computational results, the folding stability score is most likely -0.01. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] A (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2099163) step:90 - global_seqlen/min:15076 - global_seqlen/max:22705 - global_seqlen/minmax_diff:7629 - global_seqlen/balanced_min:18281 - global_seqlen/balanced_max:18304 - global_seqlen/mean:18298.25 - actor/entropy:0.25007060170173645 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519518971443176 - training/rollout_probs_diff_mean:0.005134246312081814 - training/rollout_probs_diff_std:0.013177147135138512 - training/rollout_actor_probs_pearson_corr:0.998388409614563 - rollout_corr/rollout_is_mean:0.9999549388885498 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.610581815242767e-05 - rollout_corr/rollout_is_max:1.6880807876586914 - rollout_corr/rollout_is_min:0.44652095437049866 - rollout_corr/rollout_is_std:0.04015500843524933 - rollout_corr/rollout_is_eff_sample_size:0.9983898144007888 - rollout_corr/rollout_is_seq_mean:0.9998966455459595 - rollout_corr/rollout_is_seq_std:0.0029496627394109964 - rollout_corr/rollout_is_seq_max:1.0089014768600464 - rollout_corr/rollout_is_seq_min:0.9883391261100769 - rollout_corr/rollout_is_seq_max_deviation:0.011660873889923096 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.315980046056211) - rollout_corr/training_log_ppl:np.float64(0.2705623500805814) - rollout_corr/kl:np.float64(0.0011479333818513737) - rollout_corr/k3_kl:np.float64(0.0010572865134577114) - rollout_corr/rollout_ppl:np.float64(1.314408981706947) - rollout_corr/rollout_log_ppl:np.float64(0.26941441488452256) - rollout_corr/log_ppl_diff:np.float64(0.0011479351960588247) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027164410275872797) - rollout_corr/log_ppl_diff_max:np.float64(0.013363391160964966) - rollout_corr/log_ppl_diff_min:np.float64(-0.008001953363418579) - rollout_corr/ppl_ratio:np.float64(1.0011545014567673) - rollout_corr/chi2_token:np.float64(0.0019355297554284334) - rollout_corr/chi2_seq:np.float64(0.5621733826119453) - actor/pg_loss:np.float64(-3.921461757272482e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012568379429467313) - actor/ppo_kl:np.float64(0.00014396155199847271) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4547141119837761) - perf/mfu/actor:np.float64(0.05313581446106291) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.05733489990234) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4675) - val-aux/sciknoweval/reward/std@16:np.float64(0.28097887193097565) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.59294) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22754928387157947) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.34464) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2509343043440332) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4650000000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28133902959180757) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6816800000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.14720983664825582) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.23678000000000002) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.17523408016078515) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.48656) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.24240246066242588) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.73372) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08196201028513199) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.17182000000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09563785531224381) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5071399999999999) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.19410742505337317) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7635599999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04110937154135006) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.14181999999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04166172799618433) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.52198) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.15012735298954785) - val-aux/sciknoweval/score/mean@16:np.float64(0.4675) - val-aux/sciknoweval/score/std@16:np.float64(0.28097887193097565) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.59294) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22754928387157947) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.34464) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2509343043440332) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4650000000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.28133902959180757) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6816800000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.14720983664825582) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.23678000000000002) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.17523408016078515) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.48656) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.24240246066242588) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.73372) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08196201028513199) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.17182000000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09563785531224381) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5071399999999999) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.19410742505337317) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7635599999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04110937154135006) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.14181999999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04166172799618433) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.52198) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.15012735298954785) - val-core/sciknoweval/acc/mean@16:np.float64(0.4675) - val-aux/sciknoweval/acc/std@16:np.float64(0.28097887193097565) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.59294) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22754928387157947) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.34464) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2509343043440332) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4650000000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.28133902959180757) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6816800000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.14720983664825582) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.23678000000000002) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.17523408016078515) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.48656) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.24240246066242588) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.73372) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08196201028513199) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.17182000000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09563785531224381) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5071399999999999) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.19410742505337317) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7635599999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04110937154135006) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.14181999999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04166172799618433) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.52198) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.15012735298954785) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:6 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007901686243712902 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007901686243712902 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:256.6328125 - response_length/max:1166.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.6328125 - response_length_non_aborted/max:1166.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.1875 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.221715688705444e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1986623331904411) - timing_s/agent_loop/generate_sequences/max:np.float64(7.395749419927597) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6543824151231092) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.395749419927597) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1166 - timing_s/gen:13.268566366285086 - timing_s/reward:0.05747130326926708 - timing_s/old_log_prob:2.363747773692012 - timing_s/adv:0.1281336061656475 - timing_s/update_actor:9.728703785687685 - timing_s/step:25.604432402178645 - timing_s/testing:25.468937627971172 - timing_s/save_checkpoint:6.738213421776891 - timing_s/stop_profile:3.814697265625e-05 - timing_per_token_ms/gen:0.20196301814796624 - timing_per_token_ms/update_actor:0.06645925010375094 - timing_per_token_ms/adv:0.0008753132551312797 - perf/total_num_tokens:146386 - perf/time_per_step:25.604432402178645 - perf/throughput:714.6516553299197 (TaskRunner pid=2099163) Training Progress: 90%|█████████ | 90/100 [1:06:32<05:46, 34.67s/it] (TaskRunner pid=2099163) step:91 - global_seqlen/min:13664 - global_seqlen/max:28551 - global_seqlen/minmax_diff:14887 - global_seqlen/balanced_min:19769 - global_seqlen/balanced_max:19785 - global_seqlen/mean:19781.5 - actor/entropy:0.25078386068344116 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43638211488723755 - training/rollout_probs_diff_mean:0.004951483104377985 - training/rollout_probs_diff_std:0.013005034998059273 - training/rollout_actor_probs_pearson_corr:0.9984056949615479 - rollout_corr/rollout_is_mean:0.9997699856758118 - rollout_corr/rollout_is_ratio_fraction_high:2.716062772378791e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.432125544757582e-05 - rollout_corr/rollout_is_max:2.0405869483947754 - rollout_corr/rollout_is_min:0.43364647030830383 - rollout_corr/rollout_is_std:0.0399608314037323 - rollout_corr/rollout_is_eff_sample_size:0.9984049054928523 - rollout_corr/rollout_is_seq_mean:0.9997459053993225 - rollout_corr/rollout_is_seq_std:0.002915602643042803 - rollout_corr/rollout_is_seq_max:1.009341835975647 - rollout_corr/rollout_is_seq_min:0.9896574020385742 - rollout_corr/rollout_is_seq_max_deviation:0.010342597961425781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3241292075254023) - rollout_corr/training_log_ppl:np.float64(0.2761981814110186) - rollout_corr/kl:np.float64(0.0012420743942662682) - rollout_corr/k3_kl:np.float64(0.0012305704847221932) - rollout_corr/rollout_ppl:np.float64(1.3224302087910473) - rollout_corr/rollout_log_ppl:np.float64(0.274956106179161) - rollout_corr/log_ppl_diff:np.float64(0.0012420752318575978) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025912187120411545) - rollout_corr/log_ppl_diff_max:np.float64(0.014826744794845581) - rollout_corr/log_ppl_diff_min:np.float64(-0.007786452770233154) - rollout_corr/ppl_ratio:np.float64(1.0012476784177125) - rollout_corr/chi2_token:np.float64(0.0025545565877109766) - rollout_corr/chi2_seq:np.float64(0.8443398529198021) - actor/pg_loss:np.float64(-0.00023542286362498999) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013123745520715602) - actor/ppo_kl:np.float64(7.698053936522342e-05) - actor/pg_clipfrac_lower:np.float64(1.1660447853500955e-05) - actor/grad_norm:np.float64(0.4663517475128174) - perf/mfu/actor:np.float64(0.05740788145887977) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.15924453735352) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:6 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005907436716370285 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005907436716370285 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:287.640625 - response_length/max:1198.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.640625 - response_length_non_aborted/max:1198.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8155744075775146e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3114315513521433) - timing_s/agent_loop/generate_sequences/max:np.float64(7.799869127571583) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8764019390873727) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.799869127571583) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1198 - timing_s/gen:13.566020984202623 - timing_s/reward:0.05244571715593338 - timing_s/old_log_prob:2.490916972979903 - timing_s/adv:0.13142306357622147 - timing_s/update_actor:9.768587896600366 - timing_s/step:26.121974842622876 - timing_s/stop_profile:0.00011107511818408966 - timing_per_token_ms/gen:0.18423082438213134 - timing_per_token_ms/update_actor:0.06172805333645304 - timing_per_token_ms/adv:0.0008304669993189436 - perf/total_num_tokens:158252 - perf/time_per_step:26.121974842622876 - perf/throughput:757.2742918243223 (TaskRunner pid=2099163) Training Progress: 91%|█████████ | 91/100 [1:06:59<04:49, 32.12s/it] (TaskRunner pid=2099163) step:92 - global_seqlen/min:15474 - global_seqlen/max:25697 - global_seqlen/minmax_diff:10223 - global_seqlen/balanced_min:20472 - global_seqlen/balanced_max:20489 - global_seqlen/mean:20485.25 - actor/entropy:0.25056755542755127 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5584663152694702 - training/rollout_probs_diff_mean:0.004964837804436684 - training/rollout_probs_diff_std:0.013341370038688183 - training/rollout_actor_probs_pearson_corr:0.9983838200569153 - rollout_corr/rollout_is_mean:0.9999274015426636 - rollout_corr/rollout_is_ratio_fraction_high:4.238126348354854e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.650835009873845e-05 - rollout_corr/rollout_is_max:3.1624722480773926 - rollout_corr/rollout_is_min:0.20737510919570923 - rollout_corr/rollout_is_std:0.0406433641910553 - rollout_corr/rollout_is_eff_sample_size:0.9983503658269968 - rollout_corr/rollout_is_seq_mean:0.9998391270637512 - rollout_corr/rollout_is_seq_std:0.0029326851945370436 - rollout_corr/rollout_is_seq_max:1.0081546306610107 - rollout_corr/rollout_is_seq_min:0.9876515865325928 - rollout_corr/rollout_is_seq_max_deviation:0.012348413467407227 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3244820055551827) - rollout_corr/training_log_ppl:np.float64(0.27609299635514617) - rollout_corr/kl:np.float64(0.0011841020946801173) - rollout_corr/k3_kl:np.float64(0.0013603362282310627) - rollout_corr/rollout_ppl:np.float64(1.3228559507988393) - rollout_corr/rollout_log_ppl:np.float64(0.2749088934651809) - rollout_corr/log_ppl_diff:np.float64(0.0011841028899652883) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026790284464368597) - rollout_corr/log_ppl_diff_max:np.float64(0.01207616925239563) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074624717235565186) - rollout_corr/ppl_ratio:np.float64(1.0011904705315828) - rollout_corr/chi2_token:np.float64(0.0030524577014148235) - rollout_corr/chi2_seq:np.float64(0.5289049397688359) - actor/pg_loss:np.float64(0.00013589137233793736) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002009198718951666) - actor/ppo_kl:np.float64(3.7602182644036475e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.613745704293251) - perf/mfu/actor:np.float64(0.05991881909984166) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.0025405883789) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:6 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012523663230240345 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012523663230240345 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:276.5078125 - response_length/max:1210.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:276.5078125 - response_length_non_aborted/max:1210.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:363.65625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001337248831987381 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3419308010488749) - timing_s/agent_loop/generate_sequences/max:np.float64(7.376888804137707) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7873674428483355) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.376888804137707) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:1210 - timing_s/gen:13.59444515965879 - timing_s/reward:0.05653318390250206 - timing_s/old_log_prob:2.394727673381567 - timing_s/adv:0.11553080752491951 - timing_s/update_actor:9.629633232951164 - timing_s/step:25.88193485327065 - timing_s/stop_profile:0.00010392814874649048 - timing_per_token_ms/gen:0.1920499132548638 - timing_per_token_ms/update_actor:0.05875955402637974 - timing_per_token_ms/adv:0.0007049633731887548 - perf/total_num_tokens:163882 - perf/time_per_step:25.88193485327065 - perf/throughput:791.4883534069061 (TaskRunner pid=2099163) Training Progress: 92%|█████████▏| 92/100 [1:07:25<04:02, 30.27s/it] (TaskRunner pid=2099163) step:93 - global_seqlen/min:13545 - global_seqlen/max:25655 - global_seqlen/minmax_diff:12110 - global_seqlen/balanced_min:19112 - global_seqlen/balanced_max:19881 - global_seqlen/mean:19214.125 - actor/entropy:0.23102349042892456 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3046841025352478 - training/rollout_probs_diff_mean:0.004829892423003912 - training/rollout_probs_diff_std:0.013052316382527351 - training/rollout_actor_probs_pearson_corr:0.9983353018760681 - rollout_corr/rollout_is_mean:0.9999238848686218 - rollout_corr/rollout_is_ratio_fraction_high:1.537870048196055e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.689350604778156e-05 - rollout_corr/rollout_is_max:2.5575833320617676 - rollout_corr/rollout_is_min:0.3091418743133545 - rollout_corr/rollout_is_std:0.03926188126206398 - rollout_corr/rollout_is_eff_sample_size:0.9984606395421983 - rollout_corr/rollout_is_seq_mean:0.9999052882194519 - rollout_corr/rollout_is_seq_std:0.003070805687457323 - rollout_corr/rollout_is_seq_max:1.0085487365722656 - rollout_corr/rollout_is_seq_min:0.9925494194030762 - rollout_corr/rollout_is_seq_max_deviation:0.008548736572265625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.298731068149209) - rollout_corr/training_log_ppl:np.float64(0.2568639310920844) - rollout_corr/kl:np.float64(0.001255048227367439) - rollout_corr/k3_kl:np.float64(0.001259192493506589) - rollout_corr/rollout_ppl:np.float64(1.297051596455276) - rollout_corr/rollout_log_ppl:np.float64(0.25560888355539646) - rollout_corr/log_ppl_diff:np.float64(0.0012550475366879255) - rollout_corr/log_ppl_abs_diff:np.float64(0.002867596718715504) - rollout_corr/log_ppl_diff_max:np.float64(0.02118036150932312) - rollout_corr/log_ppl_diff_min:np.float64(-0.007840782403945923) - rollout_corr/ppl_ratio:np.float64(1.0012623802758753) - rollout_corr/chi2_token:np.float64(0.002499846974387765) - rollout_corr/chi2_seq:np.float64(0.36207995703443885) - actor/pg_loss:np.float64(-4.324305336922407e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018963009379149298) - actor/ppo_kl:np.float64(0.0002913880196118157) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6442021429538727) - perf/mfu/actor:np.float64(0.05641402480565709) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.02770233154297) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:6 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014271434396505356 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014271434396505356 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:254.00390625 - response_length/max:2020.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:254.00390625 - response_length_non_aborted/max:2020.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012409500777721405 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9811534341424704) - timing_s/agent_loop/generate_sequences/max:np.float64(11.214596100151539) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4594217599369586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.214596100151539) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2020 - timing_s/gen:17.094698660075665 - timing_s/reward:0.04627825319766998 - timing_s/old_log_prob:2.2830312568694353 - timing_s/adv:0.11555880308151245 - timing_s/update_actor:9.736344682052732 - timing_s/step:29.311943497508764 - timing_s/stop_profile:0.00011210516095161438 - timing_per_token_ms/gen:0.2628942508277688 - timing_per_token_ms/update_actor:0.06334106212260987 - timing_per_token_ms/adv:0.0007517828881194984 - perf/total_num_tokens:153713 - perf/time_per_step:29.311943497508764 - perf/throughput:655.5049821801484 (TaskRunner pid=2099163) Training Progress: 93%|█████████▎| 93/100 [1:07:54<03:29, 29.99s/it] (TaskRunner pid=2099163) step:94 - global_seqlen/min:11634 - global_seqlen/max:21223 - global_seqlen/minmax_diff:9589 - global_seqlen/balanced_min:15316 - global_seqlen/balanced_max:15322 - global_seqlen/mean:15319.375 - actor/entropy:0.2614341378211975 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2891600728034973 - training/rollout_probs_diff_mean:0.005587474443018436 - training/rollout_probs_diff_std:0.013904578052461147 - training/rollout_actor_probs_pearson_corr:0.9982290267944336 - rollout_corr/rollout_is_mean:1.0003408193588257 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0101715563214384e-05 - rollout_corr/rollout_is_max:1.7715976238250732 - rollout_corr/rollout_is_min:0.2889171540737152 - rollout_corr/rollout_is_std:0.04165585711598396 - rollout_corr/rollout_is_eff_sample_size:0.9982688644919676 - rollout_corr/rollout_is_seq_mean:1.0004125833511353 - rollout_corr/rollout_is_seq_std:0.0032136309891939163 - rollout_corr/rollout_is_seq_max:1.0084282159805298 - rollout_corr/rollout_is_seq_min:0.9903004765510559 - rollout_corr/rollout_is_seq_max_deviation:0.009699523448944092 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2973201628774405) - rollout_corr/training_log_ppl:np.float64(0.25612859573448077) - rollout_corr/kl:np.float64(0.0017466846469749164) - rollout_corr/k3_kl:np.float64(0.0015775575718635082) - rollout_corr/rollout_ppl:np.float64(1.2949957121163607) - rollout_corr/rollout_log_ppl:np.float64(0.2543819086276926) - rollout_corr/log_ppl_diff:np.float64(0.0017466871067881584) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035796407610177994) - rollout_corr/log_ppl_diff_max:np.float64(0.01710490882396698) - rollout_corr/log_ppl_diff_min:np.float64(-0.010113626718521118) - rollout_corr/ppl_ratio:np.float64(1.0017579961568117) - rollout_corr/chi2_token:np.float64(0.0027468986809253693) - rollout_corr/chi2_seq:np.float64(0.5400724075734615) - actor/pg_loss:np.float64(-0.0003080737078562379) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0026132785301342665) - actor/ppo_kl:np.float64(0.0012474513576563595) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6366036757826805) - perf/mfu/actor:np.float64(0.044733689603173685) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.00236892700195) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:6 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008616097271442413 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008616097271442413 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:194.32421875 - response_length/max:710.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:194.32421875 - response_length_non_aborted/max:710.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:284.40625 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011759810149669647 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1009019035845995) - timing_s/agent_loop/generate_sequences/max:np.float64(4.770813453942537) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1518594626468257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.770813453942537) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:670 - timing_s/gen:10.57761050015688 - timing_s/reward:0.05101897194981575 - timing_s/old_log_prob:2.3127746544778347 - timing_s/adv:0.1194564662873745 - timing_s/update_actor:9.648187587037683 - timing_s/step:22.79769565165043 - timing_s/stop_profile:5.702488124370575e-05 - timing_per_token_ms/gen:0.21262810823078537 - timing_per_token_ms/update_actor:0.07872536891222458 - timing_per_token_ms/adv:0.000974717198705679 - perf/total_num_tokens:122555 - perf/time_per_step:22.79769565165043 - perf/throughput:671.9703269172716 (TaskRunner pid=2099163) Training Progress: 94%|█████████▍| 94/100 [1:08:17<02:47, 27.84s/it] (TaskRunner pid=2099163) step:95 - global_seqlen/min:11894 - global_seqlen/max:19955 - global_seqlen/minmax_diff:8061 - global_seqlen/balanced_min:16319 - global_seqlen/balanced_max:16327 - global_seqlen/mean:16323.0 - actor/entropy:0.2705520987510681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6408352255821228 - training/rollout_probs_diff_mean:0.005575786344707012 - training/rollout_probs_diff_std:0.013862286694347858 - training/rollout_actor_probs_pearson_corr:0.9983189105987549 - rollout_corr/rollout_is_mean:1.0001085996627808 - rollout_corr/rollout_is_ratio_fraction_high:5.360972136259079e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.360972136259079e-05 - rollout_corr/rollout_is_max:4.217565536499023 - rollout_corr/rollout_is_min:0.15521328151226044 - rollout_corr/rollout_is_std:0.04240885004401207 - rollout_corr/rollout_is_eff_sample_size:0.9982049558450077 - rollout_corr/rollout_is_seq_mean:1.0000663995742798 - rollout_corr/rollout_is_seq_std:0.0035495145712047815 - rollout_corr/rollout_is_seq_max:1.0250022411346436 - rollout_corr/rollout_is_seq_min:0.989392101764679 - rollout_corr/rollout_is_seq_max_deviation:0.025002241134643555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3268689936958253) - rollout_corr/training_log_ppl:np.float64(0.2789607130398508) - rollout_corr/kl:np.float64(0.001470564366549354) - rollout_corr/k3_kl:np.float64(0.001639172450438764) - rollout_corr/rollout_ppl:np.float64(1.324940457008779) - rollout_corr/rollout_log_ppl:np.float64(0.27749014867004007) - rollout_corr/log_ppl_diff:np.float64(0.0014705643698107451) - rollout_corr/log_ppl_abs_diff:np.float64(0.003353676962433383) - rollout_corr/log_ppl_diff_max:np.float64(0.019685059785842896) - rollout_corr/log_ppl_diff_min:np.float64(-0.014085322618484497) - rollout_corr/ppl_ratio:np.float64(1.0014813262969255) - rollout_corr/chi2_token:np.float64(0.0037556993775069714) - rollout_corr/chi2_seq:np.float64(0.9716611602343619) - actor/pg_loss:np.float64(0.0004548488650470972) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002927706459104229) - actor/ppo_kl:np.float64(0.0005539002601722132) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7531055063009262) - perf/mfu/actor:np.float64(0.048065567875728424) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.10460662841797) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:6 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015368120511993766 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0015368120511993766 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:218.59375 - response_length/max:779.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:218.59375 - response_length_non_aborted/max:779.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.5 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.889948129653931e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0940031111240387) - timing_s/agent_loop/generate_sequences/max:np.float64(5.365459702908993) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3305309581846814) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.365459702908993) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:779 - timing_s/gen:11.15835091844201 - timing_s/reward:0.05341525003314018 - timing_s/old_log_prob:2.3560058772563934 - timing_s/adv:0.13407864421606064 - timing_s/update_actor:9.62122217938304 - timing_s/step:23.406750781461596 - timing_s/stop_profile:0.00011238828301429749 - timing_per_token_ms/gen:0.19939869403934973 - timing_per_token_ms/update_actor:0.07367841526820315 - timing_per_token_ms/adv:0.0010267616569875379 - perf/total_num_tokens:130584 - perf/time_per_step:23.406750781461596 - perf/throughput:697.3629168952401 (TaskRunner pid=2099163) Training Progress: 95%|█████████▌| 95/100 [1:08:40<02:12, 26.53s/it] (TaskRunner pid=2099163) step:96 - global_seqlen/min:11717 - global_seqlen/max:21275 - global_seqlen/minmax_diff:9558 - global_seqlen/balanced_min:16664 - global_seqlen/balanced_max:16674 - global_seqlen/mean:16669.875 - actor/entropy:0.25728729367256165 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2606017589569092 - training/rollout_probs_diff_mean:0.005339381285011768 - training/rollout_probs_diff_std:0.013366999104619026 - training/rollout_actor_probs_pearson_corr:0.9983529448509216 - rollout_corr/rollout_is_mean:0.9999350309371948 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6451245546340942 - rollout_corr/rollout_is_min:0.5036647319793701 - rollout_corr/rollout_is_std:0.04145504906773567 - rollout_corr/rollout_is_eff_sample_size:0.9982843083294409 - rollout_corr/rollout_is_seq_mean:0.9999916553497314 - rollout_corr/rollout_is_seq_std:0.0029035070911049843 - rollout_corr/rollout_is_seq_max:1.008498191833496 - rollout_corr/rollout_is_seq_min:0.9916707277297974 - rollout_corr/rollout_is_seq_max_deviation:0.008498191833496094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3080988936126232) - rollout_corr/training_log_ppl:np.float64(0.2644939655147027) - rollout_corr/kl:np.float64(0.0013578184077687183) - rollout_corr/k3_kl:np.float64(0.001467021100893362) - rollout_corr/rollout_ppl:np.float64(1.3062553596682847) - rollout_corr/rollout_log_ppl:np.float64(0.2631361448438838) - rollout_corr/log_ppl_diff:np.float64(0.00135782067081891) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032226947078015655) - rollout_corr/log_ppl_diff_max:np.float64(0.01374003291130066) - rollout_corr/log_ppl_diff_min:np.float64(-0.008670985698699951) - rollout_corr/ppl_ratio:np.float64(1.0013662767596543) - rollout_corr/chi2_token:np.float64(0.003215978853404522) - rollout_corr/chi2_seq:np.float64(0.44253751053474844) - actor/pg_loss:np.float64(0.00011486734729260206) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020753747680828383) - actor/ppo_kl:np.float64(0.00043077189779694436) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6561485677957535) - perf/mfu/actor:np.float64(0.04976022063857087) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.02475357055664) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:6 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014713210053741932 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014713210053741932 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:219.49609375 - response_length/max:856.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:219.49609375 - response_length_non_aborted/max:856.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:301.4375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013059936463832855 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2289475314319134) - timing_s/agent_loop/generate_sequences/max:np.float64(5.740047561004758) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.310526862871484) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.740047561004758) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:856 - timing_s/gen:11.425192607566714 - timing_s/reward:0.051955023780465126 - timing_s/old_log_prob:2.3286557272076607 - timing_s/adv:0.12474869564175606 - timing_s/update_actor:9.438106391578913 - timing_s/step:23.46123932301998 - timing_s/stop_profile:3.103911876678467e-05 - timing_per_token_ms/gen:0.2033278035195443 - timing_per_token_ms/update_actor:0.07077217429329039 - timing_per_token_ms/adv:0.0009354351460475563 - perf/total_num_tokens:133359 - perf/time_per_step:23.46123932301998 - perf/throughput:710.5283216493875 (TaskRunner pid=2099163) Training Progress: 96%|█████████▌| 96/100 [1:09:04<01:42, 25.61s/it] (TaskRunner pid=2099163) step:97 - global_seqlen/min:10859 - global_seqlen/max:28193 - global_seqlen/minmax_diff:17334 - global_seqlen/balanced_min:17081 - global_seqlen/balanced_max:23683 - global_seqlen/mean:18237.375 - actor/entropy:0.22356657683849335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35336679220199585 - training/rollout_probs_diff_mean:0.004439533222466707 - training/rollout_probs_diff_std:0.01241324283182621 - training/rollout_actor_probs_pearson_corr:0.998500406742096 - rollout_corr/rollout_is_mean:1.0001307725906372 - rollout_corr/rollout_is_ratio_fraction_high:2.887044320232235e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4435221601161174e-05 - rollout_corr/rollout_is_max:3.0604419708251953 - rollout_corr/rollout_is_min:0.4321064054965973 - rollout_corr/rollout_is_std:0.03731440007686615 - rollout_corr/rollout_is_eff_sample_size:0.9986098092374474 - rollout_corr/rollout_is_seq_mean:1.000051498413086 - rollout_corr/rollout_is_seq_std:0.0030096296686679125 - rollout_corr/rollout_is_seq_max:1.0088967084884644 - rollout_corr/rollout_is_seq_min:0.9923568964004517 - rollout_corr/rollout_is_seq_max_deviation:0.008896708488464355 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.32025858014822) - rollout_corr/training_log_ppl:np.float64(0.2736287588450068) - rollout_corr/kl:np.float64(0.001147672094734986) - rollout_corr/k3_kl:np.float64(0.0012670807614085788) - rollout_corr/rollout_ppl:np.float64(1.318727934267372) - rollout_corr/rollout_log_ppl:np.float64(0.27248108579806285) - rollout_corr/log_ppl_diff:np.float64(0.0011476730469439644) - rollout_corr/log_ppl_abs_diff:np.float64(0.002580658172519179) - rollout_corr/log_ppl_diff_max:np.float64(0.010686814785003662) - rollout_corr/log_ppl_diff_min:np.float64(-0.00808987021446228) - rollout_corr/ppl_ratio:np.float64(1.0011534958612174) - rollout_corr/chi2_token:np.float64(0.002894336823374033) - rollout_corr/chi2_seq:np.float64(0.502729723462835) - actor/pg_loss:np.float64(1.6757636331021786e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014571902823945493) - actor/ppo_kl:np.float64(0.0002972853384548557) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5494845286011696) - perf/mfu/actor:np.float64(0.050740963915102175) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.98857498168945) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:6 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.03918260708451271 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.03918260708451271 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:270.60546875 - response_length/max:8192.0 - response_length/min:88.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:270.60546875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:299.3125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5369226932525635e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.058408448472619) - timing_s/agent_loop/generate_sequences/max:np.float64(49.16501975432038) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.604979175426706) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.16501975432038) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:179 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.1183186173439 - timing_s/reward:0.05663715489208698 - timing_s/old_log_prob:2.494466742500663 - timing_s/adv:0.13287332467734814 - timing_s/update_actor:10.396746782585979 - timing_s/step:68.28716198541224 - timing_s/stop_profile:0.00018381327390670776 - timing_per_token_ms/gen:0.7956451622857293 - timing_per_token_ms/update_actor:0.07125989062698153 - timing_per_token_ms/adv:0.0009107212844320259 - perf/total_num_tokens:145899 - perf/time_per_step:68.28716198541224 - perf/throughput:267.06886726228186 (TaskRunner pid=2099163) Training Progress: 97%|█████████▋| 97/100 [1:10:12<01:55, 38.43s/it] (TaskRunner pid=2099163) step:98 - global_seqlen/min:10881 - global_seqlen/max:21856 - global_seqlen/minmax_diff:10975 - global_seqlen/balanced_min:16595 - global_seqlen/balanced_max:16744 - global_seqlen/mean:16625.0 - actor/entropy:0.27939262986183167 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2851577401161194 - training/rollout_probs_diff_mean:0.00533874798566103 - training/rollout_probs_diff_std:0.01307785976678133 - training/rollout_actor_probs_pearson_corr:0.9984779357910156 - rollout_corr/rollout_is_mean:0.9997860789299011 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.518648736644536e-05 - rollout_corr/rollout_is_max:1.6622998714447021 - rollout_corr/rollout_is_min:0.3280748724937439 - rollout_corr/rollout_is_std:0.040275804698467255 - rollout_corr/rollout_is_eff_sample_size:0.9983798331417372 - rollout_corr/rollout_is_seq_mean:0.999704897403717 - rollout_corr/rollout_is_seq_std:0.0033357173670083284 - rollout_corr/rollout_is_seq_max:1.0101433992385864 - rollout_corr/rollout_is_seq_min:0.9884958267211914 - rollout_corr/rollout_is_seq_max_deviation:0.011504173278808594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3345653787255287) - rollout_corr/training_log_ppl:np.float64(0.2840118541062111) - rollout_corr/kl:np.float64(0.0016829417983466044) - rollout_corr/k3_kl:np.float64(0.0013737696041289382) - rollout_corr/rollout_ppl:np.float64(1.3323034564964473) - rollout_corr/rollout_log_ppl:np.float64(0.282328910601791) - rollout_corr/log_ppl_diff:np.float64(0.001682943504420109) - rollout_corr/log_ppl_abs_diff:np.float64(0.003258420663769357) - rollout_corr/log_ppl_diff_max:np.float64(0.02178385853767395) - rollout_corr/log_ppl_diff_min:np.float64(-0.009423941373825073) - rollout_corr/ppl_ratio:np.float64(1.0016928501427174) - rollout_corr/chi2_token:np.float64(0.002250493736937642) - rollout_corr/chi2_seq:np.float64(0.3969768069218844) - actor/pg_loss:np.float64(0.00017058837693184614) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019098859092991916) - actor/ppo_kl:np.float64(0.0005005859824507297) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6195161938667297) - perf/mfu/actor:np.float64(0.048713654755540986) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(104.02078247070312) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:6 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020672061946243048 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020672061946243048 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:222.03125 - response_length/max:1144.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:222.03125 - response_length_non_aborted/max:1144.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013743340969085693 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1269784551113844) - timing_s/agent_loop/generate_sequences/max:np.float64(7.022609854117036) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2723799158920883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.022609854117036) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1144 - timing_s/gen:12.626804063096642 - timing_s/reward:0.05744520016014576 - timing_s/old_log_prob:2.334137598052621 - timing_s/adv:0.12326470203697681 - timing_s/update_actor:9.706466360017657 - timing_s/step:24.93613464012742 - timing_s/stop_profile:0.00017854012548923492 - timing_per_token_ms/gen:0.22214644727474742 - timing_per_token_ms/update_actor:0.07298095007532072 - timing_per_token_ms/adv:0.0009268022709547129 - perf/total_num_tokens:133000 - perf/time_per_step:24.93613464012742 - perf/throughput:666.703169513968 (TaskRunner pid=2099163) Training Progress: 98%|█████████▊| 98/100 [1:10:37<01:08, 34.39s/it] (TaskRunner pid=2099163) step:99 - global_seqlen/min:10950 - global_seqlen/max:22756 - global_seqlen/minmax_diff:11806 - global_seqlen/balanced_min:15802 - global_seqlen/balanced_max:15820 - global_seqlen/mean:15813.5 - actor/entropy:0.2819356322288513 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7740305066108704 - training/rollout_probs_diff_mean:0.005276948679238558 - training/rollout_probs_diff_std:0.013337536714971066 - training/rollout_actor_probs_pearson_corr:0.9984665513038635 - rollout_corr/rollout_is_mean:1.0000571012496948 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.5403706937795505e-05 - rollout_corr/rollout_is_max:1.6278910636901855 - rollout_corr/rollout_is_min:0.07193134725093842 - rollout_corr/rollout_is_std:0.040468472987413406 - rollout_corr/rollout_is_eff_sample_size:0.9983652180867625 - rollout_corr/rollout_is_seq_mean:1.0001047849655151 - rollout_corr/rollout_is_seq_std:0.003330203937366605 - rollout_corr/rollout_is_seq_max:1.0106587409973145 - rollout_corr/rollout_is_seq_min:0.9881437420845032 - rollout_corr/rollout_is_seq_max_deviation:0.011856257915496826 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3568300306797028) - rollout_corr/training_log_ppl:np.float64(0.3001139621483162) - rollout_corr/kl:np.float64(0.0011941435956686064) - rollout_corr/k3_kl:np.float64(0.0011522117289359812) - rollout_corr/rollout_ppl:np.float64(1.3551504230126739) - rollout_corr/rollout_log_ppl:np.float64(0.29891981728724204) - rollout_corr/log_ppl_diff:np.float64(0.0011941448610741645) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030851933697704226) - rollout_corr/log_ppl_diff_max:np.float64(0.027537941932678223) - rollout_corr/log_ppl_diff_min:np.float64(-0.007955223321914673) - rollout_corr/ppl_ratio:np.float64(1.0012031164951622) - rollout_corr/chi2_token:np.float64(0.0022564970422536135) - rollout_corr/chi2_seq:np.float64(0.5483720011543483) - actor/pg_loss:np.float64(0.0006328735034912825) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009942931160367152) - actor/ppo_kl:np.float64(0.0003664185717591195) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5217531993985176) - perf/mfu/actor:np.float64(0.046482210549823294) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.91424560546875) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:7 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011117677204310894 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011117677204310894 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:211.515625 - response_length/max:1026.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:211.515625 - response_length_non_aborted/max:1026.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.65625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0004795752465724945 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0314996894448996) - timing_s/agent_loop/generate_sequences/max:np.float64(6.317301785573363) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.143986992428836) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.317301785573363) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:1026 - timing_s/gen:12.40069780871272 - timing_s/reward:0.04780052416026592 - timing_s/old_log_prob:2.5466749370098114 - timing_s/adv:0.11402323842048645 - timing_s/update_actor:9.808757556602359 - timing_s/step:25.033864453434944 - timing_s/stop_profile:0.00011504441499710083 - timing_per_token_ms/gen:0.2290148815969698 - timing_per_token_ms/update_actor:0.07753468204858474 - timing_per_token_ms/adv:0.0009013124736813992 - perf/total_num_tokens:126508 - perf/time_per_step:25.033864453434944 - perf/throughput:631.6843342111409 (TaskRunner pid=2099163) Training Progress: 99%|█████████▉| 99/100 [1:11:03<00:31, 31.74s/it] (TaskRunner pid=2099163) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2099163) validation generation end (TaskRunner pid=2099163) [prompt] system (TaskRunner pid=2099163) (TaskRunner pid=2099163) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) ... (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2099163) (TaskRunner pid=2099163) A (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) user (TaskRunner pid=2099163) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2099163) (TaskRunner pid=2099163) A: -0.01 (TaskRunner pid=2099163) B: 1.69 (TaskRunner pid=2099163) C: 2.49 (TaskRunner pid=2099163) D: 0.45 (TaskRunner pid=2099163) Please reason step by step. (TaskRunner pid=2099163) assistant (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) [response] (TaskRunner pid=2099163) The folding stability score for a given protein sequence can be determined using computational tools such as I-TAFOR, which predicts the stability of a protein's folded structure based on its sequence. The specific sequence provided, 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR', would be input into such a tool to calculate the folding stability score. Based on computational predictions, the folding stability score for this sequence is known to be 0.45. (TaskRunner pid=2099163) (TaskRunner pid=2099163) (TaskRunner pid=2099163) D (TaskRunner pid=2099163) (TaskRunner pid=2099163) [ground_truth] B (TaskRunner pid=2099163) [score] 0.0 (TaskRunner pid=2099163) [acc] 0.0 (TaskRunner pid=2099163) [pred] D (TaskRunner pid=2099163) [incorrect_format] 1 (TaskRunner pid=2099163) [feedback] (TaskRunner pid=2099163) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (TaskRunner pid=2099163) (WorkerDict pid=2099539) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2099163) step:100 - global_seqlen/min:13194 - global_seqlen/max:24171 - global_seqlen/minmax_diff:10977 - global_seqlen/balanced_min:19146 - global_seqlen/balanced_max:19154 - global_seqlen/mean:19150.75 - actor/entropy:0.2835403382778168 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3625403642654419 - training/rollout_probs_diff_mean:0.005042023025453091 - training/rollout_probs_diff_std:0.012832351960241795 - training/rollout_actor_probs_pearson_corr:0.9986383318901062 - rollout_corr/rollout_is_mean:1.000214695930481 - rollout_corr/rollout_is_ratio_fraction_high:3.143171488773078e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.7147572331596166e-05 - rollout_corr/rollout_is_max:4.838179588317871 - rollout_corr/rollout_is_min:0.209492027759552 - rollout_corr/rollout_is_std:0.040638964623212814 - rollout_corr/rollout_is_eff_sample_size:0.9983516728081235 - rollout_corr/rollout_is_seq_mean:1.000152587890625 - rollout_corr/rollout_is_seq_std:0.002947622211650014 - rollout_corr/rollout_is_seq_max:1.0129314661026 - rollout_corr/rollout_is_seq_min:0.9921380877494812 - rollout_corr/rollout_is_seq_max_deviation:0.012931466102600098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3729312745854259) - rollout_corr/training_log_ppl:np.float64(0.3114306655479595) - rollout_corr/kl:np.float64(0.0015098723376403278) - rollout_corr/k3_kl:np.float64(0.001307048618286899) - rollout_corr/rollout_ppl:np.float64(1.3707687840797007) - rollout_corr/rollout_log_ppl:np.float64(0.30992078845156357) - rollout_corr/log_ppl_diff:np.float64(0.0015098770963959396) - rollout_corr/log_ppl_abs_diff:np.float64(0.003156350867357105) - rollout_corr/log_ppl_diff_max:np.float64(0.014475628733634949) - rollout_corr/log_ppl_diff_min:np.float64(-0.009900987148284912) - rollout_corr/ppl_ratio:np.float64(1.0015187887474895) - rollout_corr/chi2_token:np.float64(0.0023139831610023975) - rollout_corr/chi2_seq:np.float64(0.866482462733984) - actor/pg_loss:np.float64(5.068804603070021e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010578534675005358) - actor/ppo_kl:np.float64(0.0007291194974783366) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.496917188167572) - perf/mfu/actor:np.float64(0.056309282462256305) - perf/max_memory_allocated_gb:np.float64(123.67111110687256) - perf/max_memory_reserved_gb:np.float64(129.48046875) - perf/cpu_memory_used_gb:np.float64(103.95842933654785) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.42625) - val-aux/sciknoweval/reward/std@16:np.float64(0.20340466932397192) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.51122) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1717994688879882) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.34018) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17528803531142753) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.42506) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2031877981200387) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5764) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12262009307652777) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.26536) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12852696069401104) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.43562000000000006) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.16364220520548403) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.62372) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08028605088755732) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.21248) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07648178927728648) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4453000000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1155248617088842) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6543800000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0472167585635409) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.18634) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.032643145410378605) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.44676000000000005) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07340249583082686) - val-aux/sciknoweval/score/mean@16:np.float64(0.42625) - val-aux/sciknoweval/score/std@16:np.float64(0.20340466932397192) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.51122) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1717994688879882) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.34018) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17528803531142753) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.42506) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2031877981200387) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5764) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12262009307652777) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.26536) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12852696069401104) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.43562000000000006) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.16364220520548403) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.62372) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08028605088755732) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.21248) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07648178927728648) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4453000000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1155248617088842) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6543800000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0472167585635409) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.18634) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.032643145410378605) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.44676000000000005) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07340249583082686) - val-core/sciknoweval/acc/mean@16:np.float64(0.42625) - val-aux/sciknoweval/acc/std@16:np.float64(0.20340466932397192) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.51122) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1717994688879882) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.34018) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17528803531142753) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.42506) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2031877981200387) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5764) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12262009307652777) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.26536) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12852696069401104) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.43562000000000006) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.16364220520548403) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.62372) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08028605088755732) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.21248) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07648178927728648) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4453000000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1155248617088842) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6543800000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0472167585635409) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.18634) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.032643145410378605) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.44676000000000005) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07340249583082686) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9975) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.009682458365518542) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9997800000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0029552123640712146) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9955799999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.012525805206779563) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99798) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.008749306906094262) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9915) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.01634315511370097) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9993599999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.004842664831259046) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.98444) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.019500890160644296) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99984) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.002504474311076614) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9751599999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.019403379135729548) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:7 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003972182981669903 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003972182981669903 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:248.5546875 - response_length/max:977.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:248.5546875 - response_length_non_aborted/max:977.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.90625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.883567690849304e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7739909812808037) - timing_s/agent_loop/generate_sequences/max:np.float64(6.653986681252718) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1721716843530885) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.653986681252718) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:977 - timing_s/gen:12.76038683950901 - timing_s/reward:0.05503258667886257 - timing_s/old_log_prob:2.375061484053731 - timing_s/adv:0.11991865560412407 - timing_s/update_actor:9.685975370928645 - timing_s/step:25.090152885764837 - timing_s/testing:66.87851952016354 - timing_s/save_checkpoint:6.8637774139642715 - timing_s/stop_profile:0.00011190585792064667 - timing_per_token_ms/gen:0.20054041866272215 - timing_per_token_ms/update_actor:0.06322190626299652 - timing_per_token_ms/adv:0.000782728193439709 - perf/total_num_tokens:153206 - perf/time_per_step:25.090152885764837 - perf/throughput:763.2775331100266 (TaskRunner pid=2099163) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2099163) "np.float64(0.42625), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2099163) "np.float64(0.20340466932397192), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2099163) "np.float64(0.51122), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2099163) "np.float64(0.1717994688879882), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2099163) "np.float64(0.34018), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2099163) "np.float64(0.17528803531142753), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2099163) "np.float64(0.42506), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2099163) "np.float64(0.2031877981200387), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2099163) "np.float64(0.5764), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2099163) "np.float64(0.12262009307652777), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2099163) "np.float64(0.26536), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2099163) "np.float64(0.12852696069401104), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2099163) "np.float64(0.43562000000000006), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2099163) "np.float64(0.16364220520548403), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2099163) "np.float64(0.62372), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2099163) "np.float64(0.08028605088755732), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2099163) "np.float64(0.21248), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2099163) "np.float64(0.07648178927728648), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2099163) "np.float64(0.4453000000000001), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2099163) "np.float64(0.1155248617088842), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2099163) "np.float64(0.6543800000000001), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2099163) "np.float64(0.0472167585635409), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2099163) "np.float64(0.18634), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2099163) "np.float64(0.032643145410378605), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2099163) "np.float64(0.44676000000000005), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2099163) "np.float64(0.07340249583082686), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2099163) "np.float64(0.42625), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2099163) "np.float64(0.20340466932397192), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2099163) "np.float64(0.51122), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2099163) "np.float64(0.1717994688879882), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2099163) "np.float64(0.34018), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2099163) "np.float64(0.17528803531142753), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2099163) "np.float64(0.42506), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2099163) "np.float64(0.2031877981200387), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2099163) "np.float64(0.5764), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2099163) "np.float64(0.12262009307652777), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2099163) "np.float64(0.26536), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2099163) "np.float64(0.12852696069401104), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2099163) "np.float64(0.43562000000000006), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2099163) "np.float64(0.16364220520548403), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2099163) "np.float64(0.62372), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2099163) "np.float64(0.08028605088755732), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2099163) "np.float64(0.21248), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2099163) "np.float64(0.07648178927728648), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2099163) "np.float64(0.4453000000000001), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2099163) "np.float64(0.1155248617088842), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2099163) "np.float64(0.6543800000000001), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2099163) "np.float64(0.0472167585635409), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2099163) "np.float64(0.18634), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2099163) "np.float64(0.032643145410378605), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2099163) "np.float64(0.44676000000000005), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2099163) "np.float64(0.07340249583082686), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2099163) "np.float64(0.42625), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2099163) "np.float64(0.20340466932397192), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2099163) "np.float64(0.51122), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2099163) "np.float64(0.1717994688879882), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2099163) "np.float64(0.34018), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2099163) "np.float64(0.17528803531142753), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2099163) "np.float64(0.42506), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2099163) "np.float64(0.2031877981200387), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2099163) "np.float64(0.5764), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2099163) "np.float64(0.12262009307652777), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2099163) "np.float64(0.26536), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2099163) "np.float64(0.12852696069401104), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2099163) "np.float64(0.43562000000000006), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2099163) "np.float64(0.16364220520548403), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2099163) "np.float64(0.62372), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2099163) "np.float64(0.08028605088755732), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2099163) "np.float64(0.21248), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2099163) "np.float64(0.07648178927728648), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2099163) "np.float64(0.4453000000000001), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2099163) "np.float64(0.1155248617088842), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2099163) "np.float64(0.6543800000000001), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2099163) "np.float64(0.0472167585635409), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2099163) "np.float64(0.18634), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2099163) "np.float64(0.032643145410378605), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2099163) "np.float64(0.44676000000000005), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2099163) 'np.float64(0.07340249583082686), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9975), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2099163) 'np.float64(0.009682458365518542), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2099163) 'np.float64(0.9997800000000001), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2099163) 'np.float64(0.0029552123640712146), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2099163) 'np.float64(0.9955799999999999), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2099163) 'np.float64(0.012525805206779563), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.99798), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2099163) 'np.float64(0.008749306906094262), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.9915), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2099163) 'np.float64(0.01634315511370097), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2099163) 'np.float64(0.9993599999999999), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2099163) 'np.float64(0.004842664831259046), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2099163) (TaskRunner pid=2099163) Training Progress: 100%|██████████| 100/100 [1:12:41<00:00, 51.89s/it] (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.98444), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2099163) 'np.float64(0.019500890160644296), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(0.99984), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2099163) 'np.float64(0.002504474311076614), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2099163) 'np.float64(0.9751599999999999), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2099163) 'np.float64(0.019403379135729548), ' (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2099163) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2099163) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2099163) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2099163) Training Progress: 100%|██████████| 100/100 [1:12:41<00:00, 43.62s/it] (TaskRunner pid=2099163) wandb: updating run metadata (TaskRunner pid=2099163) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2099163) wandb: uploading output.log (TaskRunner pid=2099163) wandb: uploading history steps 98-99, summary, console lines 546-667 (TaskRunner pid=2099163) wandb: (TaskRunner pid=2099163) wandb: Run history: (TaskRunner pid=2099163) wandb: actor/entropy ▂▄▂▁▃▁▃▃▄▄▄▃▄▄▄▄▃▄▄▄▄▄▄▅▅▇▆▆▆▆▆█▆▇▆█▇▆▆▇ (TaskRunner pid=2099163) wandb: actor/grad_norm █▃▄▃▄▄▄▂▃▃▃▂▂▂▃▂▂▂▂▂▂▂▂▂▂▂▃▃▂▄▄▂▂▁▁▂▃▃▄▂ (TaskRunner pid=2099163) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2099163) wandb: actor/lr ▁▂▄▅▆███████████████████████████████████ (TaskRunner pid=2099163) wandb: actor/pg_clipfrac ▄▄▄▄▆▄▂▃▆▃▁▂▂▂▃▃▅▃▂▄▄▂▄▃▅▂▅▅▃▄▇▄▂▂█▅▅▇█▂ (TaskRunner pid=2099163) wandb: actor/pg_clipfrac_lower ▁▁▁▁▆▁▁▁▁▁▁▁▁▁█▁▄▄▃▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▅▁▁▁ (TaskRunner pid=2099163) wandb: actor/pg_loss ▂▅▆▆█▁▆▂▇▅▃▄▃█▄▄▅▅▄▄▄▆▄▄▄▅▃▄▆▅▆▂▅▇▃▄▆▄▃█ (TaskRunner pid=2099163) wandb: actor/ppo_kl ▂▂▁▂▂▃▃▃▂▃▂▁▂▄▁▄▂▂▄█▃▂▅▂▃▃▆▆▃▄▂▃▅▄▂▃▃▂▃▃ (TaskRunner pid=2099163) wandb: critic/advantages/max ████████▁███████████████████████████████ (TaskRunner pid=2099163) wandb: critic/advantages/mean ▃▂▁▁▅▁▄▅▅▃▄▃▃▄▄▂▄▂▄▁▄▄▄▅▄▅▄▃▄█▄▄▃▄▄▄▅▃▄▄ (TaskRunner pid=2099163) wandb: +204 ... (TaskRunner pid=2099163) wandb: (TaskRunner pid=2099163) wandb: Run summary: (TaskRunner pid=2099163) wandb: actor/entropy 0.28354 (TaskRunner pid=2099163) wandb: actor/grad_norm 0.49692 (TaskRunner pid=2099163) wandb: actor/kl_loss 0 (TaskRunner pid=2099163) wandb: actor/lr 0.0 (TaskRunner pid=2099163) wandb: actor/pg_clipfrac 0.00106 (TaskRunner pid=2099163) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2099163) wandb: actor/pg_loss 5e-05 (TaskRunner pid=2099163) wandb: actor/ppo_kl 0.00073 (TaskRunner pid=2099163) wandb: critic/advantages/max 0.875 (TaskRunner pid=2099163) wandb: critic/advantages/mean -0.00397 (TaskRunner pid=2099163) wandb: +204 ... (TaskRunner pid=2099163) wandb: (TaskRunner pid=2099163) wandb: 🚀 View run qwen3gen-biology-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/94h2xs6b (TaskRunner pid=2099163) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2099163) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2099163) wandb: Find logs at: ./wandb/run-20260702_102415-94h2xs6b/logs (TaskRunner pid=2099163) Exception ignored in atexit callback: .teardown_atexit at 0x7d8258333880> (TaskRunner pid=2099163) Traceback (most recent call last): (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2099163) conn.teardown(hooks.exit_code) (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2099163) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2099163) return future.result() (TaskRunner pid=2099163) ^^^^^^^^^^^^^^^ (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2099163) return self.__get_result() (TaskRunner pid=2099163) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2099163) raise self._exception (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2099163) return await fn() (TaskRunner pid=2099163) ^^^^^^^^^^ (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2099163) await self._client.close() (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2099163) await self._writer.wait_closed() (TaskRunner pid=2099163) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2099163) await self._protocol._get_close_waiter(self) (TaskRunner pid=2099163) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 11:37:05] Finished biology grpo [2026-07-02 11:37:05] Starting biology rlsd_tr Experiment: qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: biology Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet Ray tmp: /tmp/ray_q3g_biology_rlsd_tr_Qwen3_4B 2026-07-02 11:37:06,851 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/biology Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_biology_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/biology +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_biology_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/biology', 'EXPERIMENT': 'qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 11:37:16,648 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2122883) TaskRunner hostname: mole-workspace-rw, PID: 2122883 (TaskRunner pid=2122883) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2122883) 'calculate_entropy': False, (TaskRunner pid=2122883) 'calculate_sum_pi_squared': False, (TaskRunner pid=2122883) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2122883) 'async_save': False, (TaskRunner pid=2122883) 'load_contents': ['model', (TaskRunner pid=2122883) 'optimizer', (TaskRunner pid=2122883) 'extra'], (TaskRunner pid=2122883) 'save_contents': ['model', (TaskRunner pid=2122883) 'optimizer', (TaskRunner pid=2122883) 'extra']}, (TaskRunner pid=2122883) 'clip_ratio': 0.2, (TaskRunner pid=2122883) 'clip_ratio_c': 3.0, (TaskRunner pid=2122883) 'clip_ratio_high': 0.28, (TaskRunner pid=2122883) 'clip_ratio_low': 0.2, (TaskRunner pid=2122883) 'data_loader_seed': 42, (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_coeff': 0, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'freeze_vision_tower': False, (TaskRunner pid=2122883) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'forward_only': False, (TaskRunner pid=2122883) 'forward_prefetch': False, (TaskRunner pid=2122883) 'fsdp_size': -1, (TaskRunner pid=2122883) 'full_determinism': False, (TaskRunner pid=2122883) 'model_dtype': 'fp32', (TaskRunner pid=2122883) 'offload_policy': False, (TaskRunner pid=2122883) 'optimizer_offload': False, (TaskRunner pid=2122883) 'param_offload': False, (TaskRunner pid=2122883) 'reshard_after_forward': True, (TaskRunner pid=2122883) 'seed': 42, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_orig_params': False, (TaskRunner pid=2122883) 'use_torch_compile': True, (TaskRunner pid=2122883) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2122883) 'grad_clip': 1.0, (TaskRunner pid=2122883) 'kl_loss_coef': 0.001, (TaskRunner pid=2122883) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2122883) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2122883) 'loss_scale_factor': None, (TaskRunner pid=2122883) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2122883) 'betas': [0.9, 0.999], (TaskRunner pid=2122883) 'clip_grad': 1.0, (TaskRunner pid=2122883) 'lr': 1e-06, (TaskRunner pid=2122883) 'lr_scheduler_type': 'constant', (TaskRunner pid=2122883) 'lr_warmup_steps': 10, (TaskRunner pid=2122883) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2122883) 'min_lr_ratio': 0.0, (TaskRunner pid=2122883) 'num_cycles': 0.5, (TaskRunner pid=2122883) 'optimizer': 'AdamW', (TaskRunner pid=2122883) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2122883) 'override_optimizer_config': None, (TaskRunner pid=2122883) 'total_training_steps': -1, (TaskRunner pid=2122883) 'warmup_style': None, (TaskRunner pid=2122883) 'weight_decay': 0.01}, (TaskRunner pid=2122883) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2122883) 'clip_cov_lb': 1.0, (TaskRunner pid=2122883) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2122883) 'clip_cov_ub': 5.0, (TaskRunner pid=2122883) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2122883) 'loss_mode': 'rlsd', (TaskRunner pid=2122883) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2122883) 'ppo_epochs': 1, (TaskRunner pid=2122883) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2122883) 'ppo_micro_batch_size': None, (TaskRunner pid=2122883) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2122883) 'ppo_mini_batch_size': 8, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'rollout_n': 8, (TaskRunner pid=2122883) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2122883) 'mode': 'disabled', (TaskRunner pid=2122883) 'record_file': None, (TaskRunner pid=2122883) 'replay_file': None}, (TaskRunner pid=2122883) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2122883) 'alpha': 0.5, (TaskRunner pid=2122883) 'distillation_add_tail': True, (TaskRunner pid=2122883) 'distillation_topk': 100, (TaskRunner pid=2122883) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2122883) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2122883) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'loss_weight': 0.0, (TaskRunner pid=2122883) 'mask': 'all'}, (TaskRunner pid=2122883) 'feedback_template': '\n' (TaskRunner pid=2122883) 'The ' (TaskRunner pid=2122883) 'following ' (TaskRunner pid=2122883) 'is ' (TaskRunner pid=2122883) 'feedback ' (TaskRunner pid=2122883) 'from ' (TaskRunner pid=2122883) 'your ' (TaskRunner pid=2122883) 'unsuccessful ' (TaskRunner pid=2122883) 'earlier ' (TaskRunner pid=2122883) 'attempt:\n' (TaskRunner pid=2122883) '\n' (TaskRunner pid=2122883) '{feedback_raw}', (TaskRunner pid=2122883) 'full_logit_distillation': True, (TaskRunner pid=2122883) 'include_environment_feedback': True, (TaskRunner pid=2122883) 'is_clip': 2, (TaskRunner pid=2122883) 'max_reprompt_len': 10240, (TaskRunner pid=2122883) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2122883) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2122883) '\n' (TaskRunner pid=2122883) 'Correctly ' (TaskRunner pid=2122883) 'solve ' (TaskRunner pid=2122883) 'the ' (TaskRunner pid=2122883) 'original ' (TaskRunner pid=2122883) 'question.', (TaskRunner pid=2122883) 'reprompt_truncation': 'right', (TaskRunner pid=2122883) 'solution_template': '\n' (TaskRunner pid=2122883) 'Correct ' (TaskRunner pid=2122883) 'solution:\n' (TaskRunner pid=2122883) '\n' (TaskRunner pid=2122883) '{successful_previous_attempt}', (TaskRunner pid=2122883) 'srpo_dynamic_weighting': False, (TaskRunner pid=2122883) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2122883) 'success_reward_threshold': 0.5, (TaskRunner pid=2122883) 'teacher_regularization': 'trust-region', (TaskRunner pid=2122883) 'teacher_update_rate': 0.1, (TaskRunner pid=2122883) 'token_reweight_decay_steps': 0, (TaskRunner pid=2122883) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2122883) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2122883) 'shuffle': False, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2122883) 'tau_neg': 1.05, (TaskRunner pid=2122883) 'tau_pos': 1.0, (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_dynamic_bsz': False, (TaskRunner pid=2122883) 'use_fused_kernels': False, (TaskRunner pid=2122883) 'use_kl_loss': False, (TaskRunner pid=2122883) 'use_prefix_grouper': False, (TaskRunner pid=2122883) 'use_remove_padding': True, (TaskRunner pid=2122883) 'use_torch_compile': True}, (TaskRunner pid=2122883) 'hybrid_engine': True, (TaskRunner pid=2122883) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2122883) 'custom_chat_template': None, (TaskRunner pid=2122883) 'enable_activation_offload': False, (TaskRunner pid=2122883) 'enable_gradient_checkpointing': True, (TaskRunner pid=2122883) 'exclude_modules': None, (TaskRunner pid=2122883) 'external_lib': None, (TaskRunner pid=2122883) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2122883) 'hf_config_path': None, (TaskRunner pid=2122883) 'lora_adapter_path': None, (TaskRunner pid=2122883) 'lora_alpha': 16, (TaskRunner pid=2122883) 'lora_rank': 0, (TaskRunner pid=2122883) 'override_config': {}, (TaskRunner pid=2122883) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2122883) 'target_modules': 'all-linear', (TaskRunner pid=2122883) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2122883) 'num_shards': 4}, (TaskRunner pid=2122883) 'tokenizer_path': None, (TaskRunner pid=2122883) 'trust_remote_code': True, (TaskRunner pid=2122883) 'use_fused_kernels': False, (TaskRunner pid=2122883) 'use_liger': False, (TaskRunner pid=2122883) 'use_remove_padding': True, (TaskRunner pid=2122883) 'use_shm': False}, (TaskRunner pid=2122883) 'nccl_timeout': 600, (TaskRunner pid=2122883) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'forward_only': True, (TaskRunner pid=2122883) 'forward_prefetch': False, (TaskRunner pid=2122883) 'fsdp_size': -1, (TaskRunner pid=2122883) 'full_determinism': False, (TaskRunner pid=2122883) 'model_dtype': 'fp32', (TaskRunner pid=2122883) 'offload_policy': False, (TaskRunner pid=2122883) 'optimizer_offload': False, (TaskRunner pid=2122883) 'param_offload': False, (TaskRunner pid=2122883) 'reshard_after_forward': True, (TaskRunner pid=2122883) 'seed': 42, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_orig_params': False, (TaskRunner pid=2122883) 'use_torch_compile': True, (TaskRunner pid=2122883) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2122883) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2122883) 'log_prob_micro_batch_size': None, (TaskRunner pid=2122883) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2122883) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'rollout_n': 8, (TaskRunner pid=2122883) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2122883) 'mode': 'disabled', (TaskRunner pid=2122883) 'record_file': None, (TaskRunner pid=2122883) 'replay_file': None}, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_torch_compile': True}, (TaskRunner pid=2122883) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2122883) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2122883) 'agent_loop_config_path': None, (TaskRunner pid=2122883) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2122883) 'name': None, (TaskRunner pid=2122883) 'path': None}, (TaskRunner pid=2122883) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2122883) 'num_workers': 8}, (TaskRunner pid=2122883) 'calculate_log_probs': True, (TaskRunner pid=2122883) 'cudagraph_capture_sizes': None, (TaskRunner pid=2122883) 'data_parallel_size': 1, (TaskRunner pid=2122883) 'disable_log_stats': True, (TaskRunner pid=2122883) 'do_sample': True, (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'enable_chunked_prefill': True, (TaskRunner pid=2122883) 'enable_prefix_caching': True, (TaskRunner pid=2122883) 'enable_rollout_routing_replay': False, (TaskRunner pid=2122883) 'enforce_eager': False, (TaskRunner pid=2122883) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2122883) 'expert_parallel_size': 1, (TaskRunner pid=2122883) 'free_cache_engine': True, (TaskRunner pid=2122883) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2122883) 'ignore_eos': False, (TaskRunner pid=2122883) 'layered_summon': False, (TaskRunner pid=2122883) 'load_format': 'dummy', (TaskRunner pid=2122883) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2122883) 'log_prob_micro_batch_size': None, (TaskRunner pid=2122883) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2122883) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2122883) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2122883) 'max_model_len': 10240, (TaskRunner pid=2122883) 'max_num_batched_tokens': 10240, (TaskRunner pid=2122883) 'max_num_seqs': 1024, (TaskRunner pid=2122883) 'mode': 'async', (TaskRunner pid=2122883) 'multi_stage_wake_up': False, (TaskRunner pid=2122883) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'format': 'hermes', (TaskRunner pid=2122883) 'interaction_config_path': None, (TaskRunner pid=2122883) 'max_assistant_turns': None, (TaskRunner pid=2122883) 'max_parallel_calls': 1, (TaskRunner pid=2122883) 'max_tool_response_length': 256, (TaskRunner pid=2122883) 'max_user_turns': None, (TaskRunner pid=2122883) 'num_repeat_rollouts': None, (TaskRunner pid=2122883) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2122883) 'tool_config_path': None, (TaskRunner pid=2122883) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2122883) 'use_inference_chat_template': False}, (TaskRunner pid=2122883) 'n': 8, (TaskRunner pid=2122883) 'name': 'vllm', (TaskRunner pid=2122883) 'over_sample_rate': 0, (TaskRunner pid=2122883) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2122883) 'port': 9090, (TaskRunner pid=2122883) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2122883) 'prompt_length': 2048, (TaskRunner pid=2122883) 'quantization': None, (TaskRunner pid=2122883) 'quantization_config_file': None, (TaskRunner pid=2122883) 'response_length': 8192, (TaskRunner pid=2122883) 'scheduling_policy': 'fcfs', (TaskRunner pid=2122883) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2122883) 'skip_rollout': False, (TaskRunner pid=2122883) 'skip_tokenizer_init': True, (TaskRunner pid=2122883) 'temperature': 1.0, (TaskRunner pid=2122883) 'tensor_model_parallel_size': 2, (TaskRunner pid=2122883) 'top_k': -1, (TaskRunner pid=2122883) 'top_p': 1.0, (TaskRunner pid=2122883) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2122883) 'backend': None, (TaskRunner pid=2122883) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2122883) 'token2text': False}, (TaskRunner pid=2122883) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2122883) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2122883) 'do_sample': True, (TaskRunner pid=2122883) 'n': 16, (TaskRunner pid=2122883) 'temperature': 0.6, (TaskRunner pid=2122883) 'top_k': -1, (TaskRunner pid=2122883) 'top_p': 0.95}}}, (TaskRunner pid=2122883) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2122883) 'adv_estimator': 'grpo', (TaskRunner pid=2122883) 'gamma': 1.0, (TaskRunner pid=2122883) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2122883) 'horizon': 10000, (TaskRunner pid=2122883) 'kl_coef': 0.001, (TaskRunner pid=2122883) 'target_kl': 0.1, (TaskRunner pid=2122883) 'type': 'fixed'}, (TaskRunner pid=2122883) 'kl_penalty': 'kl', (TaskRunner pid=2122883) 'lam': 1.0, (TaskRunner pid=2122883) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2122883) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2122883) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2122883) 'loss_type': 'ppo_clip', (TaskRunner pid=2122883) 'rollout_is': 'token', (TaskRunner pid=2122883) 'rollout_is_batch_normalize': False, (TaskRunner pid=2122883) 'rollout_is_threshold': 2.0, (TaskRunner pid=2122883) 'rollout_rs': None, (TaskRunner pid=2122883) 'rollout_rs_threshold': None}, (TaskRunner pid=2122883) 'use_kl_in_reward': False, (TaskRunner pid=2122883) 'use_pf_ppo': False}, (TaskRunner pid=2122883) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2122883) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2122883) 'async_save': False, (TaskRunner pid=2122883) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2122883) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2122883) 'cliprange_value': 0.5, (TaskRunner pid=2122883) 'data_loader_seed': 42, (TaskRunner pid=2122883) 'enable': None, (TaskRunner pid=2122883) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2122883) 'forward_micro_batch_size': None, (TaskRunner pid=2122883) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2122883) 'grad_clip': 1.0, (TaskRunner pid=2122883) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2122883) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2122883) 'enable_activation_offload': False, (TaskRunner pid=2122883) 'enable_gradient_checkpointing': True, (TaskRunner pid=2122883) 'external_lib': None, (TaskRunner pid=2122883) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'entropy_checkpointing': False, (TaskRunner pid=2122883) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2122883) 'forward_only': False, (TaskRunner pid=2122883) 'forward_prefetch': False, (TaskRunner pid=2122883) 'fsdp_size': -1, (TaskRunner pid=2122883) 'full_determinism': False, (TaskRunner pid=2122883) 'model_dtype': 'fp32', (TaskRunner pid=2122883) 'offload_policy': False, (TaskRunner pid=2122883) 'optimizer_offload': False, (TaskRunner pid=2122883) 'param_offload': False, (TaskRunner pid=2122883) 'reshard_after_forward': True, (TaskRunner pid=2122883) 'seed': 42, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_orig_params': False, (TaskRunner pid=2122883) 'use_torch_compile': True, (TaskRunner pid=2122883) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2122883) 'lora_alpha': 16, (TaskRunner pid=2122883) 'lora_rank': 0, (TaskRunner pid=2122883) 'override_config': {}, (TaskRunner pid=2122883) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2122883) 'target_modules': 'all-linear', (TaskRunner pid=2122883) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2122883) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2122883) 'trust_remote_code': True, (TaskRunner pid=2122883) 'use_remove_padding': False, (TaskRunner pid=2122883) 'use_shm': False}, (TaskRunner pid=2122883) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2122883) 'betas': [0.9, 0.999], (TaskRunner pid=2122883) 'clip_grad': 1.0, (TaskRunner pid=2122883) 'lr': 1e-05, (TaskRunner pid=2122883) 'lr_scheduler_type': 'constant', (TaskRunner pid=2122883) 'lr_warmup_steps': -1, (TaskRunner pid=2122883) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2122883) 'min_lr_ratio': 0.0, (TaskRunner pid=2122883) 'num_cycles': 0.5, (TaskRunner pid=2122883) 'optimizer': 'AdamW', (TaskRunner pid=2122883) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2122883) 'override_optimizer_config': None, (TaskRunner pid=2122883) 'total_training_steps': -1, (TaskRunner pid=2122883) 'warmup_style': None, (TaskRunner pid=2122883) 'weight_decay': 0.01}, (TaskRunner pid=2122883) 'ppo_epochs': 1, (TaskRunner pid=2122883) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2122883) 'ppo_micro_batch_size': None, (TaskRunner pid=2122883) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2122883) 'ppo_mini_batch_size': 8, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'rollout_n': 8, (TaskRunner pid=2122883) 'shuffle': False, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_dynamic_bsz': False}, (TaskRunner pid=2122883) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2122883) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2122883) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2122883) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2122883) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2122883) 'dataloader_num_workers': 8, (TaskRunner pid=2122883) 'filter_overlong_prompts': True, (TaskRunner pid=2122883) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2122883) 'image_key': 'images', (TaskRunner pid=2122883) 'image_patch_size': 14, (TaskRunner pid=2122883) 'max_prompt_length': 2048, (TaskRunner pid=2122883) 'max_response_length': 8192, (TaskRunner pid=2122883) 'prompt_key': 'prompt', (TaskRunner pid=2122883) 'return_full_prompt': False, (TaskRunner pid=2122883) 'return_multi_modal_inputs': True, (TaskRunner pid=2122883) 'return_raw_chat': True, (TaskRunner pid=2122883) 'return_raw_input_ids': False, (TaskRunner pid=2122883) 'reward_fn_key': 'data_source', (TaskRunner pid=2122883) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2122883) 'seed': None, (TaskRunner pid=2122883) 'shuffle': True, (TaskRunner pid=2122883) 'tokenizer': None, (TaskRunner pid=2122883) 'tool_config_path': None, (TaskRunner pid=2122883) 'train_batch_size': 32, (TaskRunner pid=2122883) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/train.parquet'], (TaskRunner pid=2122883) 'train_max_samples': 3200, (TaskRunner pid=2122883) 'truncation': 'error', (TaskRunner pid=2122883) 'trust_remote_code': True, (TaskRunner pid=2122883) 'use_shm': False, (TaskRunner pid=2122883) 'val_batch_size': None, (TaskRunner pid=2122883) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/biology/test.parquet'], (TaskRunner pid=2122883) 'val_max_samples': -1, (TaskRunner pid=2122883) 'validation_shuffle': False, (TaskRunner pid=2122883) 'video_key': 'videos'}, (TaskRunner pid=2122883) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2122883) 'cuda-memory-usage': 'true', (TaskRunner pid=2122883) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2122883) 'capture-range-end': None, (TaskRunner pid=2122883) 'cuda-graph-trace': 'graph', (TaskRunner pid=2122883) 'cuda-memory-usage': 'true', (TaskRunner pid=2122883) 'kill': 'none', (TaskRunner pid=2122883) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2122883) 'torch_memory': {'context': 'all', (TaskRunner pid=2122883) 'kw_args': {}, (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'stacks': 'all', (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2122883) 'profile_continuous_steps': False, (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'steps': None, (TaskRunner pid=2122883) 'tool': None}, (TaskRunner pid=2122883) 'max_model_len': 10240, (TaskRunner pid=2122883) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_biology_rlsd_tr_Qwen3_4B', (TaskRunner pid=2122883) 'include_dashboard': False, (TaskRunner pid=2122883) 'num_cpus': None, (TaskRunner pid=2122883) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2122883) 'TASK': 'datasets/sciknoweval/biology', (TaskRunner pid=2122883) 'USER': 'root'}}}, (TaskRunner pid=2122883) 'timeline_json_file': None}, (TaskRunner pid=2122883) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2122883) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2122883) 'name': 'custom_reward_manager', (TaskRunner pid=2122883) 'path': None}, (TaskRunner pid=2122883) 'name': 'naive', (TaskRunner pid=2122883) 'source': 'register'}, (TaskRunner pid=2122883) 'reward_model': {'enable': False, (TaskRunner pid=2122883) 'enable_resource_pool': False, (TaskRunner pid=2122883) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2122883) 'launch_reward_fn_async': False, (TaskRunner pid=2122883) 'max_length': None, (TaskRunner pid=2122883) 'micro_batch_size': None, (TaskRunner pid=2122883) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2122883) 'model': {'external_lib': None, (TaskRunner pid=2122883) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2122883) 'forward_prefetch': False, (TaskRunner pid=2122883) 'fsdp_size': -1, (TaskRunner pid=2122883) 'param_offload': False, (TaskRunner pid=2122883) 'reshard_after_forward': True, (TaskRunner pid=2122883) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2122883) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2122883) 'override_config': {}, (TaskRunner pid=2122883) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2122883) 'trust_remote_code': False, (TaskRunner pid=2122883) 'use_fused_kernels': False, (TaskRunner pid=2122883) 'use_remove_padding': False, (TaskRunner pid=2122883) 'use_shm': False}, (TaskRunner pid=2122883) 'n_gpus_per_node': 8, (TaskRunner pid=2122883) 'nnodes': 0, (TaskRunner pid=2122883) 'num_workers': 1, (TaskRunner pid=2122883) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2122883) 'all_ranks': False, (TaskRunner pid=2122883) 'enable': False, (TaskRunner pid=2122883) 'ranks': [], (TaskRunner pid=2122883) 'save_path': 'outputs/profile', (TaskRunner pid=2122883) 'tool': None, (TaskRunner pid=2122883) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2122883) 'analysis': True, (TaskRunner pid=2122883) 'contents': [], (TaskRunner pid=2122883) 'discrete': False, (TaskRunner pid=2122883) 'level': 'level0'}, (TaskRunner pid=2122883) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2122883) 'discrete': False}, (TaskRunner pid=2122883) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2122883) 'step_end': None, (TaskRunner pid=2122883) 'step_start': 0}, (TaskRunner pid=2122883) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2122883) 'stack_depth': 32, (TaskRunner pid=2122883) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2122883) 'reward_loop_class_name': None, (TaskRunner pid=2122883) 'reward_loop_module_path': None, (TaskRunner pid=2122883) 'reward_loop_source': 'register', (TaskRunner pid=2122883) 'reward_manager': 'naive', (TaskRunner pid=2122883) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2122883) 'cudagraph_capture_sizes': None, (TaskRunner pid=2122883) 'data_parallel_size': 1, (TaskRunner pid=2122883) 'disable_log_stats': True, (TaskRunner pid=2122883) 'dtype': 'bfloat16', (TaskRunner pid=2122883) 'enable_chunked_prefill': True, (TaskRunner pid=2122883) 'enable_prefix_caching': True, (TaskRunner pid=2122883) 'enforce_eager': True, (TaskRunner pid=2122883) 'engine_kwargs': {}, (TaskRunner pid=2122883) 'expert_parallel_size': 1, (TaskRunner pid=2122883) 'free_cache_engine': True, (TaskRunner pid=2122883) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2122883) 'limit_images': None, (TaskRunner pid=2122883) 'load_format': 'auto', (TaskRunner pid=2122883) 'max_model_len': None, (TaskRunner pid=2122883) 'max_num_batched_tokens': 8192, (TaskRunner pid=2122883) 'max_num_seqs': 1024, (TaskRunner pid=2122883) 'name': '???', (TaskRunner pid=2122883) 'prompt_length': 2048, (TaskRunner pid=2122883) 'response_length': 2048, (TaskRunner pid=2122883) 'skip_tokenizer_init': False, (TaskRunner pid=2122883) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2122883) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2122883) 'memory_limit_mb': 1024, (TaskRunner pid=2122883) 'url': None}, (TaskRunner pid=2122883) 'strategy': 'fsdp', (TaskRunner pid=2122883) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2122883) 'use_dynamic_bsz': False, (TaskRunner pid=2122883) 'use_reward_loop': False}, (TaskRunner pid=2122883) 'trainer': {'balance_batch': True, (TaskRunner pid=2122883) 'critic_warmup': 0, (TaskRunner pid=2122883) 'default_hdfs_dir': None, (TaskRunner pid=2122883) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2122883) 'del_local_ckpt_after_load': False, (TaskRunner pid=2122883) 'device': 'cuda', (TaskRunner pid=2122883) 'esi_redundant_time': 0, (TaskRunner pid=2122883) 'experiment_name': 'qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2122883) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2122883) 'log_val_generations': 0, (TaskRunner pid=2122883) 'logger': ['console', 'wandb'], (TaskRunner pid=2122883) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2122883) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2122883) 'n_gpus_per_node': 8, (TaskRunner pid=2122883) 'nnodes': 1, (TaskRunner pid=2122883) 'project_name': 'SDPO-root', (TaskRunner pid=2122883) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2122883) 'resume_from_path': None, (TaskRunner pid=2122883) 'resume_mode': 'auto', (TaskRunner pid=2122883) 'rollout_data_dir': None, (TaskRunner pid=2122883) 'save_freq': 10, (TaskRunner pid=2122883) 'test_freq': 10, (TaskRunner pid=2122883) 'total_epochs': 30, (TaskRunner pid=2122883) 'total_training_steps': 100, (TaskRunner pid=2122883) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2122883) 'val_before_train': False, (TaskRunner pid=2122883) 'val_only': False, (TaskRunner pid=2122883) 'validation_data_dir': None}, (TaskRunner pid=2122883) 'transfer_queue': {'enable': False}, (TaskRunner pid=2122883) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/biology', (TaskRunner pid=2122883) 'dir': '/users/root/SDPO', (TaskRunner pid=2122883) 'log_dir': '/users/root/output', (TaskRunner pid=2122883) 'task': 'datasets/sciknoweval/biology'}} (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2122883) use_critic=need_critic(config), (TaskRunner pid=2122883) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2122883) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2122883) Using dataset class: RLHFDataset (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2122883) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2122883) dataset len: 450 (TaskRunner pid=2122883) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2122883) WARNING:2026-07-02 11:37:25,419:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2122883) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/450 [00:00 (TaskRunner pid=2122883) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2122883) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2122883) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2122883) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2122883) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2122883) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 50/50 [00:00<00:00, 75.24 examples/s] (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2122883) self.use_critic = need_critic(self.config) (WorkerDict pid=2123259) [W702 11:37:33.058108451 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53649 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2123259) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2123259) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2123259) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2123266) [W702 11:37:34.147181241 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:53649 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2123259) Model config after override: Qwen3Config { (WorkerDict pid=2123259) "architectures": [ (WorkerDict pid=2123259) "Qwen3ForCausalLM" (WorkerDict pid=2123259) ], (WorkerDict pid=2123259) "attention_bias": false, (WorkerDict pid=2123259) "attention_dropout": 0.0, (WorkerDict pid=2123259) "dtype": "bfloat16", (WorkerDict pid=2123259) "eos_token_id": 151645, (WorkerDict pid=2123259) "head_dim": 128, (WorkerDict pid=2123259) "hidden_act": "silu", (WorkerDict pid=2123259) "hidden_size": 2560, (WorkerDict pid=2123259) "initializer_range": 0.02, (WorkerDict pid=2123259) "intermediate_size": 9728, (WorkerDict pid=2123259) "layer_types": [ (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention" (WorkerDict pid=2123259) ], (WorkerDict pid=2123259) "max_position_embeddings": 40960, (WorkerDict pid=2123259) "max_window_layers": 36, (WorkerDict pid=2123259) "model_type": "qwen3", (WorkerDict pid=2123259) "num_attention_heads": 32, (WorkerDict pid=2123259) "num_hidden_layers": 36, (WorkerDict pid=2123259) "num_key_value_heads": 8, (WorkerDict pid=2123259) "pad_token_id": 151643, (WorkerDict pid=2123259) "rms_norm_eps": 1e-06, (WorkerDict pid=2123259) "rope_scaling": null, (WorkerDict pid=2123259) "rope_theta": 1000000, (WorkerDict pid=2123259) "sliding_window": null, (WorkerDict pid=2123259) "tie_word_embeddings": true, (WorkerDict pid=2123259) "transformers_version": "4.57.1", (WorkerDict pid=2123259) "use_cache": true, (WorkerDict pid=2123259) "use_sliding_window": false, (WorkerDict pid=2123259) "vocab_size": 151936 (WorkerDict pid=2123259) } (WorkerDict pid=2123259) (WorkerDict pid=2123264) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2123264) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2123264) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2123264) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2123259) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2123259) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2123259) Actor use_remove_padding=True (WorkerDict pid=2123259) Actor use_fused_kernels=False (WorkerDict pid=2123259) Actor use_prefix_grouper=False (WorkerDict pid=2123265) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2123265) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2123259) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2123259) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2123265) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.95s/it] [repeated 13x across cluster] (WorkerDict pid=2123265) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.99s/it] [repeated 5x across cluster] (WorkerDict pid=2123263) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.95s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.98s/it] (WorkerDict pid=2123260) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2123260) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2123260) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2123266) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2123266) warnings.warn( (WorkerDict pid=2123265) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2123265) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2123259) reference model: Qwen/Qwen3-4B (WorkerDict pid=2123263) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 15x across cluster] (WorkerDict pid=2123263) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 15x across cluster] (WorkerDict pid=2123259) Model config after override: Qwen3Config { (WorkerDict pid=2123259) "architectures": [ (WorkerDict pid=2123259) "Qwen3ForCausalLM" (WorkerDict pid=2123259) ], (WorkerDict pid=2123259) "attention_bias": false, (WorkerDict pid=2123259) "attention_dropout": 0.0, (WorkerDict pid=2123259) "dtype": "bfloat16", (WorkerDict pid=2123259) "eos_token_id": 151645, (WorkerDict pid=2123259) "head_dim": 128, (WorkerDict pid=2123259) "hidden_act": "silu", (WorkerDict pid=2123259) "hidden_size": 2560, (WorkerDict pid=2123259) "initializer_range": 0.02, (WorkerDict pid=2123259) "intermediate_size": 9728, (WorkerDict pid=2123259) "layer_types": [ (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention", (WorkerDict pid=2123259) "full_attention" (WorkerDict pid=2123259) ], (WorkerDict pid=2123259) "max_position_embeddings": 40960, (WorkerDict pid=2123259) "max_window_layers": 36, (WorkerDict pid=2123259) "model_type": "qwen3", (WorkerDict pid=2123259) "num_attention_heads": 32, (WorkerDict pid=2123259) "num_hidden_layers": 36, (WorkerDict pid=2123259) "num_key_value_heads": 8, (WorkerDict pid=2123259) "pad_token_id": 151643, (WorkerDict pid=2123259) "rms_norm_eps": 1e-06, (WorkerDict pid=2123259) "rope_scaling": null, (WorkerDict pid=2123259) "rope_theta": 1000000, (WorkerDict pid=2123259) "sliding_window": null, (WorkerDict pid=2123259) "tie_word_embeddings": true, (WorkerDict pid=2123259) "transformers_version": "4.57.1", (WorkerDict pid=2123259) "use_cache": true, (WorkerDict pid=2123259) "use_sliding_window": false, (WorkerDict pid=2123259) "vocab_size": 151936 (WorkerDict pid=2123259) } (WorkerDict pid=2123259) (WorkerDict pid=2123259) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2123259) Ref use_remove_padding=True (WorkerDict pid=2123259) Ref use_fused_kernels=False (WorkerDict pid=2123259) Ref use_prefix_grouper=False (WorkerDict pid=2123266) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2123266) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2122883) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2122883) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2123266) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.86s/it] [repeated 15x across cluster] (WorkerDict pid=2123265) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.92s/it] [repeated 6x across cluster] (vLLMHttpServer pid=2124149) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2124149) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2124149) ['serve', (vLLMHttpServer pid=2124149) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2124149) '--dtype', (vLLMHttpServer pid=2124149) 'bfloat16', (vLLMHttpServer pid=2124149) '--load_format', (vLLMHttpServer pid=2124149) 'dummy', (vLLMHttpServer pid=2124149) '--trust_remote_code', (vLLMHttpServer pid=2124149) '--max_model_len', (vLLMHttpServer pid=2124149) '40960', (vLLMHttpServer pid=2124149) '--max_num_seqs', (vLLMHttpServer pid=2124149) '1024', (vLLMHttpServer pid=2124149) '--enable_chunked_prefill', (vLLMHttpServer pid=2124149) '--max_num_batched_tokens', (vLLMHttpServer pid=2124149) '10240', (vLLMHttpServer pid=2124149) '--enable_prefix_caching', (vLLMHttpServer pid=2124149) '--enable_sleep_mode', (vLLMHttpServer pid=2124149) '--logprobs_mode', (vLLMHttpServer pid=2124149) 'processed_logprobs', (vLLMHttpServer pid=2124149) '--disable_custom_all_reduce', (vLLMHttpServer pid=2124149) '--gpu_memory_utilization', (vLLMHttpServer pid=2124149) '0.8', (vLLMHttpServer pid=2124149) '--disable_log_stats', (vLLMHttpServer pid=2124149) '--tensor_parallel_size', (vLLMHttpServer pid=2124149) '2', (vLLMHttpServer pid=2124149) '--seed', (vLLMHttpServer pid=2124149) '0', (vLLMHttpServer pid=2124149) '--override_generation_config', (vLLMHttpServer pid=2124149) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2124149) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2124149) '--hf_overrides', (vLLMHttpServer pid=2124149) '{}', (vLLMHttpServer pid=2124149) '--scheduling_policy', (vLLMHttpServer pid=2124149) 'fcfs'] (vLLMHttpServer pid=2124149) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2124149) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2124149) WARNING 07-02 11:38:30 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2123259) WARNING 07-02 11:38:39 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2124150) WARNING 07-02 11:38:30 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2123261) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2123261) [rank2]:W0702 11:38:49.111000 2123261 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2123259.129147915196096.tmp' (vLLMHttpServer pid=2124150) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2124150) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2124150) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2124150) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2123261) 2026-07-02 11:38:53,118 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2123261) 2026-07-02 11:38:53,134 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2123259) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques such as circular dichroism, NMR, or molecular dynamics simulations. However, without specific computational tools or experimental data provided in the question, it is not possible to calculate or determine the folding stability score for the given protein sequence. Therefore, the correct answer cannot be determined from the information provided. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 0.0 (TaskRunner pid=2122883) [acc] 0.0 (TaskRunner pid=2122883) [pred] A (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2122883) step:10 - global_seqlen/min:11726 - global_seqlen/max:24851 - global_seqlen/minmax_diff:13125 - global_seqlen/balanced_min:17477 - global_seqlen/balanced_max:17949 - global_seqlen/mean:17600.125 - actor/entropy:0.20009024441242218 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8149087429046631 - training/rollout_probs_diff_mean:0.005511445924639702 - training/rollout_probs_diff_std:0.016095276921987534 - training/rollout_actor_probs_pearson_corr:0.9971664547920227 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.60546875 - self_distillation/correct_sample_fraction:0.2734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.60546875 - rollout_corr/rollout_is_mean:1.0001614093780518 - rollout_corr/rollout_is_ratio_fraction_high:4.249713310855441e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012749139568768442 - rollout_corr/rollout_is_max:3.9662325382232666 - rollout_corr/rollout_is_min:0.003977110609412193 - rollout_corr/rollout_is_std:0.045301079750061035 - rollout_corr/rollout_is_eff_sample_size:0.9979527272601818 - rollout_corr/rollout_is_seq_mean:1.0003551244735718 - rollout_corr/rollout_is_seq_std:0.0033447842579334974 - rollout_corr/rollout_is_seq_max:1.0103979110717773 - rollout_corr/rollout_is_seq_min:0.991142749786377 - rollout_corr/rollout_is_seq_max_deviation:0.010397911071777344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2529795560985804) - rollout_corr/training_log_ppl:np.float64(0.2225916018069256) - rollout_corr/kl:np.float64(0.001012606808284744) - rollout_corr/k3_kl:np.float64(0.0014178051028466143) - rollout_corr/rollout_ppl:np.float64(1.2516679712571204) - rollout_corr/rollout_log_ppl:np.float64(0.22157899395097047) - rollout_corr/log_ppl_diff:np.float64(0.0010126078559551388) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031039943860378116) - rollout_corr/log_ppl_diff_max:np.float64(0.011974595487117767) - rollout_corr/log_ppl_diff_min:np.float64(-0.011962682008743286) - rollout_corr/ppl_ratio:np.float64(1.0010205947328359) - rollout_corr/chi2_token:np.float64(0.004173249937593937) - rollout_corr/chi2_seq:np.float64(1.0239088295493275) - actor/pg_loss:np.float64(0.007663122727535665) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016782767970653367) - actor/ppo_kl:np.float64(0.0002169081007590279) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.60546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.2734375) - self_distillation/token_reweight_w_mean:np.float64(510711.6452796182) - self_distillation/token_reweight_w_std:np.float64(6248755.756001078) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0036613668780774) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11369272705633193) - self_distillation/empty_target_batch:np.float64(0.39453125) - actor/grad_norm:np.float64(0.7940817475318909) - perf/mfu/actor:np.float64(0.02968673911541976) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(141.75575494766235) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.325) - val-aux/sciknoweval/reward/std@16:np.float64(0.2554249056239752) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.42513999999999996) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.24136437801418256) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.22138000000000002) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.19069531655682617) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.32382) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2551337969977601) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5265) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.20460662938883328) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.15156000000000003) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12392120350057902) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.32012) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20443076961937032) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.61274) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1543338362640398) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.10611999999999998) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0743746346022705) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.31182) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.14713308978809259) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.67476) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.09348939416746192) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.07942) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04233228578656238) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.3041) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.1033212819785522) - val-aux/sciknoweval/score/mean@16:np.float64(0.325) - val-aux/sciknoweval/score/std@16:np.float64(0.2554249056239752) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.42513999999999996) - val-aux/sciknoweval/score/best@2/std:np.float64(0.24136437801418256) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.22138000000000002) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.19069531655682617) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.32382) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2551337969977601) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5265) - val-aux/sciknoweval/score/best@4/std:np.float64(0.20460662938883328) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.15156000000000003) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12392120350057902) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.32012) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20443076961937032) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.61274) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1543338362640398) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.10611999999999998) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0743746346022705) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.31182) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.14713308978809259) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.67476) - val-aux/sciknoweval/score/best@16/std:np.float64(0.09348939416746192) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.07942) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04233228578656238) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.3041) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.1033212819785522) - val-core/sciknoweval/acc/mean@16:np.float64(0.325) - val-aux/sciknoweval/acc/std@16:np.float64(0.2554249056239752) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.42513999999999996) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.24136437801418256) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.22138000000000002) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.19069531655682617) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.32382) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2551337969977601) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5265) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.20460662938883328) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.15156000000000003) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12392120350057902) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.32012) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20443076961937032) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.61274) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1543338362640398) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.10611999999999998) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0743746346022705) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.31182) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.14713308978809259) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.67476) - val-core/sciknoweval/acc/best@16/std:np.float64(0.09348939416746192) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.07942) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04233228578656238) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.3041) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.1033212819785522) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03743490246911931) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9988800000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.011805957445723348) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9780599999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.049950316494815884) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9881399999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.038377557849678695) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9605799999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06172430689859443) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9969) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.019827817850231286) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.93312) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06831160926205504) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99944) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008418479524687841) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.90012) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06084132628023402) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711646) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.2734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029588628094643354 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029588628094643354 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.75390625 - response_length/max:1820.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.75390625 - response_length_non_aborted/max:1820.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.25 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016302801668643951 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0913536846637726) - timing_s/agent_loop/generate_sequences/max:np.float64(10.443698544055223) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7668629826584947) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.443698544055223) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1820 - timing_s/gen:16.08851490728557 - timing_s/reward:0.05504823662340641 - timing_s/old_log_prob:2.380241384729743 - timing_s/adv:0.6068263426423073 - timing_s/update_actor:16.999784832820296 - timing_s/step:36.216833928599954 - timing_s/testing:81.34854041971266 - timing_s/save_checkpoint:7.180700551718473 - timing_s/stop_profile:0.0001176241785287857 - timing_per_token_ms/gen:0.22790524424922542 - timing_per_token_ms/update_actor:0.12073625068586372 - timing_per_token_ms/adv:0.004309815574053503 - perf/total_num_tokens:140801 - perf/time_per_step:36.216833928599954 - perf/throughput:485.96531200650907 (TaskRunner pid=2122883) Training Progress: 10%|█ | 10/100 [10:52<1:45:57, 70.64s/it] (TaskRunner pid=2122883) step:11 - global_seqlen/min:16828 - global_seqlen/max:30842 - global_seqlen/minmax_diff:14014 - global_seqlen/balanced_min:21635 - global_seqlen/balanced_max:22284 - global_seqlen/mean:21730.625 - actor/entropy:0.16385675966739655 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4429733455181122 - training/rollout_probs_diff_mean:0.004776663146913052 - training/rollout_probs_diff_std:0.014988353475928307 - training/rollout_actor_probs_pearson_corr:0.9971790313720703 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9997762441635132 - rollout_corr/rollout_is_ratio_fraction_high:2.2400179659598507e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001904015225591138 - rollout_corr/rollout_is_max:2.340348958969116 - rollout_corr/rollout_is_min:0.08248017728328705 - rollout_corr/rollout_is_std:0.04225890338420868 - rollout_corr/rollout_is_eff_sample_size:0.9982165965875319 - rollout_corr/rollout_is_seq_mean:0.9997829794883728 - rollout_corr/rollout_is_seq_std:0.003166553797200322 - rollout_corr/rollout_is_seq_max:1.0128588676452637 - rollout_corr/rollout_is_seq_min:0.9910963177680969 - rollout_corr/rollout_is_seq_max_deviation:0.012858867645263672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2296020975336432) - rollout_corr/training_log_ppl:np.float64(0.20335380466713104) - rollout_corr/kl:np.float64(0.0012903355955415918) - rollout_corr/k3_kl:np.float64(0.0012305048087455361) - rollout_corr/rollout_ppl:np.float64(1.2280050399713218) - rollout_corr/rollout_log_ppl:np.float64(0.20206346827035304) - rollout_corr/log_ppl_diff:np.float64(0.0012903363967780024) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026630688516888767) - rollout_corr/log_ppl_diff_max:np.float64(0.017947301268577576) - rollout_corr/log_ppl_diff_min:np.float64(-0.010352224111557007) - rollout_corr/ppl_ratio:np.float64(1.0012970515526831) - rollout_corr/chi2_token:np.float64(0.0022916693706065416) - rollout_corr/chi2_seq:np.float64(0.6786705595441163) - actor/pg_loss:np.float64(0.007793928729370236) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017880768228906163) - actor/ppo_kl:np.float64(-4.9033289212019326e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(369570.61190717644) - self_distillation/token_reweight_w_std:np.float64(4874752.744324681) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011554462835193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11396575406251941) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.7840257436037064) - perf/mfu/actor:np.float64(0.03512070663237796) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(142.33532333374023) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.016731534153223038 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.016731534153223038 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:348.76953125 - response_length/max:2422.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.76953125 - response_length_non_aborted/max:2422.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.3125 - prompt_length/max:500.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.0650909543037415e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0845849998295307) - timing_s/agent_loop/generate_sequences/max:np.float64(14.238213451579213) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3772238838864723) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.238213451579213) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:2422 - timing_s/gen:20.42856288701296 - timing_s/reward:0.0540193896740675 - timing_s/old_log_prob:2.60726017318666 - timing_s/adv:0.7217988539487123 - timing_s/update_actor:17.801132675260305 - timing_s/step:41.7183708678931 - timing_s/stop_profile:2.9888004064559937e-05 - timing_per_token_ms/gen:0.22880173474842314 - timing_per_token_ms/update_actor:0.10239657554292793 - timing_per_token_ms/adv:0.0041519678676333075 - perf/total_num_tokens:173845 - perf/time_per_step:41.7183708678931 - perf/throughput:520.8886288683942 (TaskRunner pid=2122883) Training Progress: 11%|█ | 11/100 [11:34<1:31:39, 61.79s/it] (TaskRunner pid=2122883) step:12 - global_seqlen/min:18510 - global_seqlen/max:42325 - global_seqlen/minmax_diff:23815 - global_seqlen/balanced_min:26152 - global_seqlen/balanced_max:32053 - global_seqlen/mean:26963.125 - actor/entropy:0.1360880583524704 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4592995345592499 - training/rollout_probs_diff_mean:0.0037858886644244194 - training/rollout_probs_diff_std:0.013429389335215092 - training/rollout_actor_probs_pearson_corr:0.9973995685577393 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000216960906982 - rollout_corr/rollout_is_ratio_fraction_high:7.299920980585739e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013788740034215152 - rollout_corr/rollout_is_max:2.5501325130462646 - rollout_corr/rollout_is_min:0.2923682630062103 - rollout_corr/rollout_is_std:0.03888509422540665 - rollout_corr/rollout_is_eff_sample_size:0.9984903510662826 - rollout_corr/rollout_is_seq_mean:0.9999573230743408 - rollout_corr/rollout_is_seq_std:0.002746041864156723 - rollout_corr/rollout_is_seq_max:1.0101428031921387 - rollout_corr/rollout_is_seq_min:0.9911365509033203 - rollout_corr/rollout_is_seq_max_deviation:0.010142803192138672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2124483888037503) - rollout_corr/training_log_ppl:np.float64(0.18915603501955047) - rollout_corr/kl:np.float64(0.001068810922657093) - rollout_corr/k3_kl:np.float64(0.0012623913938227815) - rollout_corr/rollout_ppl:np.float64(1.211122840642929) - rollout_corr/rollout_log_ppl:np.float64(0.18808722433095681) - rollout_corr/log_ppl_diff:np.float64(0.0010688106885936577) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025167708918161225) - rollout_corr/log_ppl_diff_max:np.float64(0.013781130313873291) - rollout_corr/log_ppl_diff_min:np.float64(-0.011515498161315918) - rollout_corr/ppl_ratio:np.float64(1.0010748247150332) - rollout_corr/chi2_token:np.float64(0.002950052497908473) - rollout_corr/chi2_seq:np.float64(1.5056117225904018) - actor/pg_loss:np.float64(0.009150906931608915) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016923086934639286) - actor/ppo_kl:np.float64(6.213555671763515e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(284014.9290252649) - self_distillation/token_reweight_w_std:np.float64(4492645.565195698) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025300320703536) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13327509218652267) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.8035765886306763) - perf/mfu/actor:np.float64(0.042627928291705836) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(142.23823928833008) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04189850762486458 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04189850762486458 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:481.59765625 - response_length/max:8192.0 - response_length/min:93.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:481.59765625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:361.0 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.380460500717163e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0855942126363516) - timing_s/agent_loop/generate_sequences/max:np.float64(50.55506406724453) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.47686035346851) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.55506406724453) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.429069597274065 - timing_s/reward:0.06438687071204185 - timing_s/old_log_prob:2.6902950555086136 - timing_s/adv:0.7278527393937111 - timing_s/update_actor:18.51123914681375 - timing_s/step:78.51152851432562 - timing_s/stop_profile:2.927146852016449e-05 - timing_per_token_ms/gen:0.45769752043794715 - timing_per_token_ms/update_actor:0.08581738553493776 - timing_per_token_ms/adv:0.00337429702322019 - perf/total_num_tokens:215705 - perf/time_per_step:78.51152851432562 - perf/throughput:343.42886338125703 (TaskRunner pid=2122883) Training Progress: 12%|█▏ | 12/100 [12:53<1:38:05, 66.88s/it] (TaskRunner pid=2122883) step:13 - global_seqlen/min:13951 - global_seqlen/max:25329 - global_seqlen/minmax_diff:11378 - global_seqlen/balanced_min:20946 - global_seqlen/balanced_max:20991 - global_seqlen/mean:20964.125 - actor/entropy:0.18220646679401398 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5347471237182617 - training/rollout_probs_diff_mean:0.004718068055808544 - training/rollout_probs_diff_std:0.014215486124157906 - training/rollout_actor_probs_pearson_corr:0.9977114200592041 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.34765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9998682737350464 - rollout_corr/rollout_is_ratio_fraction_high:2.2403190087061375e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.000100814359029755 - rollout_corr/rollout_is_max:2.7408487796783447 - rollout_corr/rollout_is_min:0.15083177387714386 - rollout_corr/rollout_is_std:0.04182150959968567 - rollout_corr/rollout_is_eff_sample_size:0.9982536587188602 - rollout_corr/rollout_is_seq_mean:0.9999032616615295 - rollout_corr/rollout_is_seq_std:0.0027911446522921324 - rollout_corr/rollout_is_seq_max:1.0079991817474365 - rollout_corr/rollout_is_seq_min:0.9924850463867188 - rollout_corr/rollout_is_seq_max_deviation:0.007999181747436523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2486738041043282) - rollout_corr/training_log_ppl:np.float64(0.21914312147418968) - rollout_corr/kl:np.float64(0.001034660266309828) - rollout_corr/k3_kl:np.float64(0.0011393489194801987) - rollout_corr/rollout_ppl:np.float64(1.2473835735581815) - rollout_corr/rollout_log_ppl:np.float64(0.21810846046719234) - rollout_corr/log_ppl_diff:np.float64(0.0010346610069973394) - rollout_corr/log_ppl_abs_diff:np.float64(0.00256905252172146) - rollout_corr/log_ppl_diff_max:np.float64(0.011253908276557922) - rollout_corr/log_ppl_diff_min:np.float64(-0.00934329628944397) - rollout_corr/ppl_ratio:np.float64(1.0010404218919575) - rollout_corr/chi2_token:np.float64(0.002539546461775899) - rollout_corr/chi2_seq:np.float64(0.8009660658426583) - actor/pg_loss:np.float64(0.007912968518212438) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001622687290137037) - actor/ppo_kl:np.float64(-7.460985321472435e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.34765625) - self_distillation/token_reweight_w_mean:np.float64(457333.55283458694) - self_distillation/token_reweight_w_std:np.float64(5738632.68207643) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0023352606222034) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12474831352301408) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.6706360727548599) - perf/mfu/actor:np.float64(0.03522404308837758) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(142.29450607299805) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.34765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006558533757925034 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006558533757925034 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:348.72265625 - response_length/max:1665.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.72265625 - response_length_non_aborted/max:1665.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5972723960876465e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.497961925342679) - timing_s/agent_loop/generate_sequences/max:np.float64(10.335660284385085) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4384356094669783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.335660284385085) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1665 - timing_s/gen:16.43148054368794 - timing_s/reward:0.0586068332195282 - timing_s/old_log_prob:2.5177703872323036 - timing_s/adv:0.6034593973308802 - timing_s/update_actor:17.04535499587655 - timing_s/step:36.742922419682145 - timing_s/stop_profile:0.00011501647531986237 - timing_per_token_ms/gen:0.18405879206129444 - timing_per_token_ms/update_actor:0.1016340712757899 - timing_per_token_ms/adv:0.003598167090988058 - perf/total_num_tokens:167713 - perf/time_per_step:36.742922419682145 - perf/throughput:570.5622639523663 (TaskRunner pid=2122883) Training Progress: 13%|█▎ | 13/100 [13:29<1:23:45, 57.77s/it] (TaskRunner pid=2122883) step:14 - global_seqlen/min:12670 - global_seqlen/max:36003 - global_seqlen/minmax_diff:23333 - global_seqlen/balanced_min:21692 - global_seqlen/balanced_max:22655 - global_seqlen/mean:21828.5 - actor/entropy:0.17604441940784454 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3474702835083008 - training/rollout_probs_diff_mean:0.004344606306403875 - training/rollout_probs_diff_std:0.013226086273789406 - training/rollout_actor_probs_pearson_corr:0.9979724884033203 - self_distillation/success_group_fraction:0.53125 - self_distillation/success_sample_fraction:0.515625 - self_distillation/correct_sample_fraction:0.265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.515625 - rollout_corr/rollout_is_mean:0.9997128248214722 - rollout_corr/rollout_is_ratio_fraction_high:6.112718256190419e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.131505117285997e-05 - rollout_corr/rollout_is_max:2.5963659286499023 - rollout_corr/rollout_is_min:0.29720374941825867 - rollout_corr/rollout_is_std:0.03975746035575867 - rollout_corr/rollout_is_eff_sample_size:0.9984208288720281 - rollout_corr/rollout_is_seq_mean:0.9996223449707031 - rollout_corr/rollout_is_seq_std:0.002889478811994195 - rollout_corr/rollout_is_seq_max:1.0082389116287231 - rollout_corr/rollout_is_seq_min:0.9903880953788757 - rollout_corr/rollout_is_seq_max_deviation:0.009611904621124268 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2598396032117307) - rollout_corr/training_log_ppl:np.float64(0.22693957190494984) - rollout_corr/kl:np.float64(0.001076112781067451) - rollout_corr/k3_kl:np.float64(0.0011196938060606954) - rollout_corr/rollout_ppl:np.float64(1.2584503088146448) - rollout_corr/rollout_log_ppl:np.float64(0.2258634588069981) - rollout_corr/log_ppl_diff:np.float64(0.0010761130979517475) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023237985878949985) - rollout_corr/log_ppl_diff_max:np.float64(0.010856091976165771) - rollout_corr/log_ppl_diff_min:np.float64(-0.006189435720443726) - rollout_corr/ppl_ratio:np.float64(1.0010813102126122) - rollout_corr/chi2_token:np.float64(0.002366213593631983) - rollout_corr/chi2_seq:np.float64(0.5546517146285623) - actor/pg_loss:np.float64(0.006254076375626028) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011818245492349888) - actor/ppo_kl:np.float64(-0.0002983780827854332) - actor/pg_clipfrac_lower:np.float64(4.474513389141066e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.265625) - self_distillation/token_reweight_w_mean:np.float64(268351.6325411438) - self_distillation/token_reweight_w_std:np.float64(3542681.9470955813) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0027324005495757) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09856546502851415) - self_distillation/empty_target_batch:np.float64(0.484375) - actor/grad_norm:np.float64(0.6500967741012573) - perf/mfu/actor:np.float64(0.035992356540273746) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(142.2788848876953) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008168119937181473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008168119937181473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:383.421875 - response_length/max:3009.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:383.421875 - response_length_non_aborted/max:3009.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.47657972574234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1339652109891176) - timing_s/agent_loop/generate_sequences/max:np.float64(17.04697526805103) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.657824989699293) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.04697526805103) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:3009 - timing_s/gen:23.03968021646142 - timing_s/reward:0.059686632826924324 - timing_s/old_log_prob:2.516782818362117 - timing_s/adv:0.7135154157876968 - timing_s/update_actor:17.627836145460606 - timing_s/step:44.04473150335252 - timing_s/stop_profile:0.00011361576616764069 - timing_per_token_ms/gen:0.23472513362872793 - timing_per_token_ms/update_actor:0.10094507264276407 - timing_per_token_ms/adv:0.0040859164383014 - perf/total_num_tokens:174628 - perf/time_per_step:44.04473150335252 - perf/throughput:495.5984349305999 (TaskRunner pid=2122883) Training Progress: 14%|█▍ | 14/100 [14:14<1:16:52, 53.64s/it] (TaskRunner pid=2122883) step:15 - global_seqlen/min:15607 - global_seqlen/max:29545 - global_seqlen/minmax_diff:13938 - global_seqlen/balanced_min:19738 - global_seqlen/balanced_max:20007 - global_seqlen/mean:19790.375 - actor/entropy:0.19063463807106018 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.367106169462204 - training/rollout_probs_diff_mean:0.004759525414556265 - training/rollout_probs_diff_std:0.0138920983299613 - training/rollout_actor_probs_pearson_corr:0.997823178768158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7265625 - self_distillation/correct_sample_fraction:0.37890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7265625 - rollout_corr/rollout_is_mean:1.0001037120819092 - rollout_corr/rollout_is_ratio_fraction_high:2.396386298642028e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.585545194568112e-05 - rollout_corr/rollout_is_max:2.688039779663086 - rollout_corr/rollout_is_min:0.004564262926578522 - rollout_corr/rollout_is_std:0.040728334337472916 - rollout_corr/rollout_is_eff_sample_size:0.9983443062319399 - rollout_corr/rollout_is_seq_mean:1.0002974271774292 - rollout_corr/rollout_is_seq_std:0.003095226129516959 - rollout_corr/rollout_is_seq_max:1.0108388662338257 - rollout_corr/rollout_is_seq_min:0.9914518594741821 - rollout_corr/rollout_is_seq_max_deviation:0.010838866233825684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2595397448167205) - rollout_corr/training_log_ppl:np.float64(0.22738848751032492) - rollout_corr/kl:np.float64(0.0008545509965216525) - rollout_corr/k3_kl:np.float64(0.0012174916512890377) - rollout_corr/rollout_ppl:np.float64(1.258439245633781) - rollout_corr/rollout_log_ppl:np.float64(0.2265339351433795) - rollout_corr/log_ppl_diff:np.float64(0.0008545523669454269) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027597737425821833) - rollout_corr/log_ppl_diff_max:np.float64(0.015086829662322998) - rollout_corr/log_ppl_diff_min:np.float64(-0.010798841714859009) - rollout_corr/ppl_ratio:np.float64(1.0008610265795141) - rollout_corr/chi2_token:np.float64(0.0030557208228856325) - rollout_corr/chi2_seq:np.float64(0.7724779043346643) - actor/pg_loss:np.float64(0.00614961888641119) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016605062769485812) - actor/ppo_kl:np.float64(0.00013565428764827914) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7265625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37890625) - self_distillation/token_reweight_w_mean:np.float64(380883.05325318524) - self_distillation/token_reweight_w_std:np.float64(4762354.298607164) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0042932590004057) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12382737548614386) - self_distillation/empty_target_batch:np.float64(0.2734375) - actor/grad_norm:np.float64(0.6634487509727478) - perf/mfu/actor:np.float64(0.03345707490169501) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.15536880493164) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:1 - critic/score/mean:0.37890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.02440270036458969 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.02440270036458969 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:326.01171875 - response_length/max:1897.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:326.01171875 - response_length_non_aborted/max:1897.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:292.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00036331452429294586 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0936180874705315) - timing_s/agent_loop/generate_sequences/max:np.float64(11.154400121420622) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.176229142532975) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.154400121420622) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1897 - timing_s/gen:17.43267442472279 - timing_s/reward:0.054192231968045235 - timing_s/old_log_prob:2.6273234728723764 - timing_s/adv:0.6899376567453146 - timing_s/update_actor:16.920646134763956 - timing_s/step:37.82306046411395 - timing_s/stop_profile:0.0001196097582578659 - timing_per_token_ms/gen:0.20887710642019183 - timing_per_token_ms/update_actor:0.10687421369456084 - timing_per_token_ms/adv:0.00435778539280657 - perf/total_num_tokens:158323 - perf/time_per_step:37.82306046411395 - perf/throughput:523.2356863024572 (TaskRunner pid=2122883) Training Progress: 15%|█▌ | 15/100 [14:52<1:09:23, 48.98s/it] (TaskRunner pid=2122883) step:16 - global_seqlen/min:18530 - global_seqlen/max:33812 - global_seqlen/minmax_diff:15282 - global_seqlen/balanced_min:22661 - global_seqlen/balanced_max:23200 - global_seqlen/mean:22884.375 - actor/entropy:0.1758926510810852 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7469337582588196 - training/rollout_probs_diff_mean:0.0043508936651051044 - training/rollout_probs_diff_std:0.013909836299717426 - training/rollout_actor_probs_pearson_corr:0.9977051615715027 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:0.9997097253799438 - rollout_corr/rollout_is_ratio_fraction_high:2.0211207811371423e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015158405585680157 - rollout_corr/rollout_is_max:2.464538812637329 - rollout_corr/rollout_is_min:0.059084586799144745 - rollout_corr/rollout_is_std:0.03929905965924263 - rollout_corr/rollout_is_eff_sample_size:0.998457311961611 - rollout_corr/rollout_is_seq_mean:0.9996926188468933 - rollout_corr/rollout_is_seq_std:0.0028329973574727774 - rollout_corr/rollout_is_seq_max:1.0078116655349731 - rollout_corr/rollout_is_seq_min:0.9882426857948303 - rollout_corr/rollout_is_seq_max_deviation:0.011757314205169678 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2475387873128057) - rollout_corr/training_log_ppl:np.float64(0.21804306718695443) - rollout_corr/kl:np.float64(0.0014540559303597433) - rollout_corr/k3_kl:np.float64(0.00116658230115263) - rollout_corr/rollout_ppl:np.float64(1.245679139625281) - rollout_corr/rollout_log_ppl:np.float64(0.21658901014598086) - rollout_corr/log_ppl_diff:np.float64(0.0014540570409735665) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025474778813077137) - rollout_corr/log_ppl_diff_max:np.float64(0.013031005859375) - rollout_corr/log_ppl_diff_min:np.float64(-0.008066460490226746) - rollout_corr/ppl_ratio:np.float64(1.001460450468585) - rollout_corr/chi2_token:np.float64(0.0016516137402504683) - rollout_corr/chi2_seq:np.float64(0.45585528621450067) - actor/pg_loss:np.float64(0.00893199082929641) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0021208707877349298) - actor/ppo_kl:np.float64(0.00010913769898257542) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(445003.87521046656) - self_distillation/token_reweight_w_std:np.float64(5454650.55214807) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015151123516262) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14990338099596556) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.7883509397506714) - perf/mfu/actor:np.float64(0.03755929023339058) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.36421585083008) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:1 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.027472082525491714 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.027472082525491714 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:386.54296875 - response_length/max:2789.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:386.54296875 - response_length_non_aborted/max:2789.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.59375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013099610805511475 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9211164209991693) - timing_s/agent_loop/generate_sequences/max:np.float64(16.113097727298737) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5992245447123423) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.113097727298737) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2789 - timing_s/gen:22.836567513644695 - timing_s/reward:0.060344504192471504 - timing_s/old_log_prob:2.496923489496112 - timing_s/adv:0.7712550535798073 - timing_s/update_actor:17.49838000163436 - timing_s/step:43.75420778989792 - timing_s/stop_profile:0.00011534243822097778 - timing_per_token_ms/gen:0.2307772978994967 - timing_per_token_ms/update_actor:0.09558039055924818 - timing_per_token_ms/adv:0.004212781939531926 - perf/total_num_tokens:183075 - perf/time_per_step:43.75420778989792 - perf/throughput:523.0211254169617 (TaskRunner pid=2122883) Training Progress: 16%|█▌ | 16/100 [15:36<1:06:23, 47.43s/it] (TaskRunner pid=2122883) step:17 - global_seqlen/min:11563 - global_seqlen/max:28003 - global_seqlen/minmax_diff:16440 - global_seqlen/balanced_min:19351 - global_seqlen/balanced_max:19748 - global_seqlen/mean:19445.875 - actor/entropy:0.20303893089294434 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5752326250076294 - training/rollout_probs_diff_mean:0.005234991200268269 - training/rollout_probs_diff_std:0.014962500892579556 - training/rollout_actor_probs_pearson_corr:0.9976497292518616 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7265625 - self_distillation/correct_sample_fraction:0.32421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7265625 - rollout_corr/rollout_is_mean:0.9999104142189026 - rollout_corr/rollout_is_ratio_fraction_high:6.119126919656992e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017133555957116187 - rollout_corr/rollout_is_max:3.0671093463897705 - rollout_corr/rollout_is_min:0.11059963703155518 - rollout_corr/rollout_is_std:0.04376046359539032 - rollout_corr/rollout_is_eff_sample_size:0.9980883258424627 - rollout_corr/rollout_is_seq_mean:0.9999815821647644 - rollout_corr/rollout_is_seq_std:0.0031001772731542587 - rollout_corr/rollout_is_seq_max:1.0088906288146973 - rollout_corr/rollout_is_seq_min:0.9919108748435974 - rollout_corr/rollout_is_seq_max_deviation:0.008890628814697266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2740392214618623) - rollout_corr/training_log_ppl:np.float64(0.2388933266774984) - rollout_corr/kl:np.float64(0.0015981203938295607) - rollout_corr/k3_kl:np.float64(0.0013284954461596499) - rollout_corr/rollout_ppl:np.float64(1.2719845059327781) - rollout_corr/rollout_log_ppl:np.float64(0.23729520468623377) - rollout_corr/log_ppl_diff:np.float64(0.0015981219912646338) - rollout_corr/log_ppl_abs_diff:np.float64(0.002778403417323716) - rollout_corr/log_ppl_diff_max:np.float64(0.012680396437644958) - rollout_corr/log_ppl_diff_min:np.float64(-0.008751362562179565) - rollout_corr/ppl_ratio:np.float64(1.0016051123384386) - rollout_corr/chi2_token:np.float64(0.0021377962548285723) - rollout_corr/chi2_seq:np.float64(0.34157682210206985) - actor/pg_loss:np.float64(0.006546365446411073) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002759105423024266) - actor/ppo_kl:np.float64(0.0004309166163753986) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7265625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.32421875) - self_distillation/token_reweight_w_mean:np.float64(577455.9122446231) - self_distillation/token_reweight_w_std:np.float64(6992234.631524403) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00423149834387) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14134635338268708) - self_distillation/empty_target_batch:np.float64(0.2734375) - actor/grad_norm:np.float64(0.8037751317024231) - perf/mfu/actor:np.float64(0.03241075740075369) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.29125595092773) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:1 - critic/score/mean:0.32421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.32421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.02262394316494465 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.02262394316494465 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:319.18359375 - response_length/max:2055.0 - response_length/min:83.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:319.18359375 - response_length_non_aborted/max:2055.0 - response_length_non_aborted/min:83.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001981891691684723 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9945226181298494) - timing_s/agent_loop/generate_sequences/max:np.float64(12.014069978147745) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.190817145477922) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.014069978147745) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2055 - timing_s/gen:17.9083558563143 - timing_s/reward:0.057533686980605125 - timing_s/old_log_prob:2.403095867484808 - timing_s/adv:0.6608398947864771 - timing_s/update_actor:17.16264975257218 - timing_s/step:38.281539618968964 - timing_s/stop_profile:0.00010946765542030334 - timing_per_token_ms/gen:0.2191670136984531 - timing_per_token_ms/update_actor:0.11032320320229984 - timing_per_token_ms/adv:0.004247943939180399 - perf/total_num_tokens:155567 - perf/time_per_step:38.281539618968964 - perf/throughput:507.9700344749022 (TaskRunner pid=2122883) Training Progress: 17%|█▋ | 17/100 [16:14<1:01:49, 44.69s/it] (TaskRunner pid=2122883) step:18 - global_seqlen/min:12223 - global_seqlen/max:32263 - global_seqlen/minmax_diff:20040 - global_seqlen/balanced_min:21577 - global_seqlen/balanced_max:21639 - global_seqlen/mean:21614.375 - actor/entropy:0.1750251203775406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3790506422519684 - training/rollout_probs_diff_mean:0.004264963325113058 - training/rollout_probs_diff_std:0.013276408426463604 - training/rollout_actor_probs_pearson_corr:0.9978918433189392 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0000395774841309 - rollout_corr/rollout_is_ratio_fraction_high:3.2373984140576795e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.553929754067212e-05 - rollout_corr/rollout_is_max:2.1670196056365967 - rollout_corr/rollout_is_min:0.12257643789052963 - rollout_corr/rollout_is_std:0.03953269124031067 - rollout_corr/rollout_is_eff_sample_size:0.9984397236898125 - rollout_corr/rollout_is_seq_mean:1.0000652074813843 - rollout_corr/rollout_is_seq_std:0.0029307748191058636 - rollout_corr/rollout_is_seq_max:1.0118049383163452 - rollout_corr/rollout_is_seq_min:0.988494873046875 - rollout_corr/rollout_is_seq_max_deviation:0.011804938316345215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2621581708081067) - rollout_corr/training_log_ppl:np.float64(0.2286899130413076) - rollout_corr/kl:np.float64(0.001075027593940625) - rollout_corr/k3_kl:np.float64(0.0012590645611680884) - rollout_corr/rollout_ppl:np.float64(1.260771079454571) - rollout_corr/rollout_log_ppl:np.float64(0.22761488259129692) - rollout_corr/log_ppl_diff:np.float64(0.0010750304500106722) - rollout_corr/log_ppl_abs_diff:np.float64(0.002587559982202947) - rollout_corr/log_ppl_diff_max:np.float64(0.012028634548187256) - rollout_corr/log_ppl_diff_min:np.float64(-0.015134572982788086) - rollout_corr/ppl_ratio:np.float64(1.0010810499079525) - rollout_corr/chi2_token:np.float64(0.0028927032835781574) - rollout_corr/chi2_seq:np.float64(1.2977143267635256) - actor/pg_loss:np.float64(0.00664244475774467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001839155015659344) - actor/ppo_kl:np.float64(0.00014196148600653657) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.359375) - self_distillation/token_reweight_w_mean:np.float64(623915.2676190655) - self_distillation/token_reweight_w_std:np.float64(7410794.226804514) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002825409406796) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1299431096558692) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.7296064049005508) - perf/mfu/actor:np.float64(0.036137395670919115) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.27211380004883) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:1 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002475260989740491 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002475260989740491 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.98046875 - response_length/max:2043.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.98046875 - response_length_non_aborted/max:2043.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.46875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.489618241786957e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.046178888529539) - timing_s/agent_loop/generate_sequences/max:np.float64(12.541748790070415) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4098203388639376) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.541748790070415) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:2043 - timing_s/gen:18.420527707785368 - timing_s/reward:0.060600074008107185 - timing_s/old_log_prob:2.4621987380087376 - timing_s/adv:0.54259735904634 - timing_s/update_actor:17.234087236225605 - timing_s/step:38.808553241193295 - timing_s/stop_profile:0.00011383742094039917 - timing_per_token_ms/gen:0.1987819580625829 - timing_per_token_ms/update_actor:0.09966797117789437 - timing_per_token_ms/adv:0.0031379426830890323 - perf/total_num_tokens:172915 - perf/time_per_step:38.808553241193295 - perf/throughput:556.9487444086797 (TaskRunner pid=2122883) Training Progress: 18%|█▊ | 18/100 [16:53<58:41, 42.94s/it] (TaskRunner pid=2122883) step:19 - global_seqlen/min:14234 - global_seqlen/max:27408 - global_seqlen/minmax_diff:13174 - global_seqlen/balanced_min:20990 - global_seqlen/balanced_max:21463 - global_seqlen/mean:21081.0 - actor/entropy:0.18662618100643158 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4825435280799866 - training/rollout_probs_diff_mean:0.004559643566608429 - training/rollout_probs_diff_std:0.013524374924600124 - training/rollout_actor_probs_pearson_corr:0.9978901147842407 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.6953125 - self_distillation/correct_sample_fraction:0.328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6953125 - rollout_corr/rollout_is_mean:1.0000455379486084 - rollout_corr/rollout_is_ratio_fraction_high:4.3297539377817884e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011906823783647269 - rollout_corr/rollout_is_max:4.395936012268066 - rollout_corr/rollout_is_min:0.13285663723945618 - rollout_corr/rollout_is_std:0.04109692573547363 - rollout_corr/rollout_is_eff_sample_size:0.9983140093600031 - rollout_corr/rollout_is_seq_mean:1.00007164478302 - rollout_corr/rollout_is_seq_std:0.0030954221729189157 - rollout_corr/rollout_is_seq_max:1.011547565460205 - rollout_corr/rollout_is_seq_min:0.9888474345207214 - rollout_corr/rollout_is_seq_max_deviation:0.011547565460205078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2576180850155652) - rollout_corr/training_log_ppl:np.float64(0.22579659259645268) - rollout_corr/kl:np.float64(0.0012318556480068743) - rollout_corr/k3_kl:np.float64(0.001264088400887431) - rollout_corr/rollout_ppl:np.float64(1.256009896285832) - rollout_corr/rollout_log_ppl:np.float64(0.22456473541387822) - rollout_corr/log_ppl_diff:np.float64(0.0012318571825744584) - rollout_corr/log_ppl_abs_diff:np.float64(0.002706136365304701) - rollout_corr/log_ppl_diff_max:np.float64(0.013555288314819336) - rollout_corr/log_ppl_diff_min:np.float64(-0.009048610925674438) - rollout_corr/ppl_ratio:np.float64(1.0012385330628604) - rollout_corr/chi2_token:np.float64(0.002662915736436844) - rollout_corr/chi2_seq:np.float64(0.6268946786876768) - actor/pg_loss:np.float64(0.007152381585910916) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0019675058147186064) - actor/ppo_kl:np.float64(0.00022251309200527203) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.328125) - self_distillation/token_reweight_w_mean:np.float64(426680.76371138333) - self_distillation/token_reweight_w_std:np.float64(5434152.009025335) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0037426524795592) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13176339765777811) - self_distillation/empty_target_batch:np.float64(0.3046875) - actor/grad_norm:np.float64(0.7264038622379303) - perf/mfu/actor:np.float64(0.035313067432871936) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.18925714492798) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:1 - critic/score/mean:0.328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02298964187502861 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02298964187502861 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.875 - response_length/max:2247.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.875 - response_length_non_aborted/max:2247.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.90625 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011053308844566345 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1291147619485855) - timing_s/agent_loop/generate_sequences/max:np.float64(13.091949408873916) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.563902350026183) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.091949408873916) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:2247 - timing_s/gen:19.058726267889142 - timing_s/reward:0.0590360201895237 - timing_s/old_log_prob:2.6232305262237787 - timing_s/adv:0.5377237945795059 - timing_s/update_actor:17.08998142555356 - timing_s/step:39.45427840948105 - timing_s/stop_profile:0.00012236833572387695 - timing_per_token_ms/gen:0.20629899406703694 - timing_per_token_ms/update_actor:0.10133521551132275 - timing_per_token_ms/adv:0.0031884386092897983 - perf/total_num_tokens:168648 - perf/time_per_step:39.45427840948105 - perf/throughput:534.3146763757346 (TaskRunner pid=2122883) Training Progress: 19%|█▉ | 19/100 [17:32<56:34, 41.91s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques such as molecular dynamics simulations, thermodynamic analysis, or energy minimization. However, without specific computational tools or databases that provide such scores for the given sequence, it is not possible to calculate or determine the exact folding stability score. The options provided (A: -0.01, B: 1.69, C: 2.49, D: 0.45) suggest possible scores, but none can be definitively determined without further information or computation. Therefore, the correct answer cannot be determined from the given information. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) D (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 0.0 (TaskRunner pid=2122883) [acc] 0.0 (TaskRunner pid=2122883) [pred] D (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2122883) step:20 - global_seqlen/min:11490 - global_seqlen/max:30712 - global_seqlen/minmax_diff:19222 - global_seqlen/balanced_min:21053 - global_seqlen/balanced_max:21083 - global_seqlen/mean:21076.125 - actor/entropy:0.19185848534107208 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.605046272277832 - training/rollout_probs_diff_mean:0.004658793099224567 - training/rollout_probs_diff_std:0.013916742987930775 - training/rollout_actor_probs_pearson_corr:0.997868001461029 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.6953125 - self_distillation/correct_sample_fraction:0.36328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6953125 - rollout_corr/rollout_is_mean:1.000085473060608 - rollout_corr/rollout_is_ratio_fraction_high:5.654509368469007e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014701724285259843 - rollout_corr/rollout_is_max:3.1271145343780518 - rollout_corr/rollout_is_min:0.24705207347869873 - rollout_corr/rollout_is_std:0.041785866022109985 - rollout_corr/rollout_is_eff_sample_size:0.9982572225303701 - rollout_corr/rollout_is_seq_mean:1.0000156164169312 - rollout_corr/rollout_is_seq_std:0.002887738635763526 - rollout_corr/rollout_is_seq_max:1.0120712518692017 - rollout_corr/rollout_is_seq_min:0.992546796798706 - rollout_corr/rollout_is_seq_max_deviation:0.01207125186920166 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2571831136010587) - rollout_corr/training_log_ppl:np.float64(0.22537461307365447) - rollout_corr/kl:np.float64(0.0011899686502321316) - rollout_corr/k3_kl:np.float64(0.0011254083661924597) - rollout_corr/rollout_ppl:np.float64(1.255645914003253) - rollout_corr/rollout_log_ppl:np.float64(0.22418464263319038) - rollout_corr/log_ppl_diff:np.float64(0.0011899704404640943) - rollout_corr/log_ppl_abs_diff:np.float64(0.002389262692304328) - rollout_corr/log_ppl_diff_max:np.float64(0.010065972805023193) - rollout_corr/log_ppl_diff_min:np.float64(-0.006951630115509033) - rollout_corr/ppl_ratio:np.float64(1.001195044722408) - rollout_corr/chi2_token:np.float64(0.0022096477914601564) - rollout_corr/chi2_seq:np.float64(3.44644234678708) - actor/pg_loss:np.float64(0.006273084436543286) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014769084930321696) - actor/ppo_kl:np.float64(0.00015581477314441372) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.36328125) - self_distillation/token_reweight_w_mean:np.float64(431294.86135452124) - self_distillation/token_reweight_w_std:np.float64(5506743.177664222) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.003677363274619) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12259233223448973) - self_distillation/empty_target_batch:np.float64(0.3046875) - actor/grad_norm:np.float64(0.6836053431034088) - perf/mfu/actor:np.float64(0.03593170496025079) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.2952423095703) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.32875) - val-aux/sciknoweval/reward/std@16:np.float64(0.29435610050286326) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.45084) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2764717653668717) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.20032000000000003) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.22610926818200705) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.32604) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.29362819649402) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5661599999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.2170131088439427) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.11420000000000002) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.13524791805681496) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.33177999999999996) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.2486997734125882) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6531) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.14949042862248849) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.06770000000000001) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06467335374646328) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.34218) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.20269370137809958) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.71174) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.09224539666924036) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.04794) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.025378175348871202) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.34806000000000004) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.15403633275085624) - val-aux/sciknoweval/score/mean@16:np.float64(0.32875) - val-aux/sciknoweval/score/std@16:np.float64(0.29435610050286326) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.45084) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2764717653668717) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.20032000000000003) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.22610926818200705) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.32604) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.29362819649402) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5661599999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.2170131088439427) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.11420000000000002) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.13524791805681496) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.33177999999999996) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.2486997734125882) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6531) - val-aux/sciknoweval/score/best@8/std:np.float64(0.14949042862248849) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.06770000000000001) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06467335374646328) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.34218) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.20269370137809958) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.71174) - val-aux/sciknoweval/score/best@16/std:np.float64(0.09224539666924036) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.04794) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.025378175348871202) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.34806000000000004) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.15403633275085624) - val-core/sciknoweval/acc/mean@16:np.float64(0.32875) - val-aux/sciknoweval/acc/std@16:np.float64(0.29435610050286326) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.45084) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2764717653668717) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.20032000000000003) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.22610926818200705) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.32604) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.29362819649402) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5661599999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.2170131088439427) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.11420000000000002) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.13524791805681496) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.33177999999999996) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.2486997734125882) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6531) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.14949042862248849) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.06770000000000001) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06467335374646328) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.34218) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.20269370137809958) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.71174) - val-core/sciknoweval/acc/best@16/std:np.float64(0.09224539666924036) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.04794) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.025378175348871202) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.34806000000000004) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.15403633275085624) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03862677168945583) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99836) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0139506864431683) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9766399999999998) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.050116218604848536) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.98808) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.037513975347298376) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0012642784503423287) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.95852) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.061058789332634845) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99606) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021621459553489996) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9308999999999998) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06627509521077407) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9989999999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.010527221260855181) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8992) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05853224575927146) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999600000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0012642784503423287) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:1 - critic/score/mean:0.36328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.36328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0057266042567789555 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0057266042567789555 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.41015625 - response_length/max:1419.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.41015625 - response_length_non_aborted/max:1419.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013404525816440582 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0222893320024014) - timing_s/agent_loop/generate_sequences/max:np.float64(9.325520938262343) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3090429624644457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.325520938262343) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1419 - timing_s/gen:15.228843804448843 - timing_s/reward:0.05905533581972122 - timing_s/old_log_prob:2.497826090082526 - timing_s/adv:0.5414953380823135 - timing_s/update_actor:16.95986382290721 - timing_s/step:35.37535096332431 - timing_s/testing:77.24785147979856 - timing_s/save_checkpoint:7.0980935245752335 - timing_s/stop_profile:0.00025027431547641754 - timing_per_token_ms/gen:0.1722232830585111 - timing_per_token_ms/update_actor:0.10058694270713431 - timing_per_token_ms/adv:0.0032115446867149056 - perf/total_num_tokens:168609 - perf/time_per_step:35.37535096332431 - perf/throughput:595.7856085117247 (TaskRunner pid=2122883) Training Progress: 20%|██ | 20/100 [19:32<1:27:03, 65.30s/it] (TaskRunner pid=2122883) step:21 - global_seqlen/min:13618 - global_seqlen/max:28101 - global_seqlen/minmax_diff:14483 - global_seqlen/balanced_min:19845 - global_seqlen/balanced_max:19859 - global_seqlen/mean:19853.75 - actor/entropy:0.1921236366033554 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8342254161834717 - training/rollout_probs_diff_mean:0.004657784476876259 - training/rollout_probs_diff_std:0.013867578469216824 - training/rollout_actor_probs_pearson_corr:0.9979041814804077 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9997999668121338 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.753607547376305e-05 - rollout_corr/rollout_is_max:1.7968099117279053 - rollout_corr/rollout_is_min:0.10106302052736282 - rollout_corr/rollout_is_std:0.040148328989744186 - rollout_corr/rollout_is_eff_sample_size:0.9983901708780174 - rollout_corr/rollout_is_seq_mean:0.9999698400497437 - rollout_corr/rollout_is_seq_std:0.0028422540053725243 - rollout_corr/rollout_is_seq_max:1.00948166847229 - rollout_corr/rollout_is_seq_min:0.993533730506897 - rollout_corr/rollout_is_seq_max_deviation:0.009481668472290039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2641725991852582) - rollout_corr/training_log_ppl:np.float64(0.23089192975021433) - rollout_corr/kl:np.float64(0.001236026230320597) - rollout_corr/k3_kl:np.float64(0.0011104728028499267) - rollout_corr/rollout_ppl:np.float64(1.2625776953063905) - rollout_corr/rollout_log_ppl:np.float64(0.22965590309468098) - rollout_corr/log_ppl_diff:np.float64(0.001236026655533351) - rollout_corr/log_ppl_abs_diff:np.float64(0.002413631693343632) - rollout_corr/log_ppl_diff_max:np.float64(0.010084569454193115) - rollout_corr/log_ppl_diff_min:np.float64(-0.006885886192321777) - rollout_corr/ppl_ratio:np.float64(1.0012409987393767) - rollout_corr/chi2_token:np.float64(0.0019542204681783915) - rollout_corr/chi2_seq:np.float64(0.2917343757580966) - actor/pg_loss:np.float64(0.007957952562719584) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018407491854759428) - actor/ppo_kl:np.float64(0.00021987110629595463) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.390625) - self_distillation/token_reweight_w_mean:np.float64(397908.3636041009) - self_distillation/token_reweight_w_std:np.float64(4692473.746526102) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004959150683135) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14321871772699524) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.7271679639816284) - perf/mfu/actor:np.float64(0.033430629892783156) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.5677375793457) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:1 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013565568253397942 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013565568253397942 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:339.4609375 - response_length/max:1403.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:339.4609375 - response_length_non_aborted/max:1403.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.96875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.517954170703888e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0614673774689436) - timing_s/agent_loop/generate_sequences/max:np.float64(9.238231776282191) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.313919332322257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.238231776282191) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1392 - timing_s/gen:14.915440939366817 - timing_s/reward:0.051850782707333565 - timing_s/old_log_prob:2.5148690696805716 - timing_s/adv:0.6088764388114214 - timing_s/update_actor:17.02783625945449 - timing_s/step:35.21148547902703 - timing_s/stop_profile:0.00012161023914813995 - timing_per_token_ms/gen:0.17163518606438077 - timing_per_token_ms/update_actor:0.10720793464367241 - timing_per_token_ms/adv:0.00383351028654172 - perf/total_num_tokens:158830 - perf/time_per_step:35.21148547902703 - perf/throughput:563.8430111625214 (TaskRunner pid=2122883) Training Progress: 21%|██ | 21/100 [20:07<1:14:06, 56.29s/it] (TaskRunner pid=2122883) step:22 - global_seqlen/min:16977 - global_seqlen/max:37961 - global_seqlen/minmax_diff:20984 - global_seqlen/balanced_min:23830 - global_seqlen/balanced_max:24580 - global_seqlen/mean:24009.875 - actor/entropy:0.17836344242095947 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3530232608318329 - training/rollout_probs_diff_mean:0.0041275168769061565 - training/rollout_probs_diff_std:0.012802300043404102 - training/rollout_actor_probs_pearson_corr:0.9980253577232361 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001649856567383 - rollout_corr/rollout_is_ratio_fraction_high:2.90407824650174e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.80815649300348e-05 - rollout_corr/rollout_is_max:3.3773062229156494 - rollout_corr/rollout_is_min:0.16175149381160736 - rollout_corr/rollout_is_std:0.03797151893377304 - rollout_corr/rollout_is_eff_sample_size:0.9985608338735662 - rollout_corr/rollout_is_seq_mean:1.000291109085083 - rollout_corr/rollout_is_seq_std:0.0029728421941399574 - rollout_corr/rollout_is_seq_max:1.0122034549713135 - rollout_corr/rollout_is_seq_min:0.9913510084152222 - rollout_corr/rollout_is_seq_max_deviation:0.012203454971313477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2535668751224875) - rollout_corr/training_log_ppl:np.float64(0.22167573400656693) - rollout_corr/kl:np.float64(0.0007851657160911074) - rollout_corr/k3_kl:np.float64(0.0010411924005779838) - rollout_corr/rollout_ppl:np.float64(1.2525088130496442) - rollout_corr/rollout_log_ppl:np.float64(0.22089056581899058) - rollout_corr/log_ppl_diff:np.float64(0.0007851681875763461) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021511910745175555) - rollout_corr/log_ppl_diff_max:np.float64(0.016502737998962402) - rollout_corr/log_ppl_diff_min:np.float64(-0.007668659090995789) - rollout_corr/ppl_ratio:np.float64(1.0007898204494268) - rollout_corr/chi2_token:np.float64(0.002642852021381259) - rollout_corr/chi2_seq:np.float64(1.0275017379317433) - actor/pg_loss:np.float64(0.0077134715393185616) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001129955593341947) - actor/ppo_kl:np.float64(0.00012342048217206525) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(395338.3554381167) - self_distillation/token_reweight_w_std:np.float64(5012417.347034392) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001086757518351) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1375344575062627) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.7418015450239182) - perf/mfu/actor:np.float64(0.039820292543893625) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.55425262451172) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0037849820218980312 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0037849820218980312 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:403.52734375 - response_length/max:2614.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:403.52734375 - response_length_non_aborted/max:2614.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.398426532745361e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0202389620244503) - timing_s/agent_loop/generate_sequences/max:np.float64(15.357408994808793) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9177660304267192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.357408994808793) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:2614 - timing_s/gen:21.221346884965897 - timing_s/reward:0.06203651241958141 - timing_s/old_log_prob:2.4261272829025984 - timing_s/adv:0.6985401939600706 - timing_s/update_actor:17.410948403179646 - timing_s/step:41.90618650615215 - timing_s/stop_profile:3.1556934118270874e-05 - timing_per_token_ms/gen:0.20542817619009995 - timing_per_token_ms/update_actor:0.0906447264051752 - timing_per_token_ms/adv:0.003636733812442123 - perf/total_num_tokens:192079 - perf/time_per_step:41.90618650615215 - perf/throughput:572.9434482537599 (TaskRunner pid=2122883) Training Progress: 22%|██▏ | 22/100 [20:49<1:07:34, 51.98s/it] (TaskRunner pid=2122883) step:23 - global_seqlen/min:17524 - global_seqlen/max:33089 - global_seqlen/minmax_diff:15565 - global_seqlen/balanced_min:22610 - global_seqlen/balanced_max:22638 - global_seqlen/mean:22633.0 - actor/entropy:0.19653943181037903 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4863523840904236 - training/rollout_probs_diff_mean:0.004470366053283215 - training/rollout_probs_diff_std:0.013256288133561611 - training/rollout_actor_probs_pearson_corr:0.9980781674385071 - self_distillation/success_group_fraction:0.5625 - self_distillation/success_sample_fraction:0.5546875 - self_distillation/correct_sample_fraction:0.265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5546875 - rollout_corr/rollout_is_mean:0.9998451471328735 - rollout_corr/rollout_is_ratio_fraction_high:2.0983716240152717e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010491858120076358 - rollout_corr/rollout_is_max:2.2246580123901367 - rollout_corr/rollout_is_min:0.2494165003299713 - rollout_corr/rollout_is_std:0.04001970589160919 - rollout_corr/rollout_is_eff_sample_size:0.9984005088283843 - rollout_corr/rollout_is_seq_mean:0.9997673034667969 - rollout_corr/rollout_is_seq_std:0.002586677670478821 - rollout_corr/rollout_is_seq_max:1.009292721748352 - rollout_corr/rollout_is_seq_min:0.9929970502853394 - rollout_corr/rollout_is_seq_max_deviation:0.00929272174835205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2631977992132306) - rollout_corr/training_log_ppl:np.float64(0.22998240585729945) - rollout_corr/kl:np.float64(0.0012839085080340595) - rollout_corr/k3_kl:np.float64(0.0011792379793860164) - rollout_corr/rollout_ppl:np.float64(1.261534459888935) - rollout_corr/rollout_log_ppl:np.float64(0.22869849536800757) - rollout_corr/log_ppl_diff:np.float64(0.001283910489291884) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025792625528993085) - rollout_corr/log_ppl_diff_max:np.float64(0.014777734875679016) - rollout_corr/log_ppl_diff_min:np.float64(-0.011869877576828003) - rollout_corr/ppl_ratio:np.float64(1.001290408661589) - rollout_corr/chi2_token:np.float64(0.002155705587938428) - rollout_corr/chi2_seq:np.float64(1.0525127614382654) - actor/pg_loss:np.float64(0.006339410902000964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016231782274189754) - actor/ppo_kl:np.float64(9.229942891020304e-05) - actor/pg_clipfrac_lower:np.float64(9.481189408688806e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5546875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.265625) - self_distillation/token_reweight_w_mean:np.float64(356218.2028367857) - self_distillation/token_reweight_w_std:np.float64(4629386.739539903) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010050446726382) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10248490553931333) - self_distillation/empty_target_batch:np.float64(0.4453125) - actor/grad_norm:np.float64(0.6276970654726028) - perf/mfu/actor:np.float64(0.03810892654843721) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.44520616531372) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009594804607331753 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009594804607331753 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:372.3125 - response_length/max:1512.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:372.3125 - response_length_non_aborted/max:1512.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:334.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.639476537704468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5703063905239105) - timing_s/agent_loop/generate_sequences/max:np.float64(9.733793312683702) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6518209086207207) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.733793312683702) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1512 - timing_s/gen:15.654936438426375 - timing_s/reward:0.060526276007294655 - timing_s/old_log_prob:2.4749402347952127 - timing_s/adv:0.5394423864781857 - timing_s/update_actor:16.798127077519894 - timing_s/step:35.61569402553141 - timing_s/stop_profile:0.00011201202869415283 - timing_per_token_ms/gen:0.16424937508840834 - timing_per_token_ms/update_actor:0.09277452766712264 - timing_per_token_ms/adv:0.0029792912256339506 - perf/total_num_tokens:181064 - perf/time_per_step:35.61569402553141 - perf/throughput:635.4782805516957 (TaskRunner pid=2122883) Training Progress: 23%|██▎ | 23/100 [21:25<1:00:25, 47.08s/it] (TaskRunner pid=2122883) step:24 - global_seqlen/min:12679 - global_seqlen/max:29738 - global_seqlen/minmax_diff:17059 - global_seqlen/balanced_min:18589 - global_seqlen/balanced_max:18667 - global_seqlen/mean:18608.25 - actor/entropy:0.21970593929290771 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5529158115386963 - training/rollout_probs_diff_mean:0.005068894941359758 - training/rollout_probs_diff_std:0.013992728665471077 - training/rollout_actor_probs_pearson_corr:0.9980122447013855 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.0020556449890137 - rollout_corr/rollout_is_ratio_fraction_high:3.916346759069711e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.138142922893167e-05 - rollout_corr/rollout_is_max:179.5236053466797 - rollout_corr/rollout_is_min:0.09267795085906982 - rollout_corr/rollout_is_std:0.042068056762218475 - rollout_corr/rollout_is_eff_sample_size:0.998232513960617 - rollout_corr/rollout_is_seq_mean:1.0005260705947876 - rollout_corr/rollout_is_seq_std:0.013086115941405296 - rollout_corr/rollout_is_seq_max:1.2028028964996338 - rollout_corr/rollout_is_seq_min:0.9902984499931335 - rollout_corr/rollout_is_seq_max_deviation:0.2028028964996338 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2807346642948687) - rollout_corr/training_log_ppl:np.float64(0.24428033985896036) - rollout_corr/kl:np.float64(0.001533059075450538) - rollout_corr/k3_kl:np.float64(0.001212075309354077) - rollout_corr/rollout_ppl:np.float64(1.2787316790781915) - rollout_corr/rollout_log_ppl:np.float64(0.24274727902957238) - rollout_corr/log_ppl_diff:np.float64(0.0015330608293879777) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029383527289610356) - rollout_corr/log_ppl_diff_max:np.float64(0.012038901448249817) - rollout_corr/log_ppl_diff_min:np.float64(-0.007719486951828003) - rollout_corr/ppl_ratio:np.float64(1.0015401267446578) - rollout_corr/chi2_token:np.float64(0.005337873008102179) - rollout_corr/chi2_seq:np.float64(0.7134281191974878) - actor/pg_loss:np.float64(0.007017654599621892) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001894508582608978) - actor/ppo_kl:np.float64(0.00026200509176987907) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(555852.1664180572) - self_distillation/token_reweight_w_std:np.float64(6403345.388541037) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0016352159436792) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13323489562026225) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.6621586382389069) - perf/mfu/actor:np.float64(0.031465761930366626) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.51604080200195) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005094514694064856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005094514694064856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.2265625 - response_length/max:1621.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.2265625 - response_length_non_aborted/max:1621.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015429407358169556 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2271409928798676) - timing_s/agent_loop/generate_sequences/max:np.float64(9.899323958903551) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9376148904048023) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.899323958903551) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1621 - timing_s/gen:15.829845121130347 - timing_s/reward:0.056639643386006355 - timing_s/old_log_prob:2.396368984133005 - timing_s/adv:0.5699858013540506 - timing_s/update_actor:16.899894734844565 - timing_s/step:35.841407015919685 - timing_s/stop_profile:0.00012869946658611298 - timing_per_token_ms/gen:0.20665054595350443 - timing_per_token_ms/update_actor:0.11352420791076918 - timing_per_token_ms/adv:0.0038288514593933514 - perf/total_num_tokens:148866 - perf/time_per_step:35.841407015919685 - perf/throughput:519.1830217975196 (TaskRunner pid=2122883) Training Progress: 24%|██▍ | 24/100 [22:01<55:22, 43.72s/it] (TaskRunner pid=2122883) step:25 - global_seqlen/min:17225 - global_seqlen/max:30301 - global_seqlen/minmax_diff:13076 - global_seqlen/balanced_min:25937 - global_seqlen/balanced_max:25982 - global_seqlen/mean:25966.875 - actor/entropy:0.1851537972688675 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5962941646575928 - training/rollout_probs_diff_mean:0.004132953006774187 - training/rollout_probs_diff_std:0.013088012114167213 - training/rollout_actor_probs_pearson_corr:0.9980089664459229 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.79296875 - self_distillation/correct_sample_fraction:0.40234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.79296875 - rollout_corr/rollout_is_mean:1.0000680685043335 - rollout_corr/rollout_is_ratio_fraction_high:3.482652027742006e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016542596858926117 - rollout_corr/rollout_is_max:2.9371936321258545 - rollout_corr/rollout_is_min:0.0107397036626935 - rollout_corr/rollout_is_std:0.038702256977558136 - rollout_corr/rollout_is_eff_sample_size:0.9985047320789843 - rollout_corr/rollout_is_seq_mean:1.0000791549682617 - rollout_corr/rollout_is_seq_std:0.002427029889076948 - rollout_corr/rollout_is_seq_max:1.0064553022384644 - rollout_corr/rollout_is_seq_min:0.9905536770820618 - rollout_corr/rollout_is_seq_max_deviation:0.009446322917938232 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.254920766223222) - rollout_corr/training_log_ppl:np.float64(0.22319715823687147) - rollout_corr/kl:np.float64(0.0009513927656978893) - rollout_corr/k3_kl:np.float64(0.001108306266019099) - rollout_corr/rollout_ppl:np.float64(1.253699362743646) - rollout_corr/rollout_log_ppl:np.float64(0.22224576224107295) - rollout_corr/log_ppl_diff:np.float64(0.0009513959957985207) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020687162032118067) - rollout_corr/log_ppl_diff_max:np.float64(0.012234926223754883) - rollout_corr/log_ppl_diff_min:np.float64(-0.015981703996658325) - rollout_corr/ppl_ratio:np.float64(1.0009557630401105) - rollout_corr/chi2_token:np.float64(0.002664993517100811) - rollout_corr/chi2_seq:np.float64(1.0995744443498552) - actor/pg_loss:np.float64(0.006765639060176909) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001574875209144011) - actor/ppo_kl:np.float64(3.649607397449017e-05) - actor/pg_clipfrac_lower:np.float64(4.1273967781307874e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.79296875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.79296875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40234375) - self_distillation/token_reweight_w_mean:np.float64(353427.5153041347) - self_distillation/token_reweight_w_std:np.float64(5117519.624401459) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020341072231531) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14214718576113228) - self_distillation/empty_target_batch:np.float64(0.20703125) - actor/grad_norm:np.float64(0.6468942165374756) - perf/mfu/actor:np.float64(0.04292664526543912) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.73140716552734) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003380349138751626 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003380349138751626 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:448.65234375 - response_length/max:2031.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:448.65234375 - response_length_non_aborted/max:2031.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.8125 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011907331645488739 - timing_s/agent_loop/generate_sequences/min:np.float64(1.524836651980877) - timing_s/agent_loop/generate_sequences/max:np.float64(13.144861306995153) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.346475338068558) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.144861306995153) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:2031 - timing_s/gen:18.811984948813915 - timing_s/reward:0.06409545615315437 - timing_s/old_log_prob:2.5523454006761312 - timing_s/adv:0.6613637115806341 - timing_s/update_actor:17.477145528420806 - timing_s/step:39.65423706173897 - timing_s/stop_profile:0.00011950545012950897 - timing_per_token_ms/gen:0.1637889943738968 - timing_per_token_ms/update_actor:0.08413192542624404 - timing_per_token_ms/adv:0.0031836893714618825 - perf/total_num_tokens:207735 - perf/time_per_step:39.65423706173897 - perf/throughput:654.8322934462547 (TaskRunner pid=2122883) Training Progress: 25%|██▌ | 25/100 [22:40<53:08, 42.52s/it] (TaskRunner pid=2122883) step:26 - global_seqlen/min:13828 - global_seqlen/max:27110 - global_seqlen/minmax_diff:13282 - global_seqlen/balanced_min:22022 - global_seqlen/balanced_max:22201 - global_seqlen/mean:22057.875 - actor/entropy:0.20123563706874847 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27729910612106323 - training/rollout_probs_diff_mean:0.0043639508076012135 - training/rollout_probs_diff_std:0.012655162252485752 - training/rollout_actor_probs_pearson_corr:0.998265266418457 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:1.0000321865081787 - rollout_corr/rollout_is_ratio_fraction_high:6.163265061331913e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014380952052306384 - rollout_corr/rollout_is_max:3.1523871421813965 - rollout_corr/rollout_is_min:0.22010482847690582 - rollout_corr/rollout_is_std:0.03991381824016571 - rollout_corr/rollout_is_eff_sample_size:0.9984094210263972 - rollout_corr/rollout_is_seq_mean:1.0000900030136108 - rollout_corr/rollout_is_seq_std:0.0028636239003390074 - rollout_corr/rollout_is_seq_max:1.009320616722107 - rollout_corr/rollout_is_seq_min:0.9920967221260071 - rollout_corr/rollout_is_seq_max_deviation:0.009320616722106934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.285409896634519) - rollout_corr/training_log_ppl:np.float64(0.24616308329859748) - rollout_corr/kl:np.float64(0.0010952132663355485) - rollout_corr/k3_kl:np.float64(0.0010522639622649876) - rollout_corr/rollout_ppl:np.float64(1.2839300143532455) - rollout_corr/rollout_log_ppl:np.float64(0.24506786806159653) - rollout_corr/log_ppl_diff:np.float64(0.0010952152370009571) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022666600707452744) - rollout_corr/log_ppl_diff_max:np.float64(0.009045511484146118) - rollout_corr/log_ppl_diff_min:np.float64(-0.008367002010345459) - rollout_corr/ppl_ratio:np.float64(1.0010996800847352) - rollout_corr/chi2_token:np.float64(0.002009735209867358) - rollout_corr/chi2_seq:np.float64(0.3117946165148169) - actor/pg_loss:np.float64(0.006661892984993756) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010209764229784923) - actor/ppo_kl:np.float64(0.0002103924846554861) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(235097.68068203423) - self_distillation/token_reweight_w_std:np.float64(3208859.9599546287) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0031206426210701) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11432849438278936) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.6480773836374283) - perf/mfu/actor:np.float64(0.03729586141445862) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.68837356567383) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012705313973128796 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.012705313973128796 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:380.27734375 - response_length/max:2030.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:380.27734375 - response_length_non_aborted/max:2030.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.533390402793884e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2987001277506351) - timing_s/agent_loop/generate_sequences/max:np.float64(12.182028014212847) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6352716596229584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.182028014212847) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:2030 - timing_s/gen:18.078122118487954 - timing_s/reward:0.05934295244514942 - timing_s/old_log_prob:2.4004305750131607 - timing_s/adv:0.5496773198246956 - timing_s/update_actor:16.90547569282353 - timing_s/step:38.081087835133076 - timing_s/stop_profile:0.00012539885938167572 - timing_per_token_ms/gen:0.18570042545518745 - timing_per_token_ms/update_actor:0.09580181507071471 - timing_per_token_ms/adv:0.003114972089473122 - perf/total_num_tokens:176463 - perf/time_per_step:38.081087835133076 - perf/throughput:579.2343720719479 (TaskRunner pid=2122883) Training Progress: 26%|██▌ | 26/100 [23:19<50:49, 41.20s/it] (TaskRunner pid=2122883) step:27 - global_seqlen/min:13161 - global_seqlen/max:28209 - global_seqlen/minmax_diff:15048 - global_seqlen/balanced_min:21333 - global_seqlen/balanced_max:23462 - global_seqlen/mean:21600.625 - actor/entropy:0.2077469378709793 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8482835292816162 - training/rollout_probs_diff_mean:0.004673023708164692 - training/rollout_probs_diff_std:0.013583921827375889 - training/rollout_actor_probs_pearson_corr:0.9980661869049072 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.3984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:0.9999101161956787 - rollout_corr/rollout_is_ratio_fraction_high:3.2460156944580376e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001298406277783215 - rollout_corr/rollout_is_max:2.862781047821045 - rollout_corr/rollout_is_min:0.08208515495061874 - rollout_corr/rollout_is_std:0.040227677673101425 - rollout_corr/rollout_is_eff_sample_size:0.9983839919754216 - rollout_corr/rollout_is_seq_mean:1.0000474452972412 - rollout_corr/rollout_is_seq_std:0.002930537099018693 - rollout_corr/rollout_is_seq_max:1.0101268291473389 - rollout_corr/rollout_is_seq_min:0.9879516363143921 - rollout_corr/rollout_is_seq_max_deviation:0.01204836368560791 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2790318825282156) - rollout_corr/training_log_ppl:np.float64(0.24226842416101135) - rollout_corr/kl:np.float64(0.0012188144196088047) - rollout_corr/k3_kl:np.float64(0.0012009678286233338) - rollout_corr/rollout_ppl:np.float64(1.277424986474216) - rollout_corr/rollout_log_ppl:np.float64(0.2410496080556186) - rollout_corr/log_ppl_diff:np.float64(0.0012188161053927615) - rollout_corr/log_ppl_abs_diff:np.float64(0.002619924969621934) - rollout_corr/log_ppl_diff_max:np.float64(0.013451695442199707) - rollout_corr/log_ppl_diff_min:np.float64(-0.007775008678436279) - rollout_corr/ppl_ratio:np.float64(1.0012246232945472) - rollout_corr/chi2_token:np.float64(0.0023596512619405985) - rollout_corr/chi2_seq:np.float64(0.5256561439018697) - actor/pg_loss:np.float64(0.004449836676940322) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013233652471171808) - actor/ppo_kl:np.float64(0.00026928278293070207) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3984375) - self_distillation/token_reweight_w_mean:np.float64(324819.42211573245) - self_distillation/token_reweight_w_std:np.float64(4507005.021220025) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001485469052568) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10996017501747701) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.5749102309346199) - perf/mfu/actor:np.float64(0.03605009092181908) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.96850204467773) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.3984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.027892740443348885 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.027892740443348885 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:361.01953125 - response_length/max:3699.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:361.01953125 - response_length_non_aborted/max:3699.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.0 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.86289232969284e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.594991670921445) - timing_s/agent_loop/generate_sequences/max:np.float64(19.769734585657716) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.591588780793245) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.769734585657716) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:3699 - timing_s/gen:25.607453675940633 - timing_s/reward:0.1120842956006527 - timing_s/old_log_prob:4.264432176947594 - timing_s/adv:0.5280121117830276 - timing_s/update_actor:17.25321297906339 - timing_s/step:47.86772947758436 - timing_s/stop_profile:0.00012224353849887848 - timing_per_token_ms/gen:0.2770739731872695 - timing_per_token_ms/update_actor:0.09984209356826129 - timing_per_token_ms/adv:0.003055537234356805 - perf/total_num_tokens:172805 - perf/time_per_step:47.86772947758436 - perf/throughput:451.2565194911783 (TaskRunner pid=2122883) Training Progress: 27%|██▋ | 27/100 [24:07<52:34, 43.22s/it] (TaskRunner pid=2122883) step:28 - global_seqlen/min:20190 - global_seqlen/max:30705 - global_seqlen/minmax_diff:10515 - global_seqlen/balanced_min:24004 - global_seqlen/balanced_max:24200 - global_seqlen/mean:24035.75 - actor/entropy:0.18564951419830322 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5823308825492859 - training/rollout_probs_diff_mean:0.004079106263816357 - training/rollout_probs_diff_std:0.012803311459720135 - training/rollout_actor_probs_pearson_corr:0.998200535774231 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.61328125 - self_distillation/correct_sample_fraction:0.35546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.61328125 - rollout_corr/rollout_is_mean:1.0000269412994385 - rollout_corr/rollout_is_ratio_fraction_high:1.9231139958719723e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.692455983487889e-05 - rollout_corr/rollout_is_max:6.242704391479492 - rollout_corr/rollout_is_min:0.23590925335884094 - rollout_corr/rollout_is_std:0.03762301430106163 - rollout_corr/rollout_is_eff_sample_size:0.9985865096788494 - rollout_corr/rollout_is_seq_mean:1.0000364780426025 - rollout_corr/rollout_is_seq_std:0.0030162427574396133 - rollout_corr/rollout_is_seq_max:1.0107171535491943 - rollout_corr/rollout_is_seq_min:0.9907180666923523 - rollout_corr/rollout_is_seq_max_deviation:0.010717153549194336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.25667032180354) - rollout_corr/training_log_ppl:np.float64(0.22454505835776217) - rollout_corr/kl:np.float64(0.0008663110850345435) - rollout_corr/k3_kl:np.float64(0.0010345751965132877) - rollout_corr/rollout_ppl:np.float64(1.2555355904623866) - rollout_corr/rollout_log_ppl:np.float64(0.22367874927294906) - rollout_corr/log_ppl_diff:np.float64(0.0008663090848131105) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023031624587019905) - rollout_corr/log_ppl_diff_max:np.float64(0.010847002267837524) - rollout_corr/log_ppl_diff_min:np.float64(-0.008875399827957153) - rollout_corr/ppl_ratio:np.float64(1.0008710592519492) - rollout_corr/chi2_token:np.float64(0.002394293434917927) - rollout_corr/chi2_seq:np.float64(0.8952626511454582) - actor/pg_loss:np.float64(0.005589029053226113) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011645046297417139) - actor/ppo_kl:np.float64(-2.7968522828381026e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.61328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.35546875) - self_distillation/token_reweight_w_mean:np.float64(274457.87072749576) - self_distillation/token_reweight_w_std:np.float64(3580252.375037855) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010159488301724) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0962374477967387) - self_distillation/empty_target_batch:np.float64(0.38671875) - actor/grad_norm:np.float64(0.5859245583415031) - perf/mfu/actor:np.float64(0.039479981257269675) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(149.19495010375977) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002572164870798588 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002572164870798588 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.2421875 - response_length/max:2082.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.2421875 - response_length_non_aborted/max:2082.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:344.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.947402238845825e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5013809334486723) - timing_s/agent_loop/generate_sequences/max:np.float64(13.167735269293189) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.06728039180598) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.167735269293189) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:2082 - timing_s/gen:18.91630212403834 - timing_s/reward:0.06258906796574593 - timing_s/old_log_prob:2.4403274692595005 - timing_s/adv:0.5469215027987957 - timing_s/update_actor:17.414515420794487 - timing_s/step:39.468442087993026 - timing_s/stop_profile:0.00012546591460704803 - timing_per_token_ms/gen:0.18189101832764418 - timing_per_token_ms/update_actor:0.09056569599863998 - timing_per_token_ms/adv:0.0028443126530209983 - perf/total_num_tokens:192286 - perf/time_per_step:39.468442087993026 - perf/throughput:608.9865403456623 (TaskRunner pid=2122883) Training Progress: 28%|██▊ | 28/100 [24:46<50:31, 42.11s/it] (TaskRunner pid=2122883) step:29 - global_seqlen/min:12202 - global_seqlen/max:28063 - global_seqlen/minmax_diff:15861 - global_seqlen/balanced_min:19899 - global_seqlen/balanced_max:19913 - global_seqlen/mean:19906.5 - actor/entropy:0.200120210647583 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.751859188079834 - training/rollout_probs_diff_mean:0.00466552097350359 - training/rollout_probs_diff_std:0.013881048187613487 - training/rollout_actor_probs_pearson_corr:0.9979689717292786 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9998300671577454 - rollout_corr/rollout_is_ratio_fraction_high:3.581490818760358e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001790745445759967 - rollout_corr/rollout_is_max:2.2318549156188965 - rollout_corr/rollout_is_min:0.07922481745481491 - rollout_corr/rollout_is_std:0.04076636582612991 - rollout_corr/rollout_is_eff_sample_size:0.9983403853566886 - rollout_corr/rollout_is_seq_mean:0.9997302293777466 - rollout_corr/rollout_is_seq_std:0.0028203565161675215 - rollout_corr/rollout_is_seq_max:1.0098977088928223 - rollout_corr/rollout_is_seq_min:0.9884768724441528 - rollout_corr/rollout_is_seq_max_deviation:0.011523127555847168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.268780852202326) - rollout_corr/training_log_ppl:np.float64(0.23475862003397197) - rollout_corr/kl:np.float64(0.0012591780972996958) - rollout_corr/k3_kl:np.float64(0.0014774430976558506) - rollout_corr/rollout_ppl:np.float64(1.2671397938393056) - rollout_corr/rollout_log_ppl:np.float64(0.23349944039364345) - rollout_corr/log_ppl_diff:np.float64(0.0012591796403285116) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028744479350280017) - rollout_corr/log_ppl_diff_max:np.float64(0.015908241271972656) - rollout_corr/log_ppl_diff_min:np.float64(-0.008600950241088867) - rollout_corr/ppl_ratio:np.float64(1.001266701379791) - rollout_corr/chi2_token:np.float64(0.003420991124585271) - rollout_corr/chi2_seq:np.float64(0.8588014850392938) - actor/pg_loss:np.float64(0.008906474453397095) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020709222528694227) - actor/ppo_kl:np.float64(-5.746460594657776e-05) - actor/pg_clipfrac_lower:np.float64(1.800115205696784e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(400565.17963223695) - self_distillation/token_reweight_w_std:np.float64(4787453.377732925) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003657368943095) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14216147220577113) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.782679945230484) - perf/mfu/actor:np.float64(0.03311267643104445) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.62977409362793) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:2 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.018026838079094887 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.018026838079094887 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:327.203125 - response_length/max:1399.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.203125 - response_length_non_aborted/max:1399.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00024269148707389832 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3502626232802868) - timing_s/agent_loop/generate_sequences/max:np.float64(8.990872727707028) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.25896129708417) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.990872727707028) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1399 - timing_s/gen:15.044752394780517 - timing_s/reward:0.05442681908607483 - timing_s/old_log_prob:2.595429075881839 - timing_s/adv:0.6318970713764429 - timing_s/update_actor:17.295033048838377 - timing_s/step:35.68772314488888 - timing_s/stop_profile:0.00011236034333705902 - timing_per_token_ms/gen:0.17960881040519217 - timing_per_token_ms/update_actor:0.1086016693594955 - timing_per_token_ms/adv:0.003967906659737039 - perf/total_num_tokens:159252 - perf/time_per_step:35.68772314488888 - perf/throughput:557.796862500346 (TaskRunner pid=2122883) Training Progress: 29%|██▉ | 29/100 [25:22<47:42, 40.32s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods such as energy calculations, structure prediction algorithms, or experimental techniques like circular dichroism or NMR. However, without specific computational tools or data provided, it is not possible to calculate the exact folding stability score for the given sequence. The question provides four possible scores, and without additional context or computational data, the correct answer cannot be determined through logical reasoning alone. In the absence of specific information, the answer must be selected from the given options. Based on typical folding stability scores, the most reasonable choice is 0.45. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) D (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 0.0 (TaskRunner pid=2122883) [acc] 0.0 (TaskRunner pid=2122883) [pred] D (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2122883) step:30 - global_seqlen/min:13709 - global_seqlen/max:34661 - global_seqlen/minmax_diff:20952 - global_seqlen/balanced_min:22528 - global_seqlen/balanced_max:22562 - global_seqlen/mean:22552.5 - actor/entropy:0.17858071625232697 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8049387335777283 - training/rollout_probs_diff_mean:0.004061528015881777 - training/rollout_probs_diff_std:0.012804483063519001 - training/rollout_actor_probs_pearson_corr:0.9980953335762024 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.703125 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.703125 - rollout_corr/rollout_is_mean:1.0000133514404297 - rollout_corr/rollout_is_ratio_fraction_high:2.0044097254867665e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012026457989122719 - rollout_corr/rollout_is_max:2.6347193717956543 - rollout_corr/rollout_is_min:0.1211763396859169 - rollout_corr/rollout_is_std:0.03737824037671089 - rollout_corr/rollout_is_eff_sample_size:0.9986048163833603 - rollout_corr/rollout_is_seq_mean:1.0000509023666382 - rollout_corr/rollout_is_seq_std:0.002637238474562764 - rollout_corr/rollout_is_seq_max:1.0075513124465942 - rollout_corr/rollout_is_seq_min:0.9900543093681335 - rollout_corr/rollout_is_seq_max_deviation:0.009945690631866455 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2358674858696759) - rollout_corr/training_log_ppl:np.float64(0.209219249751186) - rollout_corr/kl:np.float64(0.0009768563257512852) - rollout_corr/k3_kl:np.float64(0.000992308898958072) - rollout_corr/rollout_ppl:np.float64(1.2346490789204836) - rollout_corr/rollout_log_ppl:np.float64(0.2082423923711758) - rollout_corr/log_ppl_diff:np.float64(0.0009768573800101876) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023018353385850787) - rollout_corr/log_ppl_diff_max:np.float64(0.01429024338722229) - rollout_corr/log_ppl_diff_min:np.float64(-0.008996948599815369) - rollout_corr/ppl_ratio:np.float64(1.0009819658007473) - rollout_corr/chi2_token:np.float64(0.0020118139218539) - rollout_corr/chi2_seq:np.float64(0.9899320646654814) - actor/pg_loss:np.float64(0.005333056440576911) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007797006080636493) - actor/ppo_kl:np.float64(0.00013445281576274226) - actor/pg_clipfrac_lower:np.float64(5.455656264530262e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(224932.845420863) - self_distillation/token_reweight_w_std:np.float64(3210198.2877844325) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003551209811121) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08737150816887151) - self_distillation/empty_target_batch:np.float64(0.296875) - actor/grad_norm:np.float64(0.4824788570404053) - perf/mfu/actor:np.float64(0.03816828358530715) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.78467178344727) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.3625) - val-aux/sciknoweval/reward/std@16:np.float64(0.2837411495944002) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.48234) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.23591887606702464) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.24532) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.24386037303171185) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.3643) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28320348160575504) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.57412) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1685773153105991) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.1459) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.18180547299046135) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.37833999999999995) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23334153448406283) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.63576) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.11234238956935315) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.07646) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11232669827952936) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.39524000000000004) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1772001294855729) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.68072) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07299155127558939) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.0341) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06470539559920649) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.40950000000000003) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13626189541452347) - val-aux/sciknoweval/score/mean@16:np.float64(0.3625) - val-aux/sciknoweval/score/std@16:np.float64(0.2837411495944002) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.48234) - val-aux/sciknoweval/score/best@2/std:np.float64(0.23591887606702464) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.24532) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.24386037303171185) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.3643) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.28320348160575504) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.57412) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1685773153105991) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.1459) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.18180547299046135) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.37833999999999995) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23334153448406283) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.63576) - val-aux/sciknoweval/score/best@8/std:np.float64(0.11234238956935315) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.07646) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11232669827952936) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.39524000000000004) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1772001294855729) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.68072) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07299155127558939) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.0341) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06470539559920649) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.40950000000000003) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13626189541452347) - val-core/sciknoweval/acc/mean@16:np.float64(0.3625) - val-aux/sciknoweval/acc/std@16:np.float64(0.2837411495944002) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.48234) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.23591887606702464) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.24532) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.24386037303171185) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.3643) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.28320348160575504) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.57412) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1685773153105991) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.1459) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.18180547299046135) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.37833999999999995) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23334153448406283) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.63576) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.11234238956935315) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.07646) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11232669827952936) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.39524000000000004) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1772001294855729) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.68072) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07299155127558939) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.0341) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06470539559920649) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.40950000000000003) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13626189541452347) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.014523687548277814) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99976) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.003714879512080724) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9930199999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.01923124573696262) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99664) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.013794079474263716) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.98634) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.02515921626743821) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99882) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.008310850581084193) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9766800000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0292466087573783) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99988) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0021559110231923074) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.96274) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.029101825861985193) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:2 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003663058625534177 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003663058625534177 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:389.765625 - response_length/max:1653.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:389.765625 - response_length_non_aborted/max:1653.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.0 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010188855230808258 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4586653970181942) - timing_s/agent_loop/generate_sequences/max:np.float64(10.59416701644659) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8259682986681582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.59416701644659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1653 - timing_s/gen:16.57048099115491 - timing_s/reward:0.06174835376441479 - timing_s/old_log_prob:2.359806180000305 - timing_s/adv:0.5572093706578016 - timing_s/update_actor:17.026692068204284 - timing_s/step:36.666580606251955 - timing_s/testing:73.57902169972658 - timing_s/save_checkpoint:6.81705866008997 - timing_s/stop_profile:0.000125223770737648 - timing_per_token_ms/gen:0.16607016427294957 - timing_per_token_ms/update_actor:0.09437253113958699 - timing_per_token_ms/adv:0.0030884013449606566 - perf/total_num_tokens:180420 - perf/time_per_step:36.666580606251955 - perf/throughput:615.0696254494648 (TaskRunner pid=2122883) Training Progress: 30%|███ | 30/100 [27:19<1:13:55, 63.36s/it] (TaskRunner pid=2122883) step:31 - global_seqlen/min:14691 - global_seqlen/max:32016 - global_seqlen/minmax_diff:17325 - global_seqlen/balanced_min:21111 - global_seqlen/balanced_max:23092 - global_seqlen/mean:21372.875 - actor/entropy:0.18767856061458588 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564788341522217 - training/rollout_probs_diff_mean:0.0042718215845525265 - training/rollout_probs_diff_std:0.012846951372921467 - training/rollout_actor_probs_pearson_corr:0.9981147050857544 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.640625 - self_distillation/correct_sample_fraction:0.3828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.640625 - rollout_corr/rollout_is_mean:0.9999961256980896 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.654637425323017e-05 - rollout_corr/rollout_is_max:1.8308576345443726 - rollout_corr/rollout_is_min:0.2759072780609131 - rollout_corr/rollout_is_std:0.038326188921928406 - rollout_corr/rollout_is_eff_sample_size:0.9985331387675961 - rollout_corr/rollout_is_seq_mean:1.0000230073928833 - rollout_corr/rollout_is_seq_std:0.002981269033625722 - rollout_corr/rollout_is_seq_max:1.0086121559143066 - rollout_corr/rollout_is_seq_min:0.9902340173721313 - rollout_corr/rollout_is_seq_max_deviation:0.009765982627868652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2463521412573755) - rollout_corr/training_log_ppl:np.float64(0.21727092210494447) - rollout_corr/kl:np.float64(0.0009897176428506071) - rollout_corr/k3_kl:np.float64(0.0010703946789476504) - rollout_corr/rollout_ppl:np.float64(1.2450866210274398) - rollout_corr/rollout_log_ppl:np.float64(0.21628120230161585) - rollout_corr/log_ppl_diff:np.float64(0.0009897198033286259) - rollout_corr/log_ppl_abs_diff:np.float64(0.002593131721368991) - rollout_corr/log_ppl_diff_max:np.float64(0.013398468494415283) - rollout_corr/log_ppl_diff_min:np.float64(-0.008827343583106995) - rollout_corr/ppl_ratio:np.float64(1.0009958858136088) - rollout_corr/chi2_token:np.float64(0.0023120150435715914) - rollout_corr/chi2_seq:np.float64(0.798170599155128) - actor/pg_loss:np.float64(0.004743849392980337) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009500816249783384) - actor/ppo_kl:np.float64(0.00011712995112134195) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.640625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3828125) - self_distillation/token_reweight_w_mean:np.float64(405654.28687982494) - self_distillation/token_reweight_w_std:np.float64(5009496.861719544) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019375609699637) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10904305649455637) - self_distillation/empty_target_batch:np.float64(0.359375) - actor/grad_norm:np.float64(0.5787822306156158) - perf/mfu/actor:np.float64(0.03477636111962868) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.78802490234375) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:2 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031128779519349337 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031128779519349337 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:345.40234375 - response_length/max:3325.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:345.40234375 - response_length_non_aborted/max:3325.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001608133316040039 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1231673527508974) - timing_s/agent_loop/generate_sequences/max:np.float64(18.172121800482273) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3470864338232786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.172121800482273) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:3325 - timing_s/gen:23.99966412782669 - timing_s/reward:0.05674261972308159 - timing_s/old_log_prob:2.4767898563295603 - timing_s/adv:0.7946092803031206 - timing_s/update_actor:17.73706099949777 - timing_s/step:45.163662960752845 - timing_s/stop_profile:0.00011231191456317902 - timing_per_token_ms/gen:0.2714187951983838 - timing_per_token_ms/update_actor:0.10373581583840365 - timing_per_token_ms/adv:0.004647299908781111 - perf/total_num_tokens:170983 - perf/time_per_step:45.163662960752845 - perf/throughput:473.23165569127985 (TaskRunner pid=2122883) Training Progress: 31%|███ | 31/100 [28:05<1:06:36, 57.92s/it] (TaskRunner pid=2122883) step:32 - global_seqlen/min:16951 - global_seqlen/max:28535 - global_seqlen/minmax_diff:11584 - global_seqlen/balanced_min:22298 - global_seqlen/balanced_max:22302 - global_seqlen/mean:22300.0 - actor/entropy:0.18449454009532928 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.508357584476471 - training/rollout_probs_diff_mean:0.004164127632975578 - training/rollout_probs_diff_std:0.012944702059030533 - training/rollout_actor_probs_pearson_corr:0.9981325268745422 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:1.0003676414489746 - rollout_corr/rollout_is_ratio_fraction_high:4.371011527837254e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.834775846684352e-05 - rollout_corr/rollout_is_max:2.890216827392578 - rollout_corr/rollout_is_min:0.006242398638278246 - rollout_corr/rollout_is_std:0.038238998502492905 - rollout_corr/rollout_is_eff_sample_size:0.9985409835772864 - rollout_corr/rollout_is_seq_mean:1.0004096031188965 - rollout_corr/rollout_is_seq_std:0.0029842921067029238 - rollout_corr/rollout_is_seq_max:1.0088164806365967 - rollout_corr/rollout_is_seq_min:0.9924617409706116 - rollout_corr/rollout_is_seq_max_deviation:0.00881648063659668 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2568057388998568) - rollout_corr/training_log_ppl:np.float64(0.22407739836489782) - rollout_corr/kl:np.float64(0.0006947221124469216) - rollout_corr/k3_kl:np.float64(0.0011480630730034136) - rollout_corr/rollout_ppl:np.float64(1.2559078889898956) - rollout_corr/rollout_log_ppl:np.float64(0.22338267411396373) - rollout_corr/log_ppl_diff:np.float64(0.0006947242509340867) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024179289903258905) - rollout_corr/log_ppl_diff_max:np.float64(0.014996767044067383) - rollout_corr/log_ppl_diff_min:np.float64(-0.00923621654510498) - rollout_corr/ppl_ratio:np.float64(1.000700187170878) - rollout_corr/chi2_token:np.float64(0.0031753990333527327) - rollout_corr/chi2_seq:np.float64(0.8116656658239663) - actor/pg_loss:np.float64(0.006690458743833005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001721466425806284) - actor/ppo_kl:np.float64(0.0001673200823937293) - actor/pg_clipfrac_lower:np.float64(7.89141449786257e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(378276.8863981217) - self_distillation/token_reweight_w_std:np.float64(4767918.035825024) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.99896085774526) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13528311986010522) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.7006156742572784) - perf/mfu/actor:np.float64(0.03776695122166555) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.70980548858643) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:2 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023166360333561897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023166360333561897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:357.46875 - response_length/max:1160.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:357.46875 - response_length_non_aborted/max:1160.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:339.40625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.496323764324188e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1231606528162956) - timing_s/agent_loop/generate_sequences/max:np.float64(8.21338595636189) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.436626313610759) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.21338595636189) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1160 - timing_s/gen:14.110106036067009 - timing_s/reward:0.05653159320354462 - timing_s/old_log_prob:2.364559957757592 - timing_s/adv:0.5321131441742182 - timing_s/update_actor:17.017004685476422 - timing_s/step:34.17690355889499 - timing_s/stop_profile:0.00011146441102027893 - timing_per_token_ms/gen:0.15418858768322197 - timing_per_token_ms/update_actor:0.09538679756432972 - timing_per_token_ms/adv:0.0029826969964922546 - perf/total_num_tokens:178400 - perf/time_per_step:34.17690355889499 - perf/throughput:652.4874309216386 (TaskRunner pid=2122883) Training Progress: 32%|███▏ | 32/100 [28:39<57:35, 50.81s/it] (TaskRunner pid=2122883) step:33 - global_seqlen/min:13524 - global_seqlen/max:31971 - global_seqlen/minmax_diff:18447 - global_seqlen/balanced_min:23235 - global_seqlen/balanced_max:23263 - global_seqlen/mean:23257.125 - actor/entropy:0.18245993554592133 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8571684956550598 - training/rollout_probs_diff_mean:0.004132695496082306 - training/rollout_probs_diff_std:0.013101710006594658 - training/rollout_actor_probs_pearson_corr:0.9980323314666748 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000072956085205 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00011418250505812466 - rollout_corr/rollout_is_max:1.9873102903366089 - rollout_corr/rollout_is_min:0.05242491513490677 - rollout_corr/rollout_is_std:0.038057755678892136 - rollout_corr/rollout_is_eff_sample_size:0.9985539396692393 - rollout_corr/rollout_is_seq_mean:1.0000982284545898 - rollout_corr/rollout_is_seq_std:0.002705891616642475 - rollout_corr/rollout_is_seq_max:1.0117093324661255 - rollout_corr/rollout_is_seq_min:0.9912111759185791 - rollout_corr/rollout_is_seq_max_deviation:0.011709332466125488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2483157757669687) - rollout_corr/training_log_ppl:np.float64(0.21742995794920716) - rollout_corr/kl:np.float64(0.001100853100311383) - rollout_corr/k3_kl:np.float64(0.0011483149023092665) - rollout_corr/rollout_ppl:np.float64(1.2468630676157773) - rollout_corr/rollout_log_ppl:np.float64(0.21632910224434454) - rollout_corr/log_ppl_diff:np.float64(0.0011008557048626244) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024914654495660216) - rollout_corr/log_ppl_diff_max:np.float64(0.01017981767654419) - rollout_corr/log_ppl_diff_min:np.float64(-0.013981759548187256) - rollout_corr/ppl_ratio:np.float64(1.001106784446165) - rollout_corr/chi2_token:np.float64(0.0023680569138377905) - rollout_corr/chi2_seq:np.float64(3.1154466739390045) - actor/pg_loss:np.float64(0.006939625251106918) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017952929888451763) - actor/ppo_kl:np.float64(0.00026459825367375345) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.484375) - self_distillation/token_reweight_w_mean:np.float64(452102.9714282772) - self_distillation/token_reweight_w_std:np.float64(5585046.191044878) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017326495144516) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12278274190612137) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.6988479644060135) - perf/mfu/actor:np.float64(0.038683908285535445) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7328338623047) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:2 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007901948411017656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007901948411017656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:376.31640625 - response_length/max:1468.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:376.31640625 - response_length_non_aborted/max:1468.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:350.46875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.475834667682648e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3122014049440622) - timing_s/agent_loop/generate_sequences/max:np.float64(9.940794771537185) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.687579854595242) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.940794771537185) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1468 - timing_s/gen:15.825185041874647 - timing_s/reward:0.06117861159145832 - timing_s/old_log_prob:2.3811042718589306 - timing_s/adv:0.5547373704612255 - timing_s/update_actor:17.3419884853065 - timing_s/step:36.251486498862505 - timing_s/stop_profile:0.00011686794459819794 - timing_per_token_ms/gen:0.16426902479706287 - timing_per_token_ms/update_actor:0.09320793351127074 - timing_per_token_ms/adv:0.002981545281613836 - perf/total_num_tokens:186057 - perf/time_per_step:36.251486498862505 - perf/throughput:641.5495541328977 (TaskRunner pid=2122883) Training Progress: 33%|███▎ | 33/100 [29:15<51:52, 46.46s/it] (TaskRunner pid=2122883) step:34 - global_seqlen/min:12850 - global_seqlen/max:27960 - global_seqlen/minmax_diff:15110 - global_seqlen/balanced_min:19894 - global_seqlen/balanced_max:19913 - global_seqlen/mean:19909.5 - actor/entropy:0.1824377477169037 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8196942806243896 - training/rollout_probs_diff_mean:0.004456874914467335 - training/rollout_probs_diff_std:0.013786375522613525 - training/rollout_actor_probs_pearson_corr:0.9977785348892212 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.37890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9999569654464722 - rollout_corr/rollout_is_ratio_fraction_high:6.199474591994658e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001363884366583079 - rollout_corr/rollout_is_max:5.546136856079102 - rollout_corr/rollout_is_min:0.31902217864990234 - rollout_corr/rollout_is_std:0.04029429703950882 - rollout_corr/rollout_is_eff_sample_size:0.9983787637329614 - rollout_corr/rollout_is_seq_mean:0.9998646378517151 - rollout_corr/rollout_is_seq_std:0.0028597437776625156 - rollout_corr/rollout_is_seq_max:1.0103107690811157 - rollout_corr/rollout_is_seq_min:0.992056131362915 - rollout_corr/rollout_is_seq_max_deviation:0.010310769081115723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.236810412723571) - rollout_corr/training_log_ppl:np.float64(0.2092786459106719) - rollout_corr/kl:np.float64(0.0015006025356960606) - rollout_corr/k3_kl:np.float64(0.0015799338922874995) - rollout_corr/rollout_ppl:np.float64(1.2348826574161649) - rollout_corr/rollout_log_ppl:np.float64(0.20777804144017864) - rollout_corr/log_ppl_diff:np.float64(0.001500604470493272) - rollout_corr/log_ppl_abs_diff:np.float64(0.002933290699729696) - rollout_corr/log_ppl_diff_max:np.float64(0.01829591393470764) - rollout_corr/log_ppl_diff_min:np.float64(-0.015577703714370728) - rollout_corr/ppl_ratio:np.float64(1.0015093663241714) - rollout_corr/chi2_token:np.float64(0.003512197406962514) - rollout_corr/chi2_seq:np.float64(1.3710486507043242) - actor/pg_loss:np.float64(0.005380709771998227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017725324896673555) - actor/ppo_kl:np.float64(0.00034614508305708114) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37890625) - self_distillation/token_reweight_w_mean:np.float64(492608.6426240674) - self_distillation/token_reweight_w_std:np.float64(5396903.603151628) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017004983965307) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09993290061538573) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.6676810383796692) - perf/mfu/actor:np.float64(0.033570212113188724) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.6746826171875) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:2 - critic/score/mean:0.37890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00712474575266242 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00712474575266242 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:315.046875 - response_length/max:1277.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.046875 - response_length_non_aborted/max:1277.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.605847299098969e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2697386275976896) - timing_s/agent_loop/generate_sequences/max:np.float64(8.63854743912816) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0711136244499357) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.63854743912816) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:1277 - timing_s/gen:14.853934342041612 - timing_s/reward:0.05592981539666653 - timing_s/old_log_prob:2.4257997516542673 - timing_s/adv:0.5230183508247137 - timing_s/update_actor:16.553620226681232 - timing_s/step:34.49927463196218 - timing_s/stop_profile:0.00011586584150791168 - timing_per_token_ms/gen:0.18417316795667327 - timing_per_token_ms/update_actor:0.1039304115289261 - timing_per_token_ms/adv:0.003283723541680565 - perf/total_num_tokens:159276 - perf/time_per_step:34.49927463196218 - perf/throughput:577.099090122743 (TaskRunner pid=2122883) Training Progress: 34%|███▍ | 34/100 [29:50<47:10, 42.88s/it] (TaskRunner pid=2122883) step:35 - global_seqlen/min:11375 - global_seqlen/max:30625 - global_seqlen/minmax_diff:19250 - global_seqlen/balanced_min:20219 - global_seqlen/balanced_max:25072 - global_seqlen/mean:20838.875 - actor/entropy:0.16443729400634766 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5101940035820007 - training/rollout_probs_diff_mean:0.003969142679125071 - training/rollout_probs_diff_std:0.012587097473442554 - training/rollout_actor_probs_pearson_corr:0.9980363845825195 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9996427893638611 - rollout_corr/rollout_is_ratio_fraction_high:2.3485479687224142e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014091287448536605 - rollout_corr/rollout_is_max:2.2103183269500732 - rollout_corr/rollout_is_min:0.18281003832817078 - rollout_corr/rollout_is_std:0.03684016689658165 - rollout_corr/rollout_is_eff_sample_size:0.9986435717515305 - rollout_corr/rollout_is_seq_mean:0.9995678663253784 - rollout_corr/rollout_is_seq_std:0.0028214508201926947 - rollout_corr/rollout_is_seq_max:1.007384181022644 - rollout_corr/rollout_is_seq_min:0.9897209405899048 - rollout_corr/rollout_is_seq_max_deviation:0.010279059410095215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2260203030891716) - rollout_corr/training_log_ppl:np.float64(0.2011226593167521) - rollout_corr/kl:np.float64(0.0013201789048196133) - rollout_corr/k3_kl:np.float64(0.0011584838699150168) - rollout_corr/rollout_ppl:np.float64(1.2243667864240706) - rollout_corr/rollout_log_ppl:np.float64(0.1998024799686391) - rollout_corr/log_ppl_diff:np.float64(0.0013201793481130153) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027976697601843625) - rollout_corr/log_ppl_diff_max:np.float64(0.01783977448940277) - rollout_corr/log_ppl_diff_min:np.float64(-0.006872743368148804) - rollout_corr/ppl_ratio:np.float64(1.001327101374045) - rollout_corr/chi2_token:np.float64(0.0018826127052307129) - rollout_corr/chi2_seq:np.float64(0.425460196333006) - actor/pg_loss:np.float64(0.005690077436156571) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011552226678759325) - actor/ppo_kl:np.float64(-3.631153518313113e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(351749.36200882215) - self_distillation/token_reweight_w_std:np.float64(3531892.006555579) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999812446301803) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09159481288224924) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.683478482067585) - perf/mfu/actor:np.float64(0.032190014645935336) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.899169921875) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:2 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005905130412429571 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005905130412429571 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:332.65234375 - response_length/max:6393.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.65234375 - response_length_non_aborted/max:6393.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.5625 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.329454481601715e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1087023857980967) - timing_s/agent_loop/generate_sequences/max:np.float64(32.07997610978782) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0471223126878613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.07997610978782) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:6393 - timing_s/gen:38.27626817114651 - timing_s/reward:0.04927484318614006 - timing_s/old_log_prob:2.462094532325864 - timing_s/adv:1.0785468369722366 - timing_s/update_actor:18.53664114885032 - timing_s/step:60.43870029784739 - timing_s/stop_profile:0.00011680647730827332 - timing_per_token_ms/gen:0.44946826725474126 - timing_per_token_ms/update_actor:0.11119027028120713 - timing_per_token_ms/adv:0.006469560118841808 - perf/total_num_tokens:166711 - perf/time_per_step:60.43870029784739 - perf/throughput:344.7935659983444 (TaskRunner pid=2122883) Training Progress: 35%|███▌ | 35/100 [30:50<52:10, 48.16s/it] (TaskRunner pid=2122883) step:36 - global_seqlen/min:13174 - global_seqlen/max:28443 - global_seqlen/minmax_diff:15269 - global_seqlen/balanced_min:19485 - global_seqlen/balanced_max:19725 - global_seqlen/mean:19531.25 - actor/entropy:0.1771612912416458 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46555420756340027 - training/rollout_probs_diff_mean:0.004134125541895628 - training/rollout_probs_diff_std:0.012505745515227318 - training/rollout_actor_probs_pearson_corr:0.9981387853622437 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58203125 - self_distillation/correct_sample_fraction:0.33203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58203125 - rollout_corr/rollout_is_mean:0.9998790621757507 - rollout_corr/rollout_is_ratio_fraction_high:1.2940963642904535e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.058674186235294e-05 - rollout_corr/rollout_is_max:2.0104777812957764 - rollout_corr/rollout_is_min:0.375485897064209 - rollout_corr/rollout_is_std:0.037845734506845474 - rollout_corr/rollout_is_eff_sample_size:0.9985693923286193 - rollout_corr/rollout_is_seq_mean:0.9999881386756897 - rollout_corr/rollout_is_seq_std:0.0026660633739084005 - rollout_corr/rollout_is_seq_max:1.010130763053894 - rollout_corr/rollout_is_seq_min:0.9915474057197571 - rollout_corr/rollout_is_seq_max_deviation:0.010130763053894043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2288217013701797) - rollout_corr/training_log_ppl:np.float64(0.20217849771142937) - rollout_corr/kl:np.float64(0.0010742654510309535) - rollout_corr/k3_kl:np.float64(0.0011284284318264781) - rollout_corr/rollout_ppl:np.float64(1.2274580257944763) - rollout_corr/rollout_log_ppl:np.float64(0.2011042319209082) - rollout_corr/log_ppl_diff:np.float64(0.0010742657905211672) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025326748873339966) - rollout_corr/log_ppl_diff_max:np.float64(0.01431196928024292) - rollout_corr/log_ppl_diff_min:np.float64(-0.012727819383144379) - rollout_corr/ppl_ratio:np.float64(1.0010803742334247) - rollout_corr/chi2_token:np.float64(0.0023828367702662945) - rollout_corr/chi2_seq:np.float64(0.3923698428552598) - actor/pg_loss:np.float64(0.0049333591014146805) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015366055267804768) - actor/ppo_kl:np.float64(0.00018557149614117918) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.33203125) - self_distillation/token_reweight_w_mean:np.float64(456639.86721927836) - self_distillation/token_reweight_w_std:np.float64(5056710.059664073) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003420263528824) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09454223301145248) - self_distillation/empty_target_batch:np.float64(0.41796875) - actor/grad_norm:np.float64(0.5455604642629623) - perf/mfu/actor:np.float64(0.03263065275624072) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.63262939453125) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:2 - critic/score/mean:0.33203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006163133657537401 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006163133657537401 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.8515625 - response_length/max:1835.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.8515625 - response_length_non_aborted/max:1835.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:308.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001458302140235901 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2272989992052317) - timing_s/agent_loop/generate_sequences/max:np.float64(10.854171784594655) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8903689130966086) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.854171784594655) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:1835 - timing_s/gen:16.89564960449934 - timing_s/reward:0.05678628943860531 - timing_s/old_log_prob:2.4063503593206406 - timing_s/adv:0.5976713821291924 - timing_s/update_actor:17.047957718372345 - timing_s/step:37.09008174203336 - timing_s/stop_profile:2.9616057872772217e-05 - timing_per_token_ms/gen:0.21864598188911327 - timing_per_token_ms/update_actor:0.10910692939758301 - timing_per_token_ms/adv:0.003825096845626831 - perf/total_num_tokens:156250 - perf/time_per_step:37.09008174203336 - perf/throughput:526.5895647209014 (TaskRunner pid=2122883) Training Progress: 36%|███▌ | 36/100 [31:27<47:50, 44.85s/it] (TaskRunner pid=2122883) step:37 - global_seqlen/min:9999 - global_seqlen/max:26921 - global_seqlen/minmax_diff:16922 - global_seqlen/balanced_min:17705 - global_seqlen/balanced_max:17719 - global_seqlen/mean:17715.375 - actor/entropy:0.18285039067268372 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932346761226654 - training/rollout_probs_diff_mean:0.0044187577441334724 - training/rollout_probs_diff_std:0.012828843668103218 - training/rollout_actor_probs_pearson_corr:0.9980629086494446 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.3828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9999576210975647 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.058861981728114e-05 - rollout_corr/rollout_is_max:1.6349238157272339 - rollout_corr/rollout_is_min:0.33974406123161316 - rollout_corr/rollout_is_std:0.0383153036236763 - rollout_corr/rollout_is_eff_sample_size:0.998533970787023 - rollout_corr/rollout_is_seq_mean:1.000052809715271 - rollout_corr/rollout_is_seq_std:0.0032219039276242256 - rollout_corr/rollout_is_seq_max:1.0131325721740723 - rollout_corr/rollout_is_seq_min:0.9905257225036621 - rollout_corr/rollout_is_seq_max_deviation:0.013132572174072266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.227560852188617) - rollout_corr/training_log_ppl:np.float64(0.20202259431243874) - rollout_corr/kl:np.float64(0.001208621201424176) - rollout_corr/k3_kl:np.float64(0.0011305918565653883) - rollout_corr/rollout_ppl:np.float64(1.226063481066376) - rollout_corr/rollout_log_ppl:np.float64(0.20081397160538472) - rollout_corr/log_ppl_diff:np.float64(0.001208622707054019) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029012424347456545) - rollout_corr/log_ppl_diff_max:np.float64(0.02870221436023712) - rollout_corr/log_ppl_diff_min:np.float64(-0.010198116302490234) - rollout_corr/ppl_ratio:np.float64(1.001217273529619) - rollout_corr/chi2_token:np.float64(0.0019719612319022417) - rollout_corr/chi2_seq:np.float64(0.3310001604259014) - actor/pg_loss:np.float64(0.0050835959846153855) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015021474537206814) - actor/ppo_kl:np.float64(0.0003870755170929385) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3828125) - self_distillation/token_reweight_w_mean:np.float64(1010404.8824171) - self_distillation/token_reweight_w_std:np.float64(9621199.624808073) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0038477664347738) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11127460616989993) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.6599202007055283) - perf/mfu/actor:np.float64(0.030020447907232815) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.63792419433594) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:2 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.024739090353250504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.024739090353250504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:257.88671875 - response_length/max:1172.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:257.88671875 - response_length_non_aborted/max:1172.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.444721758365631e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1212733164429665) - timing_s/agent_loop/generate_sequences/max:np.float64(7.338839437812567) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4418798948754556) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.338839437812567) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1172 - timing_s/gen:12.895120618864894 - timing_s/reward:0.05457581765949726 - timing_s/old_log_prob:2.357027480378747 - timing_s/adv:0.533143537119031 - timing_s/update_actor:16.804465716704726 - timing_s/step:32.729485500603914 - timing_s/stop_profile:0.0001118052750825882 - timing_per_token_ms/gen:0.19532438568995128 - timing_per_token_ms/update_actor:0.11857260795145973 - timing_per_token_ms/adv:0.0037618702477299447 - perf/total_num_tokens:141723 - perf/time_per_step:32.729485500603914 - perf/throughput:541.2665286068466 (TaskRunner pid=2122883) Training Progress: 37%|███▋ | 37/100 [32:00<43:17, 41.22s/it] (TaskRunner pid=2122883) step:38 - global_seqlen/min:20533 - global_seqlen/max:27910 - global_seqlen/minmax_diff:7377 - global_seqlen/balanced_min:22402 - global_seqlen/balanced_max:22407 - global_seqlen/mean:22404.75 - actor/entropy:0.16610024869441986 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27665841579437256 - training/rollout_probs_diff_mean:0.0037956703454256058 - training/rollout_probs_diff_std:0.01207498274743557 - training/rollout_actor_probs_pearson_corr:0.9981511831283569 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999752044677734 - rollout_corr/rollout_is_ratio_fraction_high:2.2189184164744802e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.656755431322381e-05 - rollout_corr/rollout_is_max:2.013848304748535 - rollout_corr/rollout_is_min:0.1109480932354927 - rollout_corr/rollout_is_std:0.03653473034501076 - rollout_corr/rollout_is_eff_sample_size:0.9986667550813965 - rollout_corr/rollout_is_seq_mean:0.9999271631240845 - rollout_corr/rollout_is_seq_std:0.0029462629463523626 - rollout_corr/rollout_is_seq_max:1.010036587715149 - rollout_corr/rollout_is_seq_min:0.9871081113815308 - rollout_corr/rollout_is_seq_max_deviation:0.012891888618469238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2210421110503376) - rollout_corr/training_log_ppl:np.float64(0.19586755773343612) - rollout_corr/kl:np.float64(0.0014365888042209285) - rollout_corr/k3_kl:np.float64(0.0013220797646340543) - rollout_corr/rollout_ppl:np.float64(1.219245539046824) - rollout_corr/rollout_log_ppl:np.float64(0.19443096709437668) - rollout_corr/log_ppl_diff:np.float64(0.0014365906390594319) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028802211018046364) - rollout_corr/log_ppl_diff_max:np.float64(0.02172189950942993) - rollout_corr/log_ppl_diff_min:np.float64(-0.012236267328262329) - rollout_corr/ppl_ratio:np.float64(1.0014454184565693) - rollout_corr/chi2_token:np.float64(0.002300215419381857) - rollout_corr/chi2_seq:np.float64(1.5557226871605963) - actor/pg_loss:np.float64(0.0065355622209608555) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0022695219383876974) - actor/ppo_kl:np.float64(0.0004982942221190001) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(707782.7439720565) - self_distillation/token_reweight_w_std:np.float64(6931265.693862626) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0016436446458101) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11134860079619102) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.7043651267886162) - perf/mfu/actor:np.float64(0.037222466709103126) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.62496185302734) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:2 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008276565931737423 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008276565931737423 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:352.0859375 - response_length/max:1133.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:352.0859375 - response_length_non_aborted/max:1133.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:348.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016898848116397858 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1377350594848394) - timing_s/agent_loop/generate_sequences/max:np.float64(8.493686899542809) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.369795570768474) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.493686899542809) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:1133 - timing_s/gen:14.338888732716441 - timing_s/reward:0.059045134112238884 - timing_s/old_log_prob:2.425033800303936 - timing_s/adv:0.5446427445858717 - timing_s/update_actor:16.951616071164608 - timing_s/step:34.40807002224028 - timing_s/stop_profile:0.00013046711683273315 - timing_per_token_ms/gen:0.1590841273294921 - timing_per_token_ms/update_actor:0.0945760166435946 - timing_per_token_ms/adv:0.003038656672055433 - perf/total_num_tokens:179238 - perf/time_per_step:34.40807002224028 - perf/throughput:651.14811686672 (TaskRunner pid=2122883) Training Progress: 38%|███▊ | 38/100 [32:34<40:30, 39.19s/it] (TaskRunner pid=2122883) step:39 - global_seqlen/min:16482 - global_seqlen/max:26691 - global_seqlen/minmax_diff:10209 - global_seqlen/balanced_min:20524 - global_seqlen/balanced_max:22575 - global_seqlen/mean:20860.375 - actor/entropy:0.15684472024440765 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3085607886314392 - training/rollout_probs_diff_mean:0.003755766898393631 - training/rollout_probs_diff_std:0.012273234315216541 - training/rollout_actor_probs_pearson_corr:0.9981006979942322 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8203125 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8203125 - rollout_corr/rollout_is_mean:0.9999597072601318 - rollout_corr/rollout_is_ratio_fraction_high:3.471458694548346e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.6286117139970884e-05 - rollout_corr/rollout_is_max:2.4837749004364014 - rollout_corr/rollout_is_min:0.06344368308782578 - rollout_corr/rollout_is_std:0.036595042794942856 - rollout_corr/rollout_is_eff_sample_size:0.998662356110423 - rollout_corr/rollout_is_seq_mean:0.9999072551727295 - rollout_corr/rollout_is_seq_std:0.0031039086170494556 - rollout_corr/rollout_is_seq_max:1.0133365392684937 - rollout_corr/rollout_is_seq_min:0.9888757467269897 - rollout_corr/rollout_is_seq_max_deviation:0.013336539268493652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.226109724957496) - rollout_corr/training_log_ppl:np.float64(0.20032589661423117) - rollout_corr/kl:np.float64(0.001165302209578556) - rollout_corr/k3_kl:np.float64(0.0011575412143542962) - rollout_corr/rollout_ppl:np.float64(1.2246328527107835) - rollout_corr/rollout_log_ppl:np.float64(0.19916059231036343) - rollout_corr/log_ppl_diff:np.float64(0.0011653043038677424) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028654705092776567) - rollout_corr/log_ppl_diff_max:np.float64(0.016280263662338257) - rollout_corr/log_ppl_diff_min:np.float64(-0.011811941862106323) - rollout_corr/ppl_ratio:np.float64(1.0011730468831956) - rollout_corr/chi2_token:np.float64(0.0023117719683796167) - rollout_corr/chi2_seq:np.float64(0.42900348524563015) - actor/pg_loss:np.float64(0.008025142014957964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002576867142693118) - actor/ppo_kl:np.float64(0.0001556854837474475) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(1097881.6426437146) - self_distillation/token_reweight_w_std:np.float64(11470257.020113224) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0032665592152625) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1485372949828161) - self_distillation/empty_target_batch:np.float64(0.1796875) - actor/grad_norm:np.float64(0.9034479111433029) - perf/mfu/actor:np.float64(0.03487259499439749) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.6538429260254) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:2 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.021414561197161674 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.021414561197161674 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:337.57421875 - response_length/max:3975.0 - response_length/min:84.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:337.57421875 - response_length_non_aborted/max:3975.0 - response_length_non_aborted/min:84.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.3125 - prompt_length/max:498.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015822425484657288 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0175010822713375) - timing_s/agent_loop/generate_sequences/max:np.float64(21.103810861706734) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.054622892734187) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.103810861706734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:3975 - timing_s/gen:27.206532564014196 - timing_s/reward:0.05892925336956978 - timing_s/old_log_prob:2.558308009058237 - timing_s/adv:0.6190387345850468 - timing_s/update_actor:17.218717677518725 - timing_s/step:47.74760019779205 - timing_s/stop_profile:0.00010721571743488312 - timing_per_token_ms/gen:0.31482119168254896 - timing_per_token_ms/update_actor:0.10317838052718806 - timing_per_token_ms/adv:0.0037094175834869146 - perf/total_num_tokens:166883 - perf/time_per_step:47.74760019779205 - perf/throughput:436.88844912805956 (TaskRunner pid=2122883) Training Progress: 39%|███▉ | 39/100 [33:22<42:28, 41.78s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques that assess the stability of the protein's folded structure. Without specific computational tools or experimental data provided, it is not possible to calculate the exact folding stability score for the given sequence. However, based on the options provided, the most reasonable value for a folding stability score would be a positive number, as stability scores are often reported as positive values. Among the options, 1.69 is the most plausible value for a folding stability score. Therefore, the correct answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2122883) step:40 - global_seqlen/min:13634 - global_seqlen/max:24094 - global_seqlen/minmax_diff:10460 - global_seqlen/balanced_min:18012 - global_seqlen/balanced_max:18015 - global_seqlen/mean:18013.375 - actor/entropy:0.17955990135669708 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3510846793651581 - training/rollout_probs_diff_mean:0.004470017272979021 - training/rollout_probs_diff_std:0.013084249570965767 - training/rollout_actor_probs_pearson_corr:0.9979696869850159 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999790608882904 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012665642134379596 - rollout_corr/rollout_is_max:1.8637934923171997 - rollout_corr/rollout_is_min:0.17486874759197235 - rollout_corr/rollout_is_std:0.03934907540678978 - rollout_corr/rollout_is_eff_sample_size:0.9984533901986787 - rollout_corr/rollout_is_seq_mean:0.9998258948326111 - rollout_corr/rollout_is_seq_std:0.0032514510676264763 - rollout_corr/rollout_is_seq_max:1.010748267173767 - rollout_corr/rollout_is_seq_min:0.9913337826728821 - rollout_corr/rollout_is_seq_max_deviation:0.01074826717376709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2218255074694753) - rollout_corr/training_log_ppl:np.float64(0.197417336050421) - rollout_corr/kl:np.float64(0.0015267702667483007) - rollout_corr/k3_kl:np.float64(0.001369252413724098) - rollout_corr/rollout_ppl:np.float64(1.219967634882778) - rollout_corr/rollout_log_ppl:np.float64(0.195890564340516) - rollout_corr/log_ppl_diff:np.float64(0.001526771709905006) - rollout_corr/log_ppl_abs_diff:np.float64(0.003275623734225519) - rollout_corr/log_ppl_diff_max:np.float64(0.022372066974639893) - rollout_corr/log_ppl_diff_min:np.float64(-0.009259283542633057) - rollout_corr/ppl_ratio:np.float64(1.0015366142615676) - rollout_corr/chi2_token:np.float64(0.0024506186600774527) - rollout_corr/chi2_seq:np.float64(0.4685141814406961) - actor/pg_loss:np.float64(0.0045470668701455) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0022142869106573926) - actor/ppo_kl:np.float64(0.00046563480048433803) - actor/pg_clipfrac_lower:np.float64(1.6075102394097485e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(405614.0228516003) - self_distillation/token_reweight_w_std:np.float64(4572729.550735184) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009943647310138) - self_distillation/token_reweight_w_clip_frac:np.float64(0.087815267717815) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.6629709005355835) - perf/mfu/actor:np.float64(0.030678227294980077) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.72424697875977) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.37125) - val-aux/sciknoweval/reward/std@16:np.float64(0.2511125750137152) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.48288000000000003) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21298088413286884) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.26238) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21440655701062628) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.37195999999999996) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.25064049383261017) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.57182) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1477984422603609) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.17532) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14787677257029436) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.37837999999999994) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.22110761151134656) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.62862) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07917449340268014) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.12079999999999998) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0794394529731964) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.38301999999999997) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.17816152607171362) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.65348) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.029104144112121873) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.09342) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03397205873230382) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.38182000000000005) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13291962364918583) - val-aux/sciknoweval/score/mean@16:np.float64(0.37125) - val-aux/sciknoweval/score/std@16:np.float64(0.2511125750137152) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.48288000000000003) - val-aux/sciknoweval/score/best@2/std:np.float64(0.21298088413286884) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.26238) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21440655701062628) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.37195999999999996) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.25064049383261017) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.57182) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1477984422603609) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.17532) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14787677257029436) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.37837999999999994) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.22110761151134656) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.62862) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07917449340268014) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.12079999999999998) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0794394529731964) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.38301999999999997) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.17816152607171362) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.65348) - val-aux/sciknoweval/score/best@16/std:np.float64(0.029104144112121873) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.09342) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03397205873230382) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.38182000000000005) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13291962364918583) - val-core/sciknoweval/acc/mean@16:np.float64(0.37125) - val-aux/sciknoweval/acc/std@16:np.float64(0.2511125750137152) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.48288000000000003) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.21298088413286884) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.26238) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21440655701062628) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.37195999999999996) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.25064049383261017) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.57182) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1477984422603609) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.17532) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14787677257029436) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.37837999999999994) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.22110761151134656) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.62862) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07917449340268014) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.12079999999999998) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0794394529731964) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.38301999999999997) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.17816152607171362) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.65348) - val-core/sciknoweval/acc/best@16/std:np.float64(0.029104144112121873) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.09342) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03397205873230382) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.38182000000000005) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13291962364918583) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.014523687548277814) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99982) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.003070210307423338) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9933799999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.018787298197088173) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99684) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.013382865284667094) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.98726) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.02452256547346986) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99914) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.007094879776413394) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9765) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.029286838934616793) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9997800000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.003531871795362199) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9619200000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.028882305431774648) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711644) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:2 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008571078069508076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008571078069508076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:246.73046875 - response_length/max:798.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:246.73046875 - response_length_non_aborted/max:798.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:316.1875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.31955873966217e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1871335823088884) - timing_s/agent_loop/generate_sequences/max:np.float64(5.571412174031138) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4915980326622957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.571412174031138) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:764 - timing_s/gen:11.46059394814074 - timing_s/reward:0.052289627492427826 - timing_s/old_log_prob:2.3004838787019253 - timing_s/adv:0.5110577456653118 - timing_s/update_actor:16.656289303675294 - timing_s/step:31.060887914150953 - timing_s/testing:68.4701924789697 - timing_s/save_checkpoint:6.864336807280779 - timing_s/stop_profile:0.00012974068522453308 - timing_per_token_ms/gen:0.18144473739595554 - timing_per_token_ms/update_actor:0.11558279128477655 - timing_per_token_ms/adv:0.0035463769675679307 - perf/total_num_tokens:144107 - perf/time_per_step:31.060887914150953 - perf/throughput:579.9375423454437 (TaskRunner pid=2122883) Training Progress: 40%|████ | 40/100 [35:09<1:01:10, 61.18s/it] (TaskRunner pid=2122883) step:41 - global_seqlen/min:11576 - global_seqlen/max:23533 - global_seqlen/minmax_diff:11957 - global_seqlen/balanced_min:20248 - global_seqlen/balanced_max:20537 - global_seqlen/mean:20291.25 - actor/entropy:0.18663230538368225 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6220853924751282 - training/rollout_probs_diff_mean:0.004448506515473127 - training/rollout_probs_diff_std:0.01353462878614664 - training/rollout_actor_probs_pearson_corr:0.99795001745224 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0001215934753418 - rollout_corr/rollout_is_ratio_fraction_high:3.953923442168161e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011861771054100245 - rollout_corr/rollout_is_max:4.517718315124512 - rollout_corr/rollout_is_min:0.06017683818936348 - rollout_corr/rollout_is_std:0.03926188126206398 - rollout_corr/rollout_is_eff_sample_size:0.9984612337553515 - rollout_corr/rollout_is_seq_mean:1.0000065565109253 - rollout_corr/rollout_is_seq_std:0.003107651136815548 - rollout_corr/rollout_is_seq_max:1.0133030414581299 - rollout_corr/rollout_is_seq_min:0.9923636317253113 - rollout_corr/rollout_is_seq_max_deviation:0.013303041458129883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2377123944461346) - rollout_corr/training_log_ppl:np.float64(0.20936558903486002) - rollout_corr/kl:np.float64(0.001277374319045066) - rollout_corr/k3_kl:np.float64(0.0011814869795614413) - rollout_corr/rollout_ppl:np.float64(1.2360735745169222) - rollout_corr/rollout_log_ppl:np.float64(0.20808821474201977) - rollout_corr/log_ppl_diff:np.float64(0.0012773742928402498) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028327976033324376) - rollout_corr/log_ppl_diff_max:np.float64(0.01583242416381836) - rollout_corr/log_ppl_diff_min:np.float64(-0.007822096347808838) - rollout_corr/ppl_ratio:np.float64(1.0012840463314205) - rollout_corr/chi2_token:np.float64(0.0022241747938096523) - rollout_corr/chi2_seq:np.float64(0.5387362614274025) - actor/pg_loss:np.float64(0.005529339890927076) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001543819114431244) - actor/ppo_kl:np.float64(0.0003531992533591932) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(397424.65681143315) - self_distillation/token_reweight_w_std:np.float64(4439488.2469981825) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017797024920583) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11050213062844705) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.6619970500469208) - perf/mfu/actor:np.float64(0.03406975692051313) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.76568222045898) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:2 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0033542450983077288 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0033542450983077288 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:296.3828125 - response_length/max:1648.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.3828125 - response_length_non_aborted/max:1648.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:337.71875 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.941519677639008e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1566278170794249) - timing_s/agent_loop/generate_sequences/max:np.float64(9.40533797070384) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9047601231723092) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.40533797070384) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:1648 - timing_s/gen:15.249274028465152 - timing_s/reward:0.04902481287717819 - timing_s/old_log_prob:2.395838249474764 - timing_s/adv:0.6379478648304939 - timing_s/update_actor:16.939913291484118 - timing_s/step:35.36231600493193 - timing_s/stop_profile:3.432855010032654e-05 - timing_per_token_ms/gen:0.2009815487316492 - timing_per_token_ms/update_actor:0.10435479142169726 - timing_per_token_ms/adv:0.003929944340728725 - perf/total_num_tokens:162330 - perf/time_per_step:35.36231600493193 - perf/throughput:573.809984537495 (TaskRunner pid=2122883) Training Progress: 41%|████ | 41/100 [35:44<52:33, 53.44s/it] (TaskRunner pid=2122883) step:42 - global_seqlen/min:10264 - global_seqlen/max:20936 - global_seqlen/minmax_diff:10672 - global_seqlen/balanced_min:15308 - global_seqlen/balanced_max:15324 - global_seqlen/mean:15318.5 - actor/entropy:0.17958307266235352 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3282223343849182 - training/rollout_probs_diff_mean:0.004483028315007687 - training/rollout_probs_diff_std:0.013422226533293724 - training/rollout_actor_probs_pearson_corr:0.997865617275238 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.6171875 - self_distillation/correct_sample_fraction:0.3125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6171875 - rollout_corr/rollout_is_mean:0.9998947978019714 - rollout_corr/rollout_is_ratio_fraction_high:1.716207952995319e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010297247354174033 - rollout_corr/rollout_is_max:2.239044189453125 - rollout_corr/rollout_is_min:0.3549353778362274 - rollout_corr/rollout_is_std:0.039695192128419876 - rollout_corr/rollout_is_eff_sample_size:0.9984264140581771 - rollout_corr/rollout_is_seq_mean:0.9999154806137085 - rollout_corr/rollout_is_seq_std:0.0030749160796403885 - rollout_corr/rollout_is_seq_max:1.0121992826461792 - rollout_corr/rollout_is_seq_min:0.9852494597434998 - rollout_corr/rollout_is_seq_max_deviation:0.014750540256500244 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2074014260433614) - rollout_corr/training_log_ppl:np.float64(0.18636732350569218) - rollout_corr/kl:np.float64(0.0009534069710754878) - rollout_corr/k3_kl:np.float64(0.001279137213884951) - rollout_corr/rollout_ppl:np.float64(1.2062194678001106) - rollout_corr/rollout_log_ppl:np.float64(0.1854139148199465) - rollout_corr/log_ppl_diff:np.float64(0.0009534086857456714) - rollout_corr/log_ppl_abs_diff:np.float64(0.002904396125813946) - rollout_corr/log_ppl_diff_max:np.float64(0.01598384976387024) - rollout_corr/log_ppl_diff_min:np.float64(-0.010572642087936401) - rollout_corr/ppl_ratio:np.float64(1.0009608941618353) - rollout_corr/chi2_token:np.float64(0.0036230282858014107) - rollout_corr/chi2_seq:np.float64(0.45362843945622444) - actor/pg_loss:np.float64(0.0053381690522655845) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014900016758474521) - actor/ppo_kl:np.float64(-2.4106089924202934e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3125) - self_distillation/token_reweight_w_mean:np.float64(811433.6940884823) - self_distillation/token_reweight_w_std:np.float64(8261526.878942402) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001308246748522) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10496872605290264) - self_distillation/empty_target_batch:np.float64(0.3828125) - actor/grad_norm:np.float64(0.6776371896266937) - perf/mfu/actor:np.float64(0.026046903222768404) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.86044692993164) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:2 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002994782757014036 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002994782757014036 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:227.609375 - response_length/max:957.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.609375 - response_length_non_aborted/max:957.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.09375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.9033049941062927e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.001108106225729) - timing_s/agent_loop/generate_sequences/max:np.float64(6.198202773928642) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2058102839364437) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.198202773928642) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:957 - timing_s/gen:12.159217644482851 - timing_s/reward:0.05021318793296814 - timing_s/old_log_prob:2.2670325599610806 - timing_s/adv:0.49279571883380413 - timing_s/update_actor:16.662710631266236 - timing_s/step:31.7205575350672 - timing_s/stop_profile:0.00012237019836902618 - timing_per_token_ms/gen:0.20867744979204453 - timing_per_token_ms/update_actor:0.1359688500119646 - timing_per_token_ms/adv:0.004021246522454908 - perf/total_num_tokens:122548 - perf/time_per_step:31.7205575350672 - perf/throughput:482.92026339906977 (TaskRunner pid=2122883) Training Progress: 42%|████▏ | 42/100 [36:16<45:22, 46.94s/it] (TaskRunner pid=2122883) step:43 - global_seqlen/min:12638 - global_seqlen/max:28379 - global_seqlen/minmax_diff:15741 - global_seqlen/balanced_min:19084 - global_seqlen/balanced_max:19115 - global_seqlen/mean:19106.75 - actor/entropy:0.17867739498615265 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4741092324256897 - training/rollout_probs_diff_mean:0.004286498762667179 - training/rollout_probs_diff_std:0.01305386796593666 - training/rollout_actor_probs_pearson_corr:0.9979863166809082 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:1.0000731945037842 - rollout_corr/rollout_is_ratio_fraction_high:2.8147605917183682e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012666422117035836 - rollout_corr/rollout_is_max:4.272411823272705 - rollout_corr/rollout_is_min:0.2297939509153366 - rollout_corr/rollout_is_std:0.03817035257816315 - rollout_corr/rollout_is_eff_sample_size:0.99854502489099 - rollout_corr/rollout_is_seq_mean:1.0000548362731934 - rollout_corr/rollout_is_seq_std:0.0034973809961229563 - rollout_corr/rollout_is_seq_max:1.0184221267700195 - rollout_corr/rollout_is_seq_min:0.98951256275177 - rollout_corr/rollout_is_seq_max_deviation:0.01842212677001953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2184172831475735) - rollout_corr/training_log_ppl:np.float64(0.1945186269149417) - rollout_corr/kl:np.float64(0.0017048975041937808) - rollout_corr/k3_kl:np.float64(0.002482297865981309) - rollout_corr/rollout_ppl:np.float64(1.2163156094029546) - rollout_corr/rollout_log_ppl:np.float64(0.19281372765544802) - rollout_corr/log_ppl_diff:np.float64(0.0017048992594936863) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035877060872735456) - rollout_corr/log_ppl_diff_max:np.float64(0.02438490092754364) - rollout_corr/log_ppl_diff_min:np.float64(-0.022404611110687256) - rollout_corr/ppl_ratio:np.float64(1.0017193660605699) - rollout_corr/chi2_token:np.float64(0.0598582336679101) - rollout_corr/chi2_seq:np.float64(42.330826848512515) - actor/pg_loss:np.float64(0.0056234917137771845) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002230035815955489) - actor/ppo_kl:np.float64(0.0008539994699017228) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(725459.6291632976) - self_distillation/token_reweight_w_std:np.float64(7201750.69503868) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014563573058695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11069456930272281) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.7029270678758621) - perf/mfu/actor:np.float64(0.03253319803382335) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.79828262329102) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:3 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0038878880441188812 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0038878880441188812 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:277.5546875 - response_length/max:1388.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:277.5546875 - response_length_non_aborted/max:1388.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:319.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00031706131994724274 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8820750936865807) - timing_s/agent_loop/generate_sequences/max:np.float64(8.526029285043478) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7013816293983837) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.526029285043478) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:1388 - timing_s/gen:14.14283044822514 - timing_s/reward:0.05622094310820103 - timing_s/old_log_prob:2.6203188207000494 - timing_s/adv:0.5772702526301146 - timing_s/update_actor:16.599831921979785 - timing_s/step:34.095423594117165 - timing_s/stop_profile:0.00011685304343700409 - timing_per_token_ms/gen:0.19904340991675543 - timing_per_token_ms/update_actor:0.1085992641473549 - timing_per_token_ms/adv:0.003776612012967371 - perf/total_num_tokens:152854 - perf/time_per_step:34.095423594117165 - perf/throughput:560.3904567209039 (TaskRunner pid=2122883) Training Progress: 43%|████▎ | 43/100 [36:50<41:04, 43.23s/it] (TaskRunner pid=2122883) step:44 - global_seqlen/min:13160 - global_seqlen/max:23356 - global_seqlen/minmax_diff:10196 - global_seqlen/balanced_min:19260 - global_seqlen/balanced_max:19276 - global_seqlen/mean:19267.375 - actor/entropy:0.1776508092880249 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44105371832847595 - training/rollout_probs_diff_mean:0.004229406826198101 - training/rollout_probs_diff_std:0.013126127421855927 - training/rollout_actor_probs_pearson_corr:0.9979387521743774 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:0.9998519420623779 - rollout_corr/rollout_is_ratio_fraction_high:4.204801734886132e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015417607210110873 - rollout_corr/rollout_is_max:2.684969186782837 - rollout_corr/rollout_is_min:0.17924728989601135 - rollout_corr/rollout_is_std:0.038983073085546494 - rollout_corr/rollout_is_eff_sample_size:0.9984822693566601 - rollout_corr/rollout_is_seq_mean:0.9997416734695435 - rollout_corr/rollout_is_seq_std:0.003170690732076764 - rollout_corr/rollout_is_seq_max:1.01134192943573 - rollout_corr/rollout_is_seq_min:0.9876351952552795 - rollout_corr/rollout_is_seq_max_deviation:0.012364804744720459 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2239833851344883) - rollout_corr/training_log_ppl:np.float64(0.19885029891156591) - rollout_corr/kl:np.float64(0.0017507878198053106) - rollout_corr/k3_kl:np.float64(0.0012392329912813693) - rollout_corr/rollout_ppl:np.float64(1.2217664504423738) - rollout_corr/rollout_log_ppl:np.float64(0.19709950988180935) - rollout_corr/log_ppl_diff:np.float64(0.001750789029756561) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031125989626161754) - rollout_corr/log_ppl_diff_max:np.float64(0.02598017454147339) - rollout_corr/log_ppl_diff_min:np.float64(-0.010924175381660461) - rollout_corr/ppl_ratio:np.float64(1.0017614769749343) - rollout_corr/chi2_token:np.float64(0.0012613397557288408) - rollout_corr/chi2_seq:np.float64(0.3330452733207494) - actor/pg_loss:np.float64(0.00551921222358942) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012254381422280858) - actor/ppo_kl:np.float64(0.0004983369169835328) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(371231.27284245985) - self_distillation/token_reweight_w_std:np.float64(4606640.552745781) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003671881277114) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09728244636789896) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.6762457638978958) - perf/mfu/actor:np.float64(0.032491280735104155) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8349723815918) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:3 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001862376811914146 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001862376811914146 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:278.69921875 - response_length/max:1369.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:278.69921875 - response_length_non_aborted/max:1369.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:323.40625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016621500253677368 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2336528431624174) - timing_s/agent_loop/generate_sequences/max:np.float64(8.673735836520791) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.68211255150527) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.673735836520791) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1369 - timing_s/gen:14.567327432334423 - timing_s/reward:0.05603113956749439 - timing_s/old_log_prob:2.3046666514128447 - timing_s/adv:0.5825918577611446 - timing_s/update_actor:16.785698210820556 - timing_s/step:34.38467370159924 - timing_s/stop_profile:5.710311233997345e-05 - timing_per_token_ms/gen:0.20417575276233652 - timing_per_token_ms/update_actor:0.10889974770058555 - timing_per_token_ms/adv:0.003779652506900555 - perf/total_num_tokens:154139 - perf/time_per_step:34.38467370159924 - perf/throughput:560.3477632857068 (TaskRunner pid=2122883) Training Progress: 44%|████▍ | 44/100 [37:25<37:52, 40.59s/it] (TaskRunner pid=2122883) step:45 - global_seqlen/min:10370 - global_seqlen/max:27688 - global_seqlen/minmax_diff:17318 - global_seqlen/balanced_min:19743 - global_seqlen/balanced_max:20142 - global_seqlen/mean:19803.0 - actor/entropy:0.18040679395198822 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5990896821022034 - training/rollout_probs_diff_mean:0.004071939270943403 - training/rollout_probs_diff_std:0.013194762170314789 - training/rollout_actor_probs_pearson_corr:0.998019278049469 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9999013543128967 - rollout_corr/rollout_is_ratio_fraction_high:3.890041625709273e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.780083251418546e-05 - rollout_corr/rollout_is_max:2.47705340385437 - rollout_corr/rollout_is_min:0.16637524962425232 - rollout_corr/rollout_is_std:0.037288833409547806 - rollout_corr/rollout_is_eff_sample_size:0.9986111168979587 - rollout_corr/rollout_is_seq_mean:0.9998700618743896 - rollout_corr/rollout_is_seq_std:0.003353060455992818 - rollout_corr/rollout_is_seq_max:1.012258768081665 - rollout_corr/rollout_is_seq_min:0.991085410118103 - rollout_corr/rollout_is_seq_max_deviation:0.012258768081665039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2377176005393267) - rollout_corr/training_log_ppl:np.float64(0.20978421579638962) - rollout_corr/kl:np.float64(0.0012163608913304813) - rollout_corr/k3_kl:np.float64(0.0014924378000671368) - rollout_corr/rollout_ppl:np.float64(1.2362172487191856) - rollout_corr/rollout_log_ppl:np.float64(0.2085678537550848) - rollout_corr/log_ppl_diff:np.float64(0.0012163620413048193) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032108214363688603) - rollout_corr/log_ppl_diff_max:np.float64(0.03155073523521423) - rollout_corr/log_ppl_diff_min:np.float64(-0.008211210370063782) - rollout_corr/ppl_ratio:np.float64(1.001229275483638) - rollout_corr/chi2_token:np.float64(0.003470740979537368) - rollout_corr/chi2_seq:np.float64(1.005021903431043) - actor/pg_loss:np.float64(0.005921048694290221) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013443831210224744) - actor/ppo_kl:np.float64(0.00014919952121861257) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(383251.54019788885) - self_distillation/token_reweight_w_std:np.float64(4420001.027808327) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017411878798157) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10430864432419185) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.6809583902359009) - perf/mfu/actor:np.float64(0.03341682138075278) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.90021133422852) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:3 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01745494082570076 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01745494082570076 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.25 - response_length/max:1940.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.25 - response_length_non_aborted/max:1940.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.59375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.563612699508667e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.061364471912384) - timing_s/agent_loop/generate_sequences/max:np.float64(11.541284376755357) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.907274785611662) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.541284376755357) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1940 - timing_s/gen:18.06968740746379 - timing_s/reward:0.05686299502849579 - timing_s/old_log_prob:2.346899025142193 - timing_s/adv:0.6176035478711128 - timing_s/update_actor:16.911936789751053 - timing_s/step:38.08804814517498 - timing_s/stop_profile:0.00011461786925792694 - timing_per_token_ms/gen:0.23430611264864873 - timing_per_token_ms/update_actor:0.10675110330348339 - timing_per_token_ms/adv:0.0038984216272225976 - perf/total_num_tokens:158424 - perf/time_per_step:38.08804814517498 - perf/throughput:519.926879017786 (TaskRunner pid=2122883) Training Progress: 45%|████▌ | 45/100 [38:03<36:31, 39.85s/it] (TaskRunner pid=2122883) step:46 - global_seqlen/min:11122 - global_seqlen/max:29892 - global_seqlen/minmax_diff:18770 - global_seqlen/balanced_min:16452 - global_seqlen/balanced_max:16787 - global_seqlen/mean:16539.25 - actor/entropy:0.18795309960842133 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6190823316574097 - training/rollout_probs_diff_mean:0.004448164254426956 - training/rollout_probs_diff_std:0.013683785684406757 - training/rollout_actor_probs_pearson_corr:0.9979138970375061 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.999754011631012 - rollout_corr/rollout_is_ratio_fraction_high:7.895152521086857e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001263224403373897 - rollout_corr/rollout_is_max:2.5501413345336914 - rollout_corr/rollout_is_min:0.03999684005975723 - rollout_corr/rollout_is_std:0.039844319224357605 - rollout_corr/rollout_is_eff_sample_size:0.9984140554322439 - rollout_corr/rollout_is_seq_mean:0.9995894432067871 - rollout_corr/rollout_is_seq_std:0.0033229426480829716 - rollout_corr/rollout_is_seq_max:1.0124613046646118 - rollout_corr/rollout_is_seq_min:0.990471601486206 - rollout_corr/rollout_is_seq_max_deviation:0.012461304664611816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2413132754154503) - rollout_corr/training_log_ppl:np.float64(0.21247280396346468) - rollout_corr/kl:np.float64(0.002923060385167986) - rollout_corr/k3_kl:np.float64(0.0021046519883611836) - rollout_corr/rollout_ppl:np.float64(1.2376095172949135) - rollout_corr/rollout_log_ppl:np.float64(0.20954974222695455) - rollout_corr/log_ppl_diff:np.float64(0.0029230617365101352) - rollout_corr/log_ppl_abs_diff:np.float64(0.004334593817475252) - rollout_corr/log_ppl_diff_max:np.float64(0.025646045804023743) - rollout_corr/log_ppl_diff_min:np.float64(-0.013347774744033813) - rollout_corr/ppl_ratio:np.float64(1.0029449705034494) - rollout_corr/chi2_token:np.float64(0.0022582970559597015) - rollout_corr/chi2_seq:np.float64(0.4936607242561877) - actor/pg_loss:np.float64(0.0058343312703073025) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0026905024278676137) - actor/ppo_kl:np.float64(0.001411904266461761) - actor/pg_clipfrac_lower:np.float64(1.5625000742147677e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(1044202.6667105879) - self_distillation/token_reweight_w_std:np.float64(8739516.905448329) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014959073159844) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10200972099846695) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.7633071690797806) - perf/mfu/actor:np.float64(0.027947938564247113) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.91358947753906) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:3 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.026450734585523605 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.026450734585523605 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:247.3828125 - response_length/max:1912.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:247.3828125 - response_length_non_aborted/max:1912.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.46875 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012016668915748596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0365642439574003) - timing_s/agent_loop/generate_sequences/max:np.float64(10.925699835643172) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3518986795606907) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.925699835643172) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1912 - timing_s/gen:16.643203927204013 - timing_s/reward:0.05389646999537945 - timing_s/old_log_prob:2.2957371938973665 - timing_s/adv:0.5990908946841955 - timing_s/update_actor:16.899991875514388 - timing_s/step:36.581113524734974 - timing_s/stop_profile:0.0001400299370288849 - timing_per_token_ms/gen:0.2628012620749094 - timing_per_token_ms/update_actor:0.12772640745132327 - timing_per_token_ms/adv:0.004527796716025481 - perf/total_num_tokens:132314 - perf/time_per_step:36.581113524734974 - perf/throughput:452.12538401316544 (TaskRunner pid=2122883) Training Progress: 46%|████▌ | 46/100 [38:40<34:59, 38.88s/it] (TaskRunner pid=2122883) step:47 - global_seqlen/min:19683 - global_seqlen/max:34953 - global_seqlen/minmax_diff:15270 - global_seqlen/balanced_min:24795 - global_seqlen/balanced_max:24818 - global_seqlen/mean:24812.5 - actor/entropy:0.14859639108181 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5783280730247498 - training/rollout_probs_diff_mean:0.003343893215060234 - training/rollout_probs_diff_std:0.012050571851432323 - training/rollout_actor_probs_pearson_corr:0.9980299472808838 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.000154733657837 - rollout_corr/rollout_is_ratio_fraction_high:4.733234891318716e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013253057841211557 - rollout_corr/rollout_is_max:6.984804630279541 - rollout_corr/rollout_is_min:0.15933217108249664 - rollout_corr/rollout_is_std:0.0349300317466259 - rollout_corr/rollout_is_eff_sample_size:0.9987817365255202 - rollout_corr/rollout_is_seq_mean:1.0001826286315918 - rollout_corr/rollout_is_seq_std:0.003950812388211489 - rollout_corr/rollout_is_seq_max:1.04475998878479 - rollout_corr/rollout_is_seq_min:0.9881705641746521 - rollout_corr/rollout_is_seq_max_deviation:0.04475998878479004 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2160145845264196) - rollout_corr/training_log_ppl:np.float64(0.19086833937035408) - rollout_corr/kl:np.float64(0.015507104080729306) - rollout_corr/k3_kl:np.float64(0.014919463526666732) - rollout_corr/rollout_ppl:np.float64(1.196146132890135) - rollout_corr/rollout_log_ppl:np.float64(0.1753612344764406) - rollout_corr/log_ppl_diff:np.float64(0.015507104864809662) - rollout_corr/log_ppl_abs_diff:np.float64(0.016464031563373283) - rollout_corr/log_ppl_diff_max:np.float64(0.2085978388786316) - rollout_corr/log_ppl_diff_min:np.float64(-0.013799071311950684) - rollout_corr/ppl_ratio:np.float64(1.0166863345075399) - rollout_corr/chi2_token:np.float64(0.002912049414590001) - rollout_corr/chi2_seq:np.float64(0.4900429293047637) - actor/pg_loss:np.float64(0.006711778463795781) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001963024042424877) - actor/ppo_kl:np.float64(0.013789263467803181) - actor/pg_clipfrac_lower:np.float64(1.4103383364272304e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.484375) - self_distillation/token_reweight_w_mean:np.float64(396123.6195182537) - self_distillation/token_reweight_w_std:np.float64(4634816.637059211) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006289314478636) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1108224172348855) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6453308165073395) - perf/mfu/actor:np.float64(0.04141419655296415) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.81275177001953) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:3 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006173321511596441 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006173321511596441 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:412.640625 - response_length/max:1648.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:412.640625 - response_length_non_aborted/max:1648.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.221956133842468e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0964402798563242) - timing_s/agent_loop/generate_sequences/max:np.float64(11.21230830438435) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.87971582760656) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.21230830438435) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:416 - timing_s/agent_loop/slowest/response_length:1648 - timing_s/gen:16.986663652583957 - timing_s/reward:0.06316123157739639 - timing_s/old_log_prob:2.3301994521170855 - timing_s/adv:0.6273217760026455 - timing_s/update_actor:17.14669720456004 - timing_s/step:37.24170481786132 - timing_s/stop_profile:0.00011117756366729736 - timing_per_token_ms/gen:0.1608037378600473 - timing_per_token_ms/update_actor:0.08638134611869039 - timing_per_token_ms/adv:0.0031603112141191207 - perf/total_num_tokens:198500 - perf/time_per_step:37.24170481786132 - perf/throughput:666.2557506792706 (TaskRunner pid=2122883) Training Progress: 47%|████▋ | 47/100 [39:17<33:55, 38.41s/it] (TaskRunner pid=2122883) step:48 - global_seqlen/min:15753 - global_seqlen/max:27003 - global_seqlen/minmax_diff:11250 - global_seqlen/balanced_min:20074 - global_seqlen/balanced_max:20085 - global_seqlen/mean:20081.125 - actor/entropy:0.18119651079177856 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5287033319473267 - training/rollout_probs_diff_mean:0.0041629173792898655 - training/rollout_probs_diff_std:0.013067073188722134 - training/rollout_actor_probs_pearson_corr:0.9980469942092896 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998913407325745 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0001484836102463305 - rollout_corr/rollout_is_max:1.8580189943313599 - rollout_corr/rollout_is_min:0.12150196731090546 - rollout_corr/rollout_is_std:0.03766418248414993 - rollout_corr/rollout_is_eff_sample_size:0.9985830623875143 - rollout_corr/rollout_is_seq_mean:0.999824047088623 - rollout_corr/rollout_is_seq_std:0.0027110264636576176 - rollout_corr/rollout_is_seq_max:1.0075005292892456 - rollout_corr/rollout_is_seq_min:0.9895845055580139 - rollout_corr/rollout_is_seq_max_deviation:0.010415494441986084 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2318859449587762) - rollout_corr/training_log_ppl:np.float64(0.20496501913294196) - rollout_corr/kl:np.float64(0.0021551217895456887) - rollout_corr/k3_kl:np.float64(0.0023452171697613267) - rollout_corr/rollout_ppl:np.float64(1.2291390299797058) - rollout_corr/rollout_log_ppl:np.float64(0.20280989579623565) - rollout_corr/log_ppl_diff:np.float64(0.0021551233367063105) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036582308530341834) - rollout_corr/log_ppl_diff_max:np.float64(0.03887230157852173) - rollout_corr/log_ppl_diff_min:np.float64(-0.013254612684249878) - rollout_corr/ppl_ratio:np.float64(1.0021720596123487) - rollout_corr/chi2_token:np.float64(0.0768868934828788) - rollout_corr/chi2_seq:np.float64(9.817465823842213) - actor/pg_loss:np.float64(0.007189736585132778) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001849453532258849) - actor/ppo_kl:np.float64(0.001085506058089436) - actor/pg_clipfrac_lower:np.float64(3.0127496302156942e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(593021.7767671708) - self_distillation/token_reweight_w_std:np.float64(6946430.695254925) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002022563945502) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10840969366836362) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.7073291689157486) - perf/mfu/actor:np.float64(0.03384095922161414) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.00584030151367) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:3 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0143998172134161 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0143998172134161 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:315.69140625 - response_length/max:1345.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.69140625 - response_length_non_aborted/max:1345.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.84375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.757490336894989e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0721273869276047) - timing_s/agent_loop/generate_sequences/max:np.float64(8.888256212696433) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9832990684881224) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.888256212696433) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:1345 - timing_s/gen:14.835985887795687 - timing_s/reward:0.056042348966002464 - timing_s/old_log_prob:2.3734769448637962 - timing_s/adv:0.5705869812518358 - timing_s/update_actor:16.890875345095992 - timing_s/step:34.81418928876519 - timing_s/stop_profile:0.00010911934077739716 - timing_per_token_ms/gen:0.18357506326386389 - timing_per_token_ms/update_actor:0.10514149073505588 - timing_per_token_ms/adv:0.003551761799026672 - perf/total_num_tokens:160649 - perf/time_per_step:34.81418928876519 - perf/throughput:576.8086349343868 (TaskRunner pid=2122883) Training Progress: 48%|████▊ | 48/100 [39:52<32:21, 37.34s/it] (TaskRunner pid=2122883) step:49 - global_seqlen/min:12849 - global_seqlen/max:30904 - global_seqlen/minmax_diff:18055 - global_seqlen/balanced_min:19385 - global_seqlen/balanced_max:25931 - global_seqlen/mean:20211.375 - actor/entropy:0.15795230865478516 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.399758517742157 - training/rollout_probs_diff_mean:0.0037395432591438293 - training/rollout_probs_diff_std:0.012383424676954746 - training/rollout_actor_probs_pearson_corr:0.9980095028877258 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998835921287537 - rollout_corr/rollout_is_ratio_fraction_high:2.3283158952835947e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.149105269694701e-05 - rollout_corr/rollout_is_max:3.340711832046509 - rollout_corr/rollout_is_min:0.10319963097572327 - rollout_corr/rollout_is_std:0.03628097102046013 - rollout_corr/rollout_is_eff_sample_size:0.9986851836242739 - rollout_corr/rollout_is_seq_mean:0.9996575713157654 - rollout_corr/rollout_is_seq_std:0.002907124115154147 - rollout_corr/rollout_is_seq_max:1.0098154544830322 - rollout_corr/rollout_is_seq_min:0.9917104840278625 - rollout_corr/rollout_is_seq_max_deviation:0.009815454483032227 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2227676422335207) - rollout_corr/training_log_ppl:np.float64(0.19771692524227547) - rollout_corr/kl:np.float64(0.0017005499959275028) - rollout_corr/k3_kl:np.float64(0.001488717432039266) - rollout_corr/rollout_ppl:np.float64(1.2206510826945305) - rollout_corr/rollout_log_ppl:np.float64(0.19601637365849456) - rollout_corr/log_ppl_diff:np.float64(0.001700551583780907) - rollout_corr/log_ppl_abs_diff:np.float64(0.003168961076880805) - rollout_corr/log_ppl_diff_max:np.float64(0.017631560564041138) - rollout_corr/log_ppl_diff_min:np.float64(-0.011207103729248047) - rollout_corr/ppl_ratio:np.float64(1.0017099685501307) - rollout_corr/chi2_token:np.float64(0.002383438404649496) - rollout_corr/chi2_seq:np.float64(0.4581478724721819) - actor/pg_loss:np.float64(0.006989955902099609) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020739615124512056) - actor/ppo_kl:np.float64(0.0004834777322839656) - actor/pg_clipfrac_lower:np.float64(4.919710136164213e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(483083.6963332107) - self_distillation/token_reweight_w_std:np.float64(5497377.112956069) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001461925683543) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10720286803552881) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.6698927134275436) - perf/mfu/actor:np.float64(0.03237159299727145) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.85754013061523) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:3 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017634082585573196 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017634082585573196 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.54296875 - response_length/max:8192.0 - response_length/min:101.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:335.54296875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:296.0625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.936040103435516e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2021586298942566) - timing_s/agent_loop/generate_sequences/max:np.float64(48.67229326814413) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.981824685790343) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(48.67229326814413) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:54.559863248839974 - timing_s/reward:0.05820300057530403 - timing_s/old_log_prob:2.477139912545681 - timing_s/adv:0.5340056866407394 - timing_s/update_actor:18.079693160951138 - timing_s/step:75.79790463671088 - timing_s/stop_profile:0.00011322461068630219 - timing_per_token_ms/gen:0.6351629617206251 - timing_per_token_ms/update_actor:0.11181632348709042 - timing_per_token_ms/adv:0.003302630861586232 - perf/total_num_tokens:161691 - perf/time_per_step:75.79790463671088 - perf/throughput:266.6482021748542 (TaskRunner pid=2122883) Training Progress: 49%|████▉ | 49/100 [41:08<41:33, 48.89s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques such as molecular dynamics simulations or free energy calculations. However, without access to specific computational tools or databases that can calculate this score for the given sequence, it is not possible to determine the exact value. Among the provided options, the most reasonable value for a folding stability score is 0.45, as it is a common range for such scores in many computational contexts. Therefore, the most plausible answer is 0.45. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) D (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 0.0 (TaskRunner pid=2122883) [acc] 0.0 (TaskRunner pid=2122883) [pred] D (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2122883) step:50 - global_seqlen/min:10411 - global_seqlen/max:26898 - global_seqlen/minmax_diff:16487 - global_seqlen/balanced_min:20939 - global_seqlen/balanced_max:20959 - global_seqlen/mean:20951.625 - actor/entropy:0.16414521634578705 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6031620502471924 - training/rollout_probs_diff_mean:0.003770680632442236 - training/rollout_probs_diff_std:0.012851644307374954 - training/rollout_actor_probs_pearson_corr:0.9979996085166931 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001113414764404 - rollout_corr/rollout_is_ratio_fraction_high:3.634557288023643e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014538229152094573 - rollout_corr/rollout_is_max:3.7972795963287354 - rollout_corr/rollout_is_min:0.11608143150806427 - rollout_corr/rollout_is_std:0.03673160448670387 - rollout_corr/rollout_is_eff_sample_size:0.9986529638426448 - rollout_corr/rollout_is_seq_mean:1.0002330541610718 - rollout_corr/rollout_is_seq_std:0.003209285903722048 - rollout_corr/rollout_is_seq_max:1.0156590938568115 - rollout_corr/rollout_is_seq_min:0.991116464138031 - rollout_corr/rollout_is_seq_max_deviation:0.015659093856811523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.21245695091784) - rollout_corr/training_log_ppl:np.float64(0.18853073121863417) - rollout_corr/kl:np.float64(0.000940453585307921) - rollout_corr/k3_kl:np.float64(0.0011123225885398824) - rollout_corr/rollout_ppl:np.float64(1.211254812311381) - rollout_corr/rollout_log_ppl:np.float64(0.18759027698251884) - rollout_corr/log_ppl_diff:np.float64(0.000940454236115329) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025964345986722037) - rollout_corr/log_ppl_diff_max:np.float64(0.012625694274902344) - rollout_corr/log_ppl_diff_min:np.float64(-0.010028764605522156) - rollout_corr/ppl_ratio:np.float64(1.000946962274611) - rollout_corr/chi2_token:np.float64(0.003119073808193207) - rollout_corr/chi2_seq:np.float64(0.5444498066790402) - actor/pg_loss:np.float64(0.00562474865000695) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012370409526738513) - actor/ppo_kl:np.float64(0.00027735871690294164) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(552116.4400627858) - self_distillation/token_reweight_w_std:np.float64(5173609.766112542) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990502987056971) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09921138118079398) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.6405464708805084) - perf/mfu/actor:np.float64(0.03547942389442762) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8638153076172) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.41125) - val-aux/sciknoweval/reward/std@16:np.float64(0.282579055086121) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.53366) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22278060446414918) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.28978000000000004) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2565024603314222) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.41237999999999997) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.28174299705440914) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.61846) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1429779412119699) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.17951999999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.19721838418827287) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.435) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.23734796957612123) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.66646) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07860442389870713) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.10271999999999998) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11817685371390609) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.45018) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.18446648963991358) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.69634) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0437133055991171) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.061799999999999994) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05167005663723248) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.46228) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.13956489639286812) - val-aux/sciknoweval/score/mean@16:np.float64(0.41125) - val-aux/sciknoweval/score/std@16:np.float64(0.282579055086121) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.53366) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22278060446414918) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.28978000000000004) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2565024603314222) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.41237999999999997) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.28174299705440914) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.61846) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1429779412119699) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.17951999999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.19721838418827287) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.435) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.23734796957612123) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.66646) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07860442389870713) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.10271999999999998) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11817685371390609) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.45018) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.18446648963991358) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.69634) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0437133055991171) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.061799999999999994) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05167005663723248) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.46228) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.13956489639286812) - val-core/sciknoweval/acc/mean@16:np.float64(0.41125) - val-aux/sciknoweval/acc/std@16:np.float64(0.282579055086121) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.53366) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22278060446414918) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.28978000000000004) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2565024603314222) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.41237999999999997) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.28174299705440914) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.61846) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1429779412119699) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.17951999999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.19721838418827287) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.435) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.23734796957612123) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.66646) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07860442389870713) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.10271999999999998) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11817685371390609) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.45018) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.18446648963991358) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.69634) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0437133055991171) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.061799999999999994) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05167005663723248) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.46228) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.13956489639286812) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04534421173973564) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9991) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.011920881683771797) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9766400000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.05986222536212083) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.98834) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04394565641819693) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.95654) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0761555953412347) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99666) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.023358458330384288) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.92376) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08787697982491473) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99946) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008275134421193774) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.88028) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.08368042461510791) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999800000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0006321392251711644) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:3 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.015225766226649284 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.015225766226649284 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:322.42578125 - response_length/max:1273.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.42578125 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011311471462249756 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7200889475643635) - timing_s/agent_loop/generate_sequences/max:np.float64(7.788614375516772) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9231499576562783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.788614375516772) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:426 - timing_s/agent_loop/slowest/response_length:1273 - timing_s/gen:13.558132458478212 - timing_s/reward:0.05939585529267788 - timing_s/old_log_prob:2.340417541563511 - timing_s/adv:0.5785594340413809 - timing_s/update_actor:16.77560812421143 - timing_s/step:33.39719614572823 - timing_s/testing:73.396837702021 - timing_s/save_checkpoint:7.072169976308942 - timing_s/stop_profile:0.00019020028412342072 - timing_per_token_ms/gen:0.16425936756858062 - timing_per_token_ms/update_actor:0.10008536404820288 - timing_per_token_ms/adv:0.003451757525021215 - perf/total_num_tokens:167613 - perf/time_per_step:33.39719614572823 - perf/throughput:627.3468260202999 (TaskRunner pid=2122883) Training Progress: 50%|█████ | 50/100 [43:02<57:00, 68.41s/it] (TaskRunner pid=2122883) step:51 - global_seqlen/min:14724 - global_seqlen/max:24248 - global_seqlen/minmax_diff:9524 - global_seqlen/balanced_min:20154 - global_seqlen/balanced_max:20164 - global_seqlen/mean:20159.0 - actor/entropy:0.170307457447052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45877787470817566 - training/rollout_probs_diff_mean:0.004090906120836735 - training/rollout_probs_diff_std:0.012903047725558281 - training/rollout_actor_probs_pearson_corr:0.9979671835899353 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.37109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0002143383026123 - rollout_corr/rollout_is_ratio_fraction_high:5.2099614549661055e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.907471182174049e-05 - rollout_corr/rollout_is_max:2.633291482925415 - rollout_corr/rollout_is_min:0.09349794685840607 - rollout_corr/rollout_is_std:0.038960132747888565 - rollout_corr/rollout_is_eff_sample_size:0.9984850028614497 - rollout_corr/rollout_is_seq_mean:1.0002855062484741 - rollout_corr/rollout_is_seq_std:0.003179777879267931 - rollout_corr/rollout_is_seq_max:1.0115247964859009 - rollout_corr/rollout_is_seq_min:0.9862546324729919 - rollout_corr/rollout_is_seq_max_deviation:0.013745367527008057 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2051998022943735) - rollout_corr/training_log_ppl:np.float64(0.1835695655317977) - rollout_corr/kl:np.float64(0.0019460881946997688) - rollout_corr/k3_kl:np.float64(0.0018434013255159698) - rollout_corr/rollout_ppl:np.float64(1.2028441824950278) - rollout_corr/rollout_log_ppl:np.float64(0.18162347658653744) - rollout_corr/log_ppl_diff:np.float64(0.0019460889452602714) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031690965115558356) - rollout_corr/log_ppl_diff_max:np.float64(0.05628243088722229) - rollout_corr/log_ppl_diff_min:np.float64(-0.006894886493682861) - rollout_corr/ppl_ratio:np.float64(1.0019650857429951) - rollout_corr/chi2_token:np.float64(0.0026275888085365295) - rollout_corr/chi2_seq:np.float64(0.2529987192247063) - actor/pg_loss:np.float64(0.005627889535389841) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001572497030338127) - actor/ppo_kl:np.float64(0.0013443404310486962) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37109375) - self_distillation/token_reweight_w_mean:np.float64(655015.5512642735) - self_distillation/token_reweight_w_std:np.float64(7001571.50137981) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004862013738602) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09912433978752233) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.678714781999588) - perf/mfu/actor:np.float64(0.034301143289589886) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.97393035888672) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:3 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005240895785391331 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005240895785391331 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.90625 - response_length/max:1032.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.90625 - response_length_non_aborted/max:1032.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.0625 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011244602501392365 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1639153379946947) - timing_s/agent_loop/generate_sequences/max:np.float64(7.034878183156252) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9088878281690995) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.034878183156252) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:1032 - timing_s/gen:12.909786026924849 - timing_s/reward:0.05316934175789356 - timing_s/old_log_prob:2.4393669683486223 - timing_s/adv:0.5470726881176233 - timing_s/update_actor:16.608018811792135 - timing_s/step:32.66919903643429 - timing_s/stop_profile:0.00011144764721393585 - timing_per_token_ms/gen:0.16814871870017778 - timing_per_token_ms/update_actor:0.10298141532189181 - timing_per_token_ms/adv:0.0033922360243416297 - perf/total_num_tokens:161272 - perf/time_per_step:32.66919903643429 - perf/throughput:617.064409124867 (TaskRunner pid=2122883) Training Progress: 51%|█████ | 51/100 [43:34<47:07, 57.71s/it] (TaskRunner pid=2122883) step:52 - global_seqlen/min:11438 - global_seqlen/max:23198 - global_seqlen/minmax_diff:11760 - global_seqlen/balanced_min:16158 - global_seqlen/balanced_max:16373 - global_seqlen/mean:16195.625 - actor/entropy:0.1720925271511078 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7303562164306641 - training/rollout_probs_diff_mean:0.0043999128974974155 - training/rollout_probs_diff_std:0.0137338163331151 - training/rollout_actor_probs_pearson_corr:0.9976982474327087 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.6171875 - self_distillation/correct_sample_fraction:0.4609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6171875 - rollout_corr/rollout_is_mean:0.999703049659729 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013763441529590636 - rollout_corr/rollout_is_max:1.7618439197540283 - rollout_corr/rollout_is_min:0.03414538502693176 - rollout_corr/rollout_is_std:0.03829740732908249 - rollout_corr/rollout_is_eff_sample_size:0.9985346839476352 - rollout_corr/rollout_is_seq_mean:0.9996437430381775 - rollout_corr/rollout_is_seq_std:0.003205541055649519 - rollout_corr/rollout_is_seq_max:1.008802890777588 - rollout_corr/rollout_is_seq_min:0.9890128374099731 - rollout_corr/rollout_is_seq_max_deviation:0.010987162590026855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2097071488387883) - rollout_corr/training_log_ppl:np.float64(0.18765684607205912) - rollout_corr/kl:np.float64(0.0014238237613923843) - rollout_corr/k3_kl:np.float64(0.0011359687883611969) - rollout_corr/rollout_ppl:np.float64(1.2079426697455347) - rollout_corr/rollout_log_ppl:np.float64(0.18623302060586866) - rollout_corr/log_ppl_diff:np.float64(0.0014238254661904648) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030733039166079834) - rollout_corr/log_ppl_diff_max:np.float64(0.016380250453948975) - rollout_corr/log_ppl_diff_min:np.float64(-0.01520422101020813) - rollout_corr/ppl_ratio:np.float64(1.0014326500240713) - rollout_corr/chi2_token:np.float64(0.001753966324031353) - rollout_corr/chi2_seq:np.float64(0.811463154386729) - actor/pg_loss:np.float64(0.0041076281340792775) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010653491672201199) - actor/ppo_kl:np.float64(0.00019217796503312456) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4609375) - self_distillation/token_reweight_w_mean:np.float64(546943.084549105) - self_distillation/token_reweight_w_std:np.float64(5536941.048060441) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003938139416277) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07578522652329411) - self_distillation/empty_target_batch:np.float64(0.3828125) - actor/grad_norm:np.float64(0.6149652451276779) - perf/mfu/actor:np.float64(0.02765064554259175) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.88561248779297) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:3 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01032258104532957 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01032258104532957 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:227.05078125 - response_length/max:1591.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.05078125 - response_length_non_aborted/max:1591.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.0625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001474972814321518 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1343602985143661) - timing_s/agent_loop/generate_sequences/max:np.float64(8.92716247588396) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.213136707934609) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.92716247588396) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1591 - timing_s/gen:14.800771605223417 - timing_s/reward:0.05206748843193054 - timing_s/old_log_prob:2.2524862699210644 - timing_s/adv:0.5898469276726246 - timing_s/update_actor:16.551839008927345 - timing_s/step:34.33575841598213 - timing_s/stop_profile:2.8537586331367493e-05 - timing_per_token_ms/gen:0.2546369308425534 - timing_per_token_ms/update_actor:0.12774930736639792 - timing_per_token_ms/adv:0.0045525174829052955 - perf/total_num_tokens:129565 - perf/time_per_step:34.33575841598213 - perf/throughput:471.68391633549845 (TaskRunner pid=2122883) Training Progress: 52%|█████▏ | 52/100 [44:09<40:33, 50.70s/it] (TaskRunner pid=2122883) step:53 - global_seqlen/min:13746 - global_seqlen/max:27399 - global_seqlen/minmax_diff:13653 - global_seqlen/balanced_min:20157 - global_seqlen/balanced_max:20178 - global_seqlen/mean:20171.5 - actor/entropy:0.17745019495487213 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.717710018157959 - training/rollout_probs_diff_mean:0.003805093001574278 - training/rollout_probs_diff_std:0.012700757943093777 - training/rollout_actor_probs_pearson_corr:0.9981307983398438 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.4453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.999994158744812 - rollout_corr/rollout_is_ratio_fraction_high:6.55514159006998e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001573234039824456 - rollout_corr/rollout_is_max:5.53007698059082 - rollout_corr/rollout_is_min:0.014541520737111568 - rollout_corr/rollout_is_std:0.036859575659036636 - rollout_corr/rollout_is_eff_sample_size:0.9986429773213285 - rollout_corr/rollout_is_seq_mean:1.0000602006912231 - rollout_corr/rollout_is_seq_std:0.003147674258798361 - rollout_corr/rollout_is_seq_max:1.0142172574996948 - rollout_corr/rollout_is_seq_min:0.9902289509773254 - rollout_corr/rollout_is_seq_max_deviation:0.014217257499694824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2197709884494543) - rollout_corr/training_log_ppl:np.float64(0.19364592210331466) - rollout_corr/kl:np.float64(0.0007903323407845164) - rollout_corr/k3_kl:np.float64(0.0009314046881740978) - rollout_corr/rollout_ppl:np.float64(1.218785626348108) - rollout_corr/rollout_log_ppl:np.float64(0.1928555882332148) - rollout_corr/log_ppl_diff:np.float64(0.0007903338700998574) - rollout_corr/log_ppl_abs_diff:np.float64(0.002397695294348523) - rollout_corr/log_ppl_diff_max:np.float64(0.010247737169265747) - rollout_corr/log_ppl_diff_min:np.float64(-0.007017850875854492) - rollout_corr/ppl_ratio:np.float64(1.0007952337618917) - rollout_corr/chi2_token:np.float64(0.0022767260670661926) - rollout_corr/chi2_seq:np.float64(0.6853257885668427) - actor/pg_loss:np.float64(0.003560527227818966) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000947475578641388) - actor/ppo_kl:np.float64(-5.2774539890165784e-05) - actor/pg_clipfrac_lower:np.float64(1.4051259313418996e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4453125) - self_distillation/token_reweight_w_mean:np.float64(511538.4202222314) - self_distillation/token_reweight_w_std:np.float64(5941356.751985744) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015118459705263) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08175502343510743) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.4588034823536873) - perf/mfu/actor:np.float64(0.034160046659816826) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.91546630859375) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:3 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.016197755932807922 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.016197755932807922 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:297.953125 - response_length/max:1277.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:297.953125 - response_length_non_aborted/max:1277.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:332.40625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.4145469069480896e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7408090196549892) - timing_s/agent_loop/generate_sequences/max:np.float64(8.517019476741552) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6340090800222242) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.517019476741552) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1277 - timing_s/gen:14.336498888209462 - timing_s/reward:0.05450882762670517 - timing_s/old_log_prob:2.330040417611599 - timing_s/adv:0.5633099637925625 - timing_s/update_actor:16.740238409489393 - timing_s/step:34.11834845691919 - timing_s/stop_profile:0.0001228339970111847 - timing_per_token_ms/gen:0.18795556778291286 - timing_per_token_ms/update_actor:0.10373694574950669 - timing_per_token_ms/adv:0.003490754057659089 - perf/total_num_tokens:161372 - perf/time_per_step:34.11834845691919 - perf/throughput:591.2214662286569 (TaskRunner pid=2122883) Training Progress: 53%|█████▎ | 53/100 [44:43<35:49, 45.74s/it] (TaskRunner pid=2122883) step:54 - global_seqlen/min:11593 - global_seqlen/max:21800 - global_seqlen/minmax_diff:10207 - global_seqlen/balanced_min:18264 - global_seqlen/balanced_max:18625 - global_seqlen/mean:18316.875 - actor/entropy:0.19040422141551971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5750171542167664 - training/rollout_probs_diff_mean:0.004019932821393013 - training/rollout_probs_diff_std:0.012364245019853115 - training/rollout_actor_probs_pearson_corr:0.9983108639717102 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.82421875 - self_distillation/correct_sample_fraction:0.51171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.82421875 - rollout_corr/rollout_is_mean:0.9999626874923706 - rollout_corr/rollout_is_ratio_fraction_high:4.251760765328072e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014172536612022668 - rollout_corr/rollout_is_max:3.3674063682556152 - rollout_corr/rollout_is_min:0.07618959248065948 - rollout_corr/rollout_is_std:0.03785360977053642 - rollout_corr/rollout_is_eff_sample_size:0.9985689168550453 - rollout_corr/rollout_is_seq_mean:0.9999775290489197 - rollout_corr/rollout_is_seq_std:0.002974021015688777 - rollout_corr/rollout_is_seq_max:1.0119009017944336 - rollout_corr/rollout_is_seq_min:0.9925116896629333 - rollout_corr/rollout_is_seq_max_deviation:0.011900901794433594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2383890594355762) - rollout_corr/training_log_ppl:np.float64(0.2090079904155573) - rollout_corr/kl:np.float64(0.001236176363808461) - rollout_corr/k3_kl:np.float64(0.0011164603104845128) - rollout_corr/rollout_ppl:np.float64(1.2368075558915734) - rollout_corr/rollout_log_ppl:np.float64(0.2077718133223243) - rollout_corr/log_ppl_diff:np.float64(0.0012361770932329819) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026738402812043205) - rollout_corr/log_ppl_diff_max:np.float64(0.011941254138946533) - rollout_corr/log_ppl_diff_min:np.float64(-0.008814394474029541) - rollout_corr/ppl_ratio:np.float64(1.0012423689477146) - rollout_corr/chi2_token:np.float64(0.00228139734826982) - rollout_corr/chi2_seq:np.float64(0.4417079407721758) - actor/pg_loss:np.float64(0.004966978449374437) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016709646415620227) - actor/ppo_kl:np.float64(0.000317133653719992) - actor/pg_clipfrac_lower:np.float64(1.306438116444042e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.82421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51171875) - self_distillation/token_reweight_w_mean:np.float64(693708.3943432588) - self_distillation/token_reweight_w_std:np.float64(7959025.133003578) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0026721898466349) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1158001013827743) - self_distillation/empty_target_batch:np.float64(0.17578125) - actor/grad_norm:np.float64(0.660299763083458) - perf/mfu/actor:np.float64(0.030901043838188515) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.95029067993164) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:3 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.02446356974542141 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.02446356974542141 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.62109375 - response_length/max:1859.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.62109375 - response_length_non_aborted/max:1859.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.864196181297302e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6340610478073359) - timing_s/agent_loop/generate_sequences/max:np.float64(10.501076608896255) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3894221871960326) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.501076608896255) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:1859 - timing_s/gen:16.715160904452205 - timing_s/reward:0.07296787202358246 - timing_s/old_log_prob:2.283285263925791 - timing_s/adv:0.6101323831826448 - timing_s/update_actor:16.79112665913999 - timing_s/step:36.570522367954254 - timing_s/stop_profile:0.0001197177916765213 - timing_per_token_ms/gen:0.23689622733389368 - timing_per_token_ms/update_actor:0.11458782310806286 - timing_per_token_ms/adv:0.004163731416949158 - perf/total_num_tokens:146535 - perf/time_per_step:36.570522367954254 - perf/throughput:500.86446170237303 (TaskRunner pid=2122883) Training Progress: 54%|█████▍ | 54/100 [45:19<32:58, 43.00s/it] (TaskRunner pid=2122883) step:55 - global_seqlen/min:14316 - global_seqlen/max:31135 - global_seqlen/minmax_diff:16819 - global_seqlen/balanced_min:21521 - global_seqlen/balanced_max:21741 - global_seqlen/mean:21562.625 - actor/entropy:0.1820475459098816 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6819348335266113 - training/rollout_probs_diff_mean:0.0036203861236572266 - training/rollout_probs_diff_std:0.01213095709681511 - training/rollout_actor_probs_pearson_corr:0.9983159303665161 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000014305114746 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:9.1028050519526e-05 - rollout_corr/rollout_is_max:1.9132329225540161 - rollout_corr/rollout_is_min:0.05214012414216995 - rollout_corr/rollout_is_std:0.03391006961464882 - rollout_corr/rollout_is_eff_sample_size:0.9988514279277574 - rollout_corr/rollout_is_seq_mean:0.9999558925628662 - rollout_corr/rollout_is_seq_std:0.002855920698493719 - rollout_corr/rollout_is_seq_max:1.0095938444137573 - rollout_corr/rollout_is_seq_min:0.9906392693519592 - rollout_corr/rollout_is_seq_max_deviation:0.009593844413757324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2300612814724445) - rollout_corr/training_log_ppl:np.float64(0.2026208189490717) - rollout_corr/kl:np.float64(0.0009841419239613458) - rollout_corr/k3_kl:np.float64(0.0009232644666212764) - rollout_corr/rollout_ppl:np.float64(1.2287878943607211) - rollout_corr/rollout_log_ppl:np.float64(0.20163667347515002) - rollout_corr/log_ppl_diff:np.float64(0.0009841454739216715) - rollout_corr/log_ppl_abs_diff:np.float64(0.002437286515487358) - rollout_corr/log_ppl_diff_max:np.float64(0.011938422918319702) - rollout_corr/log_ppl_diff_min:np.float64(-0.008513659238815308) - rollout_corr/ppl_ratio:np.float64(1.0009896873962134) - rollout_corr/chi2_token:np.float64(0.0016990478616207838) - rollout_corr/chi2_seq:np.float64(0.6990436653140932) - actor/pg_loss:np.float64(0.0053936580661684275) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010540454873080307) - actor/ppo_kl:np.float64(0.0001687855753793599) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(464849.280399007) - self_distillation/token_reweight_w_std:np.float64(4769288.998040117) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998792761471123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11009695939719677) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.6255688071250916) - perf/mfu/actor:np.float64(0.035841929135174845) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8391571044922) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:3 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0347784049808979 - critic/advantages/max:0.75 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0347784049808979 - critic/returns/max:0.75 - critic/returns/min:-0.75 - response_length/mean:343.30078125 - response_length/max:1852.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.30078125 - response_length_non_aborted/max:1852.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:498.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010434910655021667 - timing_s/agent_loop/generate_sequences/min:np.float64(1.195070143789053) - timing_s/agent_loop/generate_sequences/max:np.float64(11.193672521039844) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2173899157933192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.193672521039844) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:432 - timing_s/agent_loop/slowest/response_length:1852 - timing_s/gen:17.2607198394835 - timing_s/reward:0.060050373896956444 - timing_s/old_log_prob:2.3352784402668476 - timing_s/adv:0.6170326750725508 - timing_s/update_actor:17.13137748092413 - timing_s/step:37.49287431128323 - timing_s/stop_profile:0.00011833943426609039 - timing_per_token_ms/gen:0.19640120429519828 - timing_per_token_ms/update_actor:0.09931175750241523 - timing_per_token_ms/adv:0.0035769802787957794 - perf/total_num_tokens:172501 - perf/time_per_step:37.49287431128323 - perf/throughput:575.112615292631 (TaskRunner pid=2122883) Training Progress: 55%|█████▌ | 55/100 [45:57<31:01, 41.36s/it] (TaskRunner pid=2122883) step:56 - global_seqlen/min:11656 - global_seqlen/max:32815 - global_seqlen/minmax_diff:21159 - global_seqlen/balanced_min:19457 - global_seqlen/balanced_max:20231 - global_seqlen/mean:19623.75 - actor/entropy:0.2085179090499878 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5829838514328003 - training/rollout_probs_diff_mean:0.00421536061912775 - training/rollout_probs_diff_std:0.012636987492442131 - training/rollout_actor_probs_pearson_corr:0.9983205199241638 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.51953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:1.0000214576721191 - rollout_corr/rollout_is_ratio_fraction_high:4.0095157601172104e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.355536894872785e-05 - rollout_corr/rollout_is_max:2.7182812690734863 - rollout_corr/rollout_is_min:0.10317684710025787 - rollout_corr/rollout_is_std:0.037414900958538055 - rollout_corr/rollout_is_eff_sample_size:0.9986023199750391 - rollout_corr/rollout_is_seq_mean:1.0000838041305542 - rollout_corr/rollout_is_seq_std:0.0029523512348532677 - rollout_corr/rollout_is_seq_max:1.0097237825393677 - rollout_corr/rollout_is_seq_min:0.9897148013114929 - rollout_corr/rollout_is_seq_max_deviation:0.01028519868850708 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2682426059618592) - rollout_corr/training_log_ppl:np.float64(0.23284589135437272) - rollout_corr/kl:np.float64(0.001132030002182205) - rollout_corr/k3_kl:np.float64(0.0013115676476758154) - rollout_corr/rollout_ppl:np.float64(1.2667685416527092) - rollout_corr/rollout_log_ppl:np.float64(0.23171385923342314) - rollout_corr/log_ppl_diff:np.float64(0.0011320321209495887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028422162431525066) - rollout_corr/log_ppl_diff_max:np.float64(0.017789900302886963) - rollout_corr/log_ppl_diff_min:np.float64(-0.010757237672805786) - rollout_corr/ppl_ratio:np.float64(1.0011398843489587) - rollout_corr/chi2_token:np.float64(0.0028780815191566944) - rollout_corr/chi2_seq:np.float64(0.5465702745132148) - actor/pg_loss:np.float64(0.004153699497692287) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017756936499608855) - actor/ppo_kl:np.float64(0.00032732845443295133) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51953125) - self_distillation/token_reweight_w_mean:np.float64(616782.949018436) - self_distillation/token_reweight_w_std:np.float64(6379718.014931552) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017636623233557) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10283011330466252) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.5960900634527206) - perf/mfu/actor:np.float64(0.032728240701524886) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.97694778442383) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:3 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006097805453464389 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006097805453464389 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:292.2734375 - response_length/max:2520.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:292.2734375 - response_length_non_aborted/max:2520.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:320.96875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016748160123825073 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7558852564543486) - timing_s/agent_loop/generate_sequences/max:np.float64(13.91053842753172) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4859224638566957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.91053842753172) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:2520 - timing_s/gen:19.682910569012165 - timing_s/reward:0.058423759415745735 - timing_s/old_log_prob:2.3729702048003674 - timing_s/adv:0.6314276736229658 - timing_s/update_actor:17.3532410915941 - timing_s/step:40.18750685453415 - timing_s/stop_profile:0.00011737458407878876 - timing_per_token_ms/gen:0.2630631441155297 - timing_per_token_ms/update_actor:0.1105372386240786 - timing_per_token_ms/adv:0.004022088500050741 - perf/total_num_tokens:156990 - perf/time_per_step:40.18750685453415 - perf/throughput:488.30473786372625 (TaskRunner pid=2122883) Training Progress: 56%|█████▌ | 56/100 [46:37<30:05, 41.03s/it] (TaskRunner pid=2122883) step:57 - global_seqlen/min:16389 - global_seqlen/max:26616 - global_seqlen/minmax_diff:10227 - global_seqlen/balanced_min:20467 - global_seqlen/balanced_max:20671 - global_seqlen/mean:20501.625 - actor/entropy:0.18013454973697662 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5182572603225708 - training/rollout_probs_diff_mean:0.0037767167668789625 - training/rollout_probs_diff_std:0.011676754802465439 - training/rollout_actor_probs_pearson_corr:0.9983941316604614 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0001684427261353 - rollout_corr/rollout_is_ratio_fraction_high:2.403008511464577e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2015042557322886e-05 - rollout_corr/rollout_is_max:4.898484230041504 - rollout_corr/rollout_is_min:0.0921703428030014 - rollout_corr/rollout_is_std:0.03423544391989708 - rollout_corr/rollout_is_eff_sample_size:0.9988296631861381 - rollout_corr/rollout_is_seq_mean:1.0001211166381836 - rollout_corr/rollout_is_seq_std:0.0027834821958094835 - rollout_corr/rollout_is_seq_max:1.0092366933822632 - rollout_corr/rollout_is_seq_min:0.9900839328765869 - rollout_corr/rollout_is_seq_max_deviation:0.009916067123413086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.225984800606966) - rollout_corr/training_log_ppl:np.float64(0.199987623404013) - rollout_corr/kl:np.float64(0.0013101380760858206) - rollout_corr/k3_kl:np.float64(0.0010357795563322725) - rollout_corr/rollout_ppl:np.float64(1.2243200689554214) - rollout_corr/rollout_log_ppl:np.float64(0.19867748432443477) - rollout_corr/log_ppl_diff:np.float64(0.0013101390795782208) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026235540281049907) - rollout_corr/log_ppl_diff_max:np.float64(0.028497636318206787) - rollout_corr/log_ppl_diff_min:np.float64(-0.01019953191280365) - rollout_corr/ppl_ratio:np.float64(1.0013179222587496) - rollout_corr/chi2_token:np.float64(0.001341148978099227) - rollout_corr/chi2_seq:np.float64(0.48321546288207173) - actor/pg_loss:np.float64(0.00407763512339443) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010287672778304113) - actor/ppo_kl:np.float64(0.0006644770689980817) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(196380.35628112592) - self_distillation/token_reweight_w_std:np.float64(3012296.647777552) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001179282553494) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08650214066437911) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.47980305552482605) - perf/mfu/actor:np.float64(0.03425298709048464) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7446174621582) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:4 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015083384700119495 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015083384700119495 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.11328125 - response_length/max:1789.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:325.11328125 - response_length_non_aborted/max:1789.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.5625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005521513521671295 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1972173005342484) - timing_s/agent_loop/generate_sequences/max:np.float64(10.842901173979044) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.050342304566584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.842901173979044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1789 - timing_s/gen:17.024502815678716 - timing_s/reward:0.07034870982170105 - timing_s/old_log_prob:2.4137939233332872 - timing_s/adv:0.6624380126595497 - timing_s/update_actor:16.968600936233997 - timing_s/step:37.206819297745824 - timing_s/stop_profile:0.00011276639997959137 - timing_per_token_ms/gen:0.20455013055159518 - timing_per_token_ms/update_actor:0.10345887787086389 - timing_per_token_ms/adv:0.004038936015191172 - perf/total_num_tokens:164013 - perf/time_per_step:37.206819297745824 - perf/throughput:551.0179420588659 (TaskRunner pid=2122883) Training Progress: 57%|█████▋ | 57/100 [47:15<28:41, 40.03s/it] (TaskRunner pid=2122883) step:58 - global_seqlen/min:11162 - global_seqlen/max:25766 - global_seqlen/minmax_diff:14604 - global_seqlen/balanced_min:18802 - global_seqlen/balanced_max:18833 - global_seqlen/mean:18818.25 - actor/entropy:0.19926981627941132 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5310959815979004 - training/rollout_probs_diff_mean:0.004314659163355827 - training/rollout_probs_diff_std:0.012799865566194057 - training/rollout_actor_probs_pearson_corr:0.9982891082763672 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.796875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.796875 - rollout_corr/rollout_is_mean:1.0001503229141235 - rollout_corr/rollout_is_ratio_fraction_high:5.6700591812841594e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.8350295906420797e-05 - rollout_corr/rollout_is_max:6.627471923828125 - rollout_corr/rollout_is_min:0.4177379608154297 - rollout_corr/rollout_is_std:0.0387515090405941 - rollout_corr/rollout_is_eff_sample_size:0.9985008099435283 - rollout_corr/rollout_is_seq_mean:1.000192403793335 - rollout_corr/rollout_is_seq_std:0.003255679737776518 - rollout_corr/rollout_is_seq_max:1.0241525173187256 - rollout_corr/rollout_is_seq_min:0.9918437004089355 - rollout_corr/rollout_is_seq_max_deviation:0.024152517318725586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2417523465119302) - rollout_corr/training_log_ppl:np.float64(0.21233980829128996) - rollout_corr/kl:np.float64(0.001212370029840315) - rollout_corr/k3_kl:np.float64(0.0011637865334535036) - rollout_corr/rollout_ppl:np.float64(1.2402377198450267) - rollout_corr/rollout_log_ppl:np.float64(0.21112743644334842) - rollout_corr/log_ppl_diff:np.float64(0.0012123718479415402) - rollout_corr/log_ppl_abs_diff:np.float64(0.002773932719719596) - rollout_corr/log_ppl_diff_max:np.float64(0.011079519987106323) - rollout_corr/log_ppl_diff_min:np.float64(-0.012328743934631348) - rollout_corr/ppl_ratio:np.float64(1.0012194933369756) - rollout_corr/chi2_token:np.float64(0.0026497936341911554) - rollout_corr/chi2_seq:np.float64(0.6413168597500771) - actor/pg_loss:np.float64(0.004801217350177467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016218156224567792) - actor/ppo_kl:np.float64(0.0004635169452154031) - actor/pg_clipfrac_lower:np.float64(1.594387686054688e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(672229.3530433087) - self_distillation/token_reweight_w_std:np.float64(7644332.062132216) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019736965186894) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09522422318696044) - self_distillation/empty_target_batch:np.float64(0.203125) - actor/grad_norm:np.float64(0.655727356672287) - perf/mfu/actor:np.float64(0.031754083710448994) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.72417449951172) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:4 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005737391300499439 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005737391300499439 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.5703125 - response_length/max:1404.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.5703125 - response_length_non_aborted/max:1404.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.204843521118164e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0642540324479342) - timing_s/agent_loop/generate_sequences/max:np.float64(8.915480151772499) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6220424274288234) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.915480151772499) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1404 - timing_s/gen:14.68929124251008 - timing_s/reward:0.08670529164373875 - timing_s/old_log_prob:2.2912397608160973 - timing_s/adv:0.5384073723107576 - timing_s/update_actor:16.798460753634572 - timing_s/step:34.49156256392598 - timing_s/stop_profile:0.0001234114170074463 - timing_per_token_ms/gen:0.208222879291669 - timing_per_token_ms/update_actor:0.11158357414766631 - timing_per_token_ms/adv:0.0035763645152362576 - perf/total_num_tokens:150546 - perf/time_per_step:34.49156256392598 - perf/throughput:545.5899530536672 (TaskRunner pid=2122883) Training Progress: 58%|█████▊ | 58/100 [47:49<26:52, 38.38s/it] (TaskRunner pid=2122883) step:59 - global_seqlen/min:14214 - global_seqlen/max:27268 - global_seqlen/minmax_diff:13054 - global_seqlen/balanced_min:19736 - global_seqlen/balanced_max:19748 - global_seqlen/mean:19742.625 - actor/entropy:0.20446516573429108 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7574851512908936 - training/rollout_probs_diff_mean:0.004479140043258667 - training/rollout_probs_diff_std:0.01373197603970766 - training/rollout_actor_probs_pearson_corr:0.9980234503746033 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0000429153442383 - rollout_corr/rollout_is_ratio_fraction_high:5.430281817098148e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001357570436084643 - rollout_corr/rollout_is_max:2.6759519577026367 - rollout_corr/rollout_is_min:0.12958838045597076 - rollout_corr/rollout_is_std:0.03966965898871422 - rollout_corr/rollout_is_eff_sample_size:0.9984289095870793 - rollout_corr/rollout_is_seq_mean:1.0000368356704712 - rollout_corr/rollout_is_seq_std:0.003124730195850134 - rollout_corr/rollout_is_seq_max:1.0123945474624634 - rollout_corr/rollout_is_seq_min:0.9908593893051147 - rollout_corr/rollout_is_seq_max_deviation:0.012394547462463379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2470811461098492) - rollout_corr/training_log_ppl:np.float64(0.2167818500311114) - rollout_corr/kl:np.float64(0.00108407483580919) - rollout_corr/k3_kl:np.float64(0.0011458040500684774) - rollout_corr/rollout_ppl:np.float64(1.2456995188258588) - rollout_corr/rollout_log_ppl:np.float64(0.2156977732811356) - rollout_corr/log_ppl_diff:np.float64(0.001084076749975793) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028995445900363848) - rollout_corr/log_ppl_diff_max:np.float64(0.022052377462387085) - rollout_corr/log_ppl_diff_min:np.float64(-0.008070424199104309) - rollout_corr/ppl_ratio:np.float64(1.001091847429052) - rollout_corr/chi2_token:np.float64(0.0023028659634292126) - rollout_corr/chi2_seq:np.float64(0.7561908534262329) - actor/pg_loss:np.float64(0.00433993327897042) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012840861065797071) - actor/ppo_kl:np.float64(0.00017092815455699295) - actor/pg_clipfrac_lower:np.float64(1.198236168420408e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(333998.30719705485) - self_distillation/token_reweight_w_std:np.float64(3725829.127407816) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999660367378965) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10178501263726503) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.561421312391758) - perf/mfu/actor:np.float64(0.03354169990330243) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7161407470703) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:4 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005727250594645739 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005727250594645739 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:287.73828125 - response_length/max:1149.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.73828125 - response_length_non_aborted/max:1149.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:329.21875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011886097490787506 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9756097439676523) - timing_s/agent_loop/generate_sequences/max:np.float64(7.369482506066561) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.742447040465777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.369482506066561) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1127 - timing_s/gen:13.280925517901778 - timing_s/reward:0.05712872929871082 - timing_s/old_log_prob:2.2685175091028214 - timing_s/adv:0.5434238649904728 - timing_s/update_actor:16.652599392458797 - timing_s/step:32.89270040392876 - timing_s/stop_profile:0.00012143328785896301 - timing_per_token_ms/gen:0.1802979258753177 - timing_per_token_ms/update_actor:0.10543557019683804 - timing_per_token_ms/adv:0.003440676360099485 - perf/total_num_tokens:157941 - perf/time_per_step:32.89270040392876 - perf/throughput:600.2129578160725 (TaskRunner pid=2122883) Training Progress: 59%|█████▉ | 59/100 [48:22<25:06, 36.75s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques, which are not provided here. Since the question does not provide the necessary context or data to calculate the score, the answer must be based on the given options. Among the options, 1.69 is the most reasonable value for a folding stability score, as it is a positive value that indicates a stable structure. Therefore, the most plausible answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2122883) step:60 - global_seqlen/min:10032 - global_seqlen/max:24065 - global_seqlen/minmax_diff:14033 - global_seqlen/balanced_min:15921 - global_seqlen/balanced_max:15934 - global_seqlen/mean:15929.5 - actor/entropy:0.20668578147888184 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7802466750144958 - training/rollout_probs_diff_mean:0.004730229265987873 - training/rollout_probs_diff_std:0.014051761478185654 - training/rollout_actor_probs_pearson_corr:0.997917652130127 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.5703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:0.9999483227729797 - rollout_corr/rollout_is_ratio_fraction_high:8.867134602041915e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010640561959007755 - rollout_corr/rollout_is_max:2.6651501655578613 - rollout_corr/rollout_is_min:0.12561684846878052 - rollout_corr/rollout_is_std:0.040434580296278 - rollout_corr/rollout_is_eff_sample_size:0.9983674756689369 - rollout_corr/rollout_is_seq_mean:0.9998451471328735 - rollout_corr/rollout_is_seq_std:0.0032432135194540024 - rollout_corr/rollout_is_seq_max:1.0128782987594604 - rollout_corr/rollout_is_seq_min:0.9875000715255737 - rollout_corr/rollout_is_seq_max_deviation:0.01287829875946045 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2383267022669315) - rollout_corr/training_log_ppl:np.float64(0.2105867949721869) - rollout_corr/kl:np.float64(0.0015159379858360467) - rollout_corr/k3_kl:np.float64(0.001174004459585376) - rollout_corr/rollout_ppl:np.float64(1.2364251278340816) - rollout_corr/rollout_log_ppl:np.float64(0.20907085489307065) - rollout_corr/log_ppl_diff:np.float64(0.0015159400791162625) - rollout_corr/log_ppl_abs_diff:np.float64(0.003122222449746914) - rollout_corr/log_ppl_diff_max:np.float64(0.01887175440788269) - rollout_corr/log_ppl_diff_min:np.float64(-0.01356002688407898) - rollout_corr/ppl_ratio:np.float64(1.0015250875148922) - rollout_corr/chi2_token:np.float64(0.001692107180133462) - rollout_corr/chi2_seq:np.float64(1.5169144957326353) - actor/pg_loss:np.float64(0.005143438349477947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001860541247879155) - actor/ppo_kl:np.float64(0.000468954139691391) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5703125) - self_distillation/token_reweight_w_mean:np.float64(806074.5182150835) - self_distillation/token_reweight_w_std:np.float64(8268406.419227212) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000429536215961) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12600182130699977) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.6441592276096344) - perf/mfu/actor:np.float64(0.027118054410325103) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.64035034179688) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.40625) - val-aux/sciknoweval/reward/std@16:np.float64(0.23654045275840702) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5031399999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1897584829312906) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.30985999999999997) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2073300701260786) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4065) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.2355960011007953) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.5783400000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.13053708350897217) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.22804) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.16287994961540497) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.41476) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18970814666766017) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6298999999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08378253995073645) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.15946) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11946173439217271) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.41358000000000006) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13246666664586865) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6596000000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04367187754785185) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.10722) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07124780446176789) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.40738) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.08970724337030629) - val-aux/sciknoweval/score/mean@16:np.float64(0.40625) - val-aux/sciknoweval/score/std@16:np.float64(0.23654045275840702) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5031399999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1897584829312906) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.30985999999999997) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.2073300701260786) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4065) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.2355960011007953) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.5783400000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.13053708350897217) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.22804) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.16287994961540497) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.41476) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18970814666766017) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6298999999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08378253995073645) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.15946) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11946173439217271) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.41358000000000006) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13246666664586865) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6596000000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04367187754785185) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.10722) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07124780446176789) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.40738) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.08970724337030629) - val-core/sciknoweval/acc/mean@16:np.float64(0.40625) - val-aux/sciknoweval/acc/std@16:np.float64(0.23654045275840702) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5031399999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1897584829312906) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.30985999999999997) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.2073300701260786) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4065) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.2355960011007953) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.5783400000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.13053708350897217) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.22804) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.16287994961540497) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.41476) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18970814666766017) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6298999999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08378253995073645) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.15946) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11946173439217271) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.41358000000000006) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13246666664586865) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6596000000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04367187754785185) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.10722) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07124780446176789) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.40738) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.08970724337030629) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.004841229182759271) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99992) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0012623787070447602) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9973400000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.006791494680848981) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99866) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.005000439980641703) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9951399999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008577901841359575) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9996200000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0027304944607158617) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9918399999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.009829262434180908) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0014106735979665884) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.98726) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.009617296917533533) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:4 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.019443409517407417 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.019443409517407417 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:220.265625 - response_length/max:1047.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:220.265625 - response_length_non_aborted/max:1047.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013137049973011017 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9903028048574924) - timing_s/agent_loop/generate_sequences/max:np.float64(6.458359025418758) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1430912712676218) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.458359025418758) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1047 - timing_s/gen:12.397510398179293 - timing_s/reward:0.04977481998503208 - timing_s/old_log_prob:2.2000094763934612 - timing_s/adv:0.503861652687192 - timing_s/update_actor:16.62571001984179 - timing_s/step:31.86980871669948 - timing_s/testing:66.30790391378105 - timing_s/save_checkpoint:6.745530238375068 - timing_s/stop_profile:3.4166499972343445e-05 - timing_per_token_ms/gen:0.21986079304425218 - timing_per_token_ms/update_actor:0.13046321306257094 - timing_per_token_ms/adv:0.003953840772522615 - perf/total_num_tokens:127436 - perf/time_per_step:31.86980871669948 - perf/throughput:499.8304238849445 (TaskRunner pid=2122883) Training Progress: 60%|██████ | 60/100 [50:07<38:08, 57.22s/it] (TaskRunner pid=2122883) step:61 - global_seqlen/min:11855 - global_seqlen/max:23118 - global_seqlen/minmax_diff:11263 - global_seqlen/balanced_min:16706 - global_seqlen/balanced_max:16857 - global_seqlen/mean:16734.875 - actor/entropy:0.23181812465190887 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7169718742370605 - training/rollout_probs_diff_mean:0.004951599985361099 - training/rollout_probs_diff_std:0.01399071142077446 - training/rollout_actor_probs_pearson_corr:0.99810391664505 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.4296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0000967979431152 - rollout_corr/rollout_is_ratio_fraction_high:5.225843415246345e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012193634756840765 - rollout_corr/rollout_is_max:3.395094156265259 - rollout_corr/rollout_is_min:0.23580829799175262 - rollout_corr/rollout_is_std:0.04068733751773834 - rollout_corr/rollout_is_eff_sample_size:0.9983476330593353 - rollout_corr/rollout_is_seq_mean:1.0001016855239868 - rollout_corr/rollout_is_seq_std:0.0031705843284726143 - rollout_corr/rollout_is_seq_max:1.012374997138977 - rollout_corr/rollout_is_seq_min:0.9923515915870667 - rollout_corr/rollout_is_seq_max_deviation:0.01237499713897705 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2725287731736898) - rollout_corr/training_log_ppl:np.float64(0.2377089764631819) - rollout_corr/kl:np.float64(0.0013348886182598108) - rollout_corr/k3_kl:np.float64(0.001518071228900908) - rollout_corr/rollout_ppl:np.float64(1.2708931332454085) - rollout_corr/rollout_log_ppl:np.float64(0.23637408643844537) - rollout_corr/log_ppl_diff:np.float64(0.0013348900247365236) - rollout_corr/log_ppl_abs_diff:np.float64(0.003386163560207933) - rollout_corr/log_ppl_diff_max:np.float64(0.022667646408081055) - rollout_corr/log_ppl_diff_min:np.float64(-0.016180813312530518) - rollout_corr/ppl_ratio:np.float64(1.0013469324912876) - rollout_corr/chi2_token:np.float64(0.003425400471314788) - rollout_corr/chi2_seq:np.float64(1.5097244810312986) - actor/pg_loss:np.float64(0.006231680745258927) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015820763669580629) - actor/ppo_kl:np.float64(0.00055076309819313) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4296875) - self_distillation/token_reweight_w_mean:np.float64(578582.0100038159) - self_distillation/token_reweight_w_std:np.float64(6712461.685952961) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001360923051834) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11931510880822316) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.691587895154953) - perf/mfu/actor:np.float64(0.02863750714432481) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.93616485595703) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:4 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005421812646090984 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005421812646090984 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:224.24609375 - response_length/max:1217.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:224.24609375 - response_length_non_aborted/max:1217.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6639550924301147e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.926751097664237) - timing_s/agent_loop/generate_sequences/max:np.float64(7.615216936916113) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.241809466169798) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.615216936916113) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1217 - timing_s/gen:13.372081184759736 - timing_s/reward:0.05007507652044296 - timing_s/old_log_prob:2.441918570548296 - timing_s/adv:0.5068533644080162 - timing_s/update_actor:16.665765838697553 - timing_s/step:33.135390520095825 - timing_s/stop_profile:0.0001241099089384079 - timing_per_token_ms/gen:0.2329346801741902 - timing_per_token_ms/update_actor:0.12448379386384387 - timing_per_token_ms/adv:0.003785906411072806 - perf/total_num_tokens:133879 - perf/time_per_step:33.135390520095825 - perf/throughput:505.04535293919946 (TaskRunner pid=2122883) Training Progress: 61%|██████ | 61/100 [50:41<32:30, 50.01s/it] (TaskRunner pid=2122883) step:62 - global_seqlen/min:11541 - global_seqlen/max:23017 - global_seqlen/minmax_diff:11476 - global_seqlen/balanced_min:17683 - global_seqlen/balanced_max:17705 - global_seqlen/mean:17697.125 - actor/entropy:0.19811317324638367 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5048303008079529 - training/rollout_probs_diff_mean:0.004269558470696211 - training/rollout_probs_diff_std:0.012749578803777695 - training/rollout_actor_probs_pearson_corr:0.9981906414031982 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.51171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:1.0000804662704468 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.53727144910954e-05 - rollout_corr/rollout_is_max:1.944337010383606 - rollout_corr/rollout_is_min:0.35625290870666504 - rollout_corr/rollout_is_std:0.03687412664294243 - rollout_corr/rollout_is_eff_sample_size:0.9986423828918339 - rollout_corr/rollout_is_seq_mean:1.0000014305114746 - rollout_corr/rollout_is_seq_std:0.0029181051068007946 - rollout_corr/rollout_is_seq_max:1.008512258529663 - rollout_corr/rollout_is_seq_min:0.9900585412979126 - rollout_corr/rollout_is_seq_max_deviation:0.009941458702087402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2438335889019072) - rollout_corr/training_log_ppl:np.float64(0.2145167674170807) - rollout_corr/kl:np.float64(0.0009783608772053753) - rollout_corr/k3_kl:np.float64(0.0011070482911037516) - rollout_corr/rollout_ppl:np.float64(1.242590891662985) - rollout_corr/rollout_log_ppl:np.float64(0.21353840544179548) - rollout_corr/log_ppl_diff:np.float64(0.0009783619752852246) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027773703768616542) - rollout_corr/log_ppl_diff_max:np.float64(0.011119604110717773) - rollout_corr/log_ppl_diff_min:np.float64(-0.014483973383903503) - rollout_corr/ppl_ratio:np.float64(1.0009853248484433) - rollout_corr/chi2_token:np.float64(0.0024234983138740063) - rollout_corr/chi2_seq:np.float64(0.7688207423780113) - actor/pg_loss:np.float64(0.0037221149541437626) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010598614235277637) - actor/ppo_kl:np.float64(0.00016833341987876338) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51171875) - self_distillation/token_reweight_w_mean:np.float64(444199.96810927615) - self_distillation/token_reweight_w_std:np.float64(4653540.987991009) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000051069073379) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07645721273729578) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.5245166197419167) - perf/mfu/actor:np.float64(0.030100797751914456) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8307876586914) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:4 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006728899199515581 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006728899199515581 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:259.12890625 - response_length/max:1326.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:259.12890625 - response_length_non_aborted/max:1326.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.90625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013496167957782745 - timing_s/agent_loop/generate_sequences/min:np.float64(1.061777088791132) - timing_s/agent_loop/generate_sequences/max:np.float64(8.351498238742352) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.499284286684997) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.351498238742352) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:1326 - timing_s/gen:14.470805488526821 - timing_s/reward:0.05649738386273384 - timing_s/old_log_prob:2.302702836692333 - timing_s/adv:0.5540448240935802 - timing_s/update_actor:16.689813977107406 - timing_s/step:34.16248192638159 - timing_s/stop_profile:0.00013281218707561493 - timing_per_token_ms/gen:0.21814078852716917 - timing_per_token_ms/update_actor:0.11788506591541992 - timing_per_token_ms/adv:0.003913381581002425 - perf/total_num_tokens:141577 - perf/time_per_step:34.16248192638159 - perf/throughput:518.0280823312663 (TaskRunner pid=2122883) Training Progress: 62%|██████▏ | 62/100 [51:15<28:40, 45.27s/it] (TaskRunner pid=2122883) step:63 - global_seqlen/min:14930 - global_seqlen/max:23507 - global_seqlen/minmax_diff:8577 - global_seqlen/balanced_min:19130 - global_seqlen/balanced_max:19152 - global_seqlen/mean:19145.25 - actor/entropy:0.21696506440639496 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49148619174957275 - training/rollout_probs_diff_mean:0.004545337986201048 - training/rollout_probs_diff_std:0.012920488603413105 - training/rollout_actor_probs_pearson_corr:0.9983041882514954 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000380277633667 - rollout_corr/rollout_is_ratio_fraction_high:1.4841639313090127e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.936655725236051e-05 - rollout_corr/rollout_is_max:2.3148374557495117 - rollout_corr/rollout_is_min:0.14844170212745667 - rollout_corr/rollout_is_std:0.03836194425821304 - rollout_corr/rollout_is_eff_sample_size:0.9985305238584241 - rollout_corr/rollout_is_seq_mean:1.000018835067749 - rollout_corr/rollout_is_seq_std:0.00292221549898386 - rollout_corr/rollout_is_seq_max:1.0082601308822632 - rollout_corr/rollout_is_seq_min:0.9906377792358398 - rollout_corr/rollout_is_seq_max_deviation:0.009362220764160156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2752052140422165) - rollout_corr/training_log_ppl:np.float64(0.23854760386166163) - rollout_corr/kl:np.float64(0.0010151628624388565) - rollout_corr/k3_kl:np.float64(0.0012473572794249321) - rollout_corr/rollout_ppl:np.float64(1.2738490118645132) - rollout_corr/rollout_log_ppl:np.float64(0.23753243846294936) - rollout_corr/log_ppl_diff:np.float64(0.00101516539871227) - rollout_corr/log_ppl_abs_diff:np.float64(0.002944408348412253) - rollout_corr/log_ppl_diff_max:np.float64(0.019116565585136414) - rollout_corr/log_ppl_diff_min:np.float64(-0.01233530044555664) - rollout_corr/ppl_ratio:np.float64(1.0010242883581668) - rollout_corr/chi2_token:np.float64(0.0029916237108409405) - rollout_corr/chi2_seq:np.float64(0.7363117965869606) - actor/pg_loss:np.float64(0.0054677382577210665) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017338892685074825) - actor/ppo_kl:np.float64(0.00015598068775091178) - actor/pg_clipfrac_lower:np.float64(1.3706140634894837e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(478414.26378723653) - self_distillation/token_reweight_w_std:np.float64(5065154.901935935) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991468349471688) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12116794451139867) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6787499040365219) - perf/mfu/actor:np.float64(0.03271239188344616) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.80411529541016) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:4 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003213214920833707 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003213214920833707 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:263.1953125 - response_length/max:1198.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.1953125 - response_length_non_aborted/max:1198.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:335.09375 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001463424414396286 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1881604846566916) - timing_s/agent_loop/generate_sequences/max:np.float64(7.47804525308311) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5504021373853902) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.47804525308311) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1198 - timing_s/gen:13.364666033536196 - timing_s/reward:0.055124446749687195 - timing_s/old_log_prob:2.247605225071311 - timing_s/adv:0.5140605103224516 - timing_s/update_actor:16.550543075427413 - timing_s/step:32.820558559149504 - timing_s/stop_profile:0.00011437758803367615 - timing_per_token_ms/gen:0.19835355803876928 - timing_per_token_ms/update_actor:0.10805906866864766 - timing_per_token_ms/adv:0.00335631886709792 - perf/total_num_tokens:153162 - perf/time_per_step:32.820558559149504 - perf/throughput:583.3310230079193 (TaskRunner pid=2122883) Training Progress: 63%|██████▎ | 63/100 [51:48<25:37, 41.55s/it] (TaskRunner pid=2122883) step:64 - global_seqlen/min:16199 - global_seqlen/max:26881 - global_seqlen/minmax_diff:10682 - global_seqlen/balanced_min:21490 - global_seqlen/balanced_max:21502 - global_seqlen/mean:21497.75 - actor/entropy:0.21458002924919128 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3945135474205017 - training/rollout_probs_diff_mean:0.0047240303829312325 - training/rollout_probs_diff_std:0.01327318325638771 - training/rollout_actor_probs_pearson_corr:0.9981720447540283 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:1.0000271797180176 - rollout_corr/rollout_is_ratio_fraction_high:5.3534622566075996e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010706924513215199 - rollout_corr/rollout_is_max:6.235465049743652 - rollout_corr/rollout_is_min:0.25675758719444275 - rollout_corr/rollout_is_std:0.04021582379937172 - rollout_corr/rollout_is_eff_sample_size:0.9983852990445926 - rollout_corr/rollout_is_seq_mean:1.000148892402649 - rollout_corr/rollout_is_seq_std:0.0032988900784403086 - rollout_corr/rollout_is_seq_max:1.0242486000061035 - rollout_corr/rollout_is_seq_min:0.9885872006416321 - rollout_corr/rollout_is_seq_max_deviation:0.024248600006103516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.268503988161683) - rollout_corr/training_log_ppl:np.float64(0.2344126374082407) - rollout_corr/kl:np.float64(0.0013722575135601822) - rollout_corr/k3_kl:np.float64(0.0013257689138868045) - rollout_corr/rollout_ppl:np.float64(1.266730026807636) - rollout_corr/rollout_log_ppl:np.float64(0.23304037815250922) - rollout_corr/log_ppl_diff:np.float64(0.0013722592557314783) - rollout_corr/log_ppl_abs_diff:np.float64(0.002840419387212023) - rollout_corr/log_ppl_diff_max:np.float64(0.015095949172973633) - rollout_corr/log_ppl_diff_min:np.float64(-0.010050714015960693) - rollout_corr/ppl_ratio:np.float64(1.0013799467124045) - rollout_corr/chi2_token:np.float64(0.00259414897300303) - rollout_corr/chi2_seq:np.float64(0.5995087241753936) - actor/pg_loss:np.float64(0.007810348994098604) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017692785711460601) - actor/ppo_kl:np.float64(0.0005061967070645323) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(587268.0798286705) - self_distillation/token_reweight_w_std:np.float64(7362155.60160229) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012550125829875) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1439109018247109) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.7157184034585953) - perf/mfu/actor:np.float64(0.036492575998587316) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.8278350830078) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:4 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02531183883547783 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02531183883547783 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.8671875 - response_length/max:1070.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.8671875 - response_length_non_aborted/max:1070.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.9375 - prompt_length/max:499.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.023211896419525e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.191472725942731) - timing_s/agent_loop/generate_sequences/max:np.float64(7.287185866385698) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9950591388187604) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.287185866385698) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:1070 - timing_s/gen:13.135733580216765 - timing_s/reward:0.06018144264817238 - timing_s/old_log_prob:2.2917927093803883 - timing_s/adv:0.5165592208504677 - timing_s/update_actor:16.707958148792386 - timing_s/step:32.80109519511461 - timing_s/stop_profile:0.00012754648923873901 - timing_per_token_ms/gen:0.17580413796162592 - timing_per_token_ms/update_actor:0.09714945836652897 - timing_per_token_ms/adv:0.0030035656106480193 - perf/total_num_tokens:171982 - perf/time_per_step:32.80109519511461 - perf/throughput:655.3973235382052 (TaskRunner pid=2122883) Training Progress: 64%|██████▍ | 64/100 [52:20<23:21, 38.94s/it] (TaskRunner pid=2122883) step:65 - global_seqlen/min:13134 - global_seqlen/max:22137 - global_seqlen/minmax_diff:9003 - global_seqlen/balanced_min:17936 - global_seqlen/balanced_max:18093 - global_seqlen/mean:17977.125 - actor/entropy:0.21280328929424286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.407850444316864 - training/rollout_probs_diff_mean:0.004871057812124491 - training/rollout_probs_diff_std:0.013702865689992905 - training/rollout_actor_probs_pearson_corr:0.9980465769767761 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.5234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:1.0000638961791992 - rollout_corr/rollout_is_ratio_fraction_high:1.5778594388393685e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012622875510714948 - rollout_corr/rollout_is_max:2.3150124549865723 - rollout_corr/rollout_is_min:0.2893199920654297 - rollout_corr/rollout_is_std:0.041086770594120026 - rollout_corr/rollout_is_eff_sample_size:0.9983149598221633 - rollout_corr/rollout_is_seq_mean:1.0001049041748047 - rollout_corr/rollout_is_seq_std:0.0032439520582556725 - rollout_corr/rollout_is_seq_max:1.013520359992981 - rollout_corr/rollout_is_seq_min:0.9887478947639465 - rollout_corr/rollout_is_seq_max_deviation:0.013520359992980957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.256556692533195) - rollout_corr/training_log_ppl:np.float64(0.22500694784685038) - rollout_corr/kl:np.float64(0.001361132893192174) - rollout_corr/k3_kl:np.float64(0.0013893124849744254) - rollout_corr/rollout_ppl:np.float64(1.25478261616081) - rollout_corr/rollout_log_ppl:np.float64(0.22364581374858972) - rollout_corr/log_ppl_diff:np.float64(0.0013611340982606634) - rollout_corr/log_ppl_abs_diff:np.float64(0.003076999870245345) - rollout_corr/log_ppl_diff_max:np.float64(0.015015125274658203) - rollout_corr/log_ppl_diff_min:np.float64(-0.010976061224937439) - rollout_corr/ppl_ratio:np.float64(1.001369426259771) - rollout_corr/chi2_token:np.float64(0.0027954219840466976) - rollout_corr/chi2_seq:np.float64(0.498077409574762) - actor/pg_loss:np.float64(0.0047087157145142555) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.002266843486495418) - actor/ppo_kl:np.float64(0.0005259367158743089) - actor/pg_clipfrac_lower:np.float64(2.0451570890145376e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5234375) - self_distillation/token_reweight_w_mean:np.float64(575063.0486043207) - self_distillation/token_reweight_w_std:np.float64(6029142.028431675) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999974900856614) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1101971018651966) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.6160394102334976) - perf/mfu/actor:np.float64(0.03089012319613719) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.76618576049805) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:4 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02519841492176056 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02519841492176056 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:247.56640625 - response_length/max:1273.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:247.56640625 - response_length_non_aborted/max:1273.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.21875 - prompt_length/max:498.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014909543097019196 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2503177151083946) - timing_s/agent_loop/generate_sequences/max:np.float64(7.673106657341123) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.481383962018299) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.673106657341123) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:1218 - timing_s/gen:13.46312802657485 - timing_s/reward:0.06146645545959473 - timing_s/old_log_prob:2.3153125289827585 - timing_s/adv:0.553697656840086 - timing_s/update_actor:16.608258245512843 - timing_s/step:33.08947719074786 - timing_s/stop_profile:0.00012203864753246307 - timing_per_token_ms/gen:0.21242924131111995 - timing_per_token_ms/update_actor:0.11548188493372023 - timing_per_token_ms/adv:0.0038500153447790317 - perf/total_num_tokens:143817 - perf/time_per_step:33.08947719074786 - perf/throughput:543.2882754952254 (TaskRunner pid=2122883) Training Progress: 65%|██████▌ | 65/100 [52:54<21:41, 37.20s/it] (TaskRunner pid=2122883) step:66 - global_seqlen/min:12651 - global_seqlen/max:23559 - global_seqlen/minmax_diff:10908 - global_seqlen/balanced_min:16718 - global_seqlen/balanced_max:16741 - global_seqlen/mean:16734.625 - actor/entropy:0.19700400531291962 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3154793083667755 - training/rollout_probs_diff_mean:0.00472779106348753 - training/rollout_probs_diff_std:0.013478272594511509 - training/rollout_actor_probs_pearson_corr:0.9979892373085022 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.0000770092010498 - rollout_corr/rollout_is_ratio_fraction_high:3.4337710530962795e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.867542106192559e-05 - rollout_corr/rollout_is_max:2.0585169792175293 - rollout_corr/rollout_is_min:0.3337665796279907 - rollout_corr/rollout_is_std:0.04114910215139389 - rollout_corr/rollout_is_eff_sample_size:0.9983098511093312 - rollout_corr/rollout_is_seq_mean:1.0000991821289062 - rollout_corr/rollout_is_seq_std:0.0033235810697078705 - rollout_corr/rollout_is_seq_max:1.010388731956482 - rollout_corr/rollout_is_seq_min:0.9896685481071472 - rollout_corr/rollout_is_seq_max_deviation:0.010388731956481934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.230994427576661) - rollout_corr/training_log_ppl:np.float64(0.2043733007449191) - rollout_corr/kl:np.float64(0.0012404392970850608) - rollout_corr/k3_kl:np.float64(0.0012966901686013443) - rollout_corr/rollout_ppl:np.float64(1.229399942792952) - rollout_corr/rollout_log_ppl:np.float64(0.20313286200689618) - rollout_corr/log_ppl_diff:np.float64(0.001240438738022931) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031714067881694064) - rollout_corr/log_ppl_diff_max:np.float64(0.016345471143722534) - rollout_corr/log_ppl_diff_min:np.float64(-0.008899062871932983) - rollout_corr/ppl_ratio:np.float64(1.0012497496791184) - rollout_corr/chi2_token:np.float64(0.0026735947467386723) - rollout_corr/chi2_seq:np.float64(1.3038442207034677) - actor/pg_loss:np.float64(0.004394229501485825) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015602343514729) - actor/ppo_kl:np.float64(0.0004241243286600138) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(540545.633164251) - self_distillation/token_reweight_w_std:np.float64(5716813.33122617) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003452564124018) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07450340743525885) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.5204346254467964) - perf/mfu/actor:np.float64(0.028829480161249834) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7582664489746) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:4 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005770881660282612 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.005770881660282612 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:227.51953125 - response_length/max:1157.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.51953125 - response_length_non_aborted/max:1157.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.4375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.591877460479736e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6880578771233559) - timing_s/agent_loop/generate_sequences/max:np.float64(5.9939240626990795) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.0637885364340036) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.9939240626990795) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:435 - timing_s/agent_loop/slowest/response_length:1157 - timing_s/gen:11.854472966864705 - timing_s/reward:0.05460910499095917 - timing_s/old_log_prob:2.2059535086154938 - timing_s/adv:0.5360113754868507 - timing_s/update_actor:16.550181152299047 - timing_s/step:31.287991248071194 - timing_s/stop_profile:3.69735062122345e-05 - timing_per_token_ms/gen:0.20352773571748142 - timing_per_token_ms/update_actor:0.12362228875982466 - timing_per_token_ms/adv:0.004003759984813305 - perf/total_num_tokens:133877 - perf/time_per_step:31.287991248071194 - perf/throughput:534.8577627536774 (TaskRunner pid=2122883) Training Progress: 66%|██████▌ | 66/100 [53:25<20:04, 35.43s/it] (TaskRunner pid=2122883) step:67 - global_seqlen/min:14763 - global_seqlen/max:22066 - global_seqlen/minmax_diff:7303 - global_seqlen/balanced_min:18043 - global_seqlen/balanced_max:18191 - global_seqlen/mean:18069.125 - actor/entropy:0.21443374454975128 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3140978515148163 - training/rollout_probs_diff_mean:0.0051235631108284 - training/rollout_probs_diff_std:0.013876632787287235 - training/rollout_actor_probs_pearson_corr:0.9979842305183411 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:1.0000861883163452 - rollout_corr/rollout_is_ratio_fraction_high:1.738979153742548e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.2169376431265846e-05 - rollout_corr/rollout_is_max:2.3516223430633545 - rollout_corr/rollout_is_min:0.3772035539150238 - rollout_corr/rollout_is_std:0.042313169687986374 - rollout_corr/rollout_is_eff_sample_size:0.9982130330660876 - rollout_corr/rollout_is_seq_mean:1.0001331567764282 - rollout_corr/rollout_is_seq_std:0.003116749692708254 - rollout_corr/rollout_is_seq_max:1.0102242231369019 - rollout_corr/rollout_is_seq_min:0.9922493100166321 - rollout_corr/rollout_is_seq_max_deviation:0.010224223136901855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2491217982023954) - rollout_corr/training_log_ppl:np.float64(0.21858850728312973) - rollout_corr/kl:np.float64(0.0013292867849905576) - rollout_corr/k3_kl:np.float64(0.0013373246251262572) - rollout_corr/rollout_ppl:np.float64(1.247418281622231) - rollout_corr/rollout_log_ppl:np.float64(0.21725921976030804) - rollout_corr/log_ppl_diff:np.float64(0.0013292875228216872) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031616015621693805) - rollout_corr/log_ppl_diff_max:np.float64(0.029362037777900696) - rollout_corr/log_ppl_diff_min:np.float64(-0.00867721438407898) - rollout_corr/ppl_ratio:np.float64(1.0013392963446677) - rollout_corr/chi2_token:np.float64(0.0027240454219281673) - rollout_corr/chi2_seq:np.float64(0.4304043233860284) - actor/pg_loss:np.float64(0.003972164820879698) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017918323310368578) - actor/ppo_kl:np.float64(0.0005243421803386994) - actor/pg_clipfrac_lower:np.float64(1.6344141840818338e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(706441.2849378223) - self_distillation/token_reweight_w_std:np.float64(6269580.666031567) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0021861053537577) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09675114441779442) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.5632684454321861) - perf/mfu/actor:np.float64(0.031125756559966692) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7952880859375) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:4 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004525693599134684 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004525693599134684 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:224.62890625 - response_length/max:1078.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:224.62890625 - response_length_non_aborted/max:1078.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:340.03125 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.785880446434021e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1052221357822418) - timing_s/agent_loop/generate_sequences/max:np.float64(6.942589655518532) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4225184994356823) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.942589655518532) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1078 - timing_s/gen:12.853008408099413 - timing_s/reward:0.05389631725847721 - timing_s/old_log_prob:2.25601764023304 - timing_s/adv:0.5043847281485796 - timing_s/update_actor:16.623241163790226 - timing_s/step:32.37739512138069 - timing_s/stop_profile:0.00012384913861751556 - timing_per_token_ms/gen:0.22351114525866295 - timing_per_token_ms/update_actor:0.11499755220431417 - timing_per_token_ms/adv:0.0034892719497248733 - perf/total_num_tokens:144553 - perf/time_per_step:32.37739512138069 - perf/throughput:558.0784041538876 (TaskRunner pid=2122883) Training Progress: 67%|██████▋ | 67/100 [53:57<18:59, 34.53s/it] (TaskRunner pid=2122883) step:68 - global_seqlen/min:9746 - global_seqlen/max:20472 - global_seqlen/minmax_diff:10726 - global_seqlen/balanced_min:14882 - global_seqlen/balanced_max:14892 - global_seqlen/mean:14888.375 - actor/entropy:0.19889450073242188 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27853599190711975 - training/rollout_probs_diff_mean:0.0050133708864450455 - training/rollout_probs_diff_std:0.013850436545908451 - training/rollout_actor_probs_pearson_corr:0.9978574514389038 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.0000970363616943 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.141498983604833e-05 - rollout_corr/rollout_is_max:1.8664900064468384 - rollout_corr/rollout_is_min:0.3767696022987366 - rollout_corr/rollout_is_std:0.04154696688055992 - rollout_corr/rollout_is_eff_sample_size:0.9982771803450756 - rollout_corr/rollout_is_seq_mean:1.0001392364501953 - rollout_corr/rollout_is_seq_std:0.003491344628855586 - rollout_corr/rollout_is_seq_max:1.0124340057373047 - rollout_corr/rollout_is_seq_min:0.9904762506484985 - rollout_corr/rollout_is_seq_max_deviation:0.012434005737304688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2253266391344368) - rollout_corr/training_log_ppl:np.float64(0.19990111159859225) - rollout_corr/kl:np.float64(0.0012003272438523283) - rollout_corr/k3_kl:np.float64(0.0011535957297610366) - rollout_corr/rollout_ppl:np.float64(1.2238059635274112) - rollout_corr/rollout_log_ppl:np.float64(0.19870078393432777) - rollout_corr/log_ppl_diff:np.float64(0.0012003276642644778) - rollout_corr/log_ppl_abs_diff:np.float64(0.003384005030966364) - rollout_corr/log_ppl_diff_max:np.float64(0.01639336347579956) - rollout_corr/log_ppl_diff_min:np.float64(-0.012207046151161194) - rollout_corr/ppl_ratio:np.float64(1.0012101489119232) - rollout_corr/chi2_token:np.float64(0.0021951296366751194) - rollout_corr/chi2_seq:np.float64(0.5914839308243245) - actor/pg_loss:np.float64(0.004780508461408317) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013941395154688507) - actor/ppo_kl:np.float64(0.00043155668406047454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(868609.1069036205) - self_distillation/token_reweight_w_std:np.float64(7623714.479851294) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011716361623257) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09727005992317572) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.7055970579385757) - perf/mfu/actor:np.float64(0.02586255147692835) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.74933242797852) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:4 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005760110914707184 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005760110914707184 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:191.91796875 - response_length/max:889.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:191.91796875 - response_length_non_aborted/max:889.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.34375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014156848192214966 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0743454471230507) - timing_s/agent_loop/generate_sequences/max:np.float64(5.634837629273534) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.979868456000986) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.634837629273534) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:889 - timing_s/gen:11.382458966225386 - timing_s/reward:0.04163476265966892 - timing_s/old_log_prob:2.2224585078656673 - timing_s/adv:0.5000829771161079 - timing_s/update_actor:16.325130246579647 - timing_s/step:30.537147894501686 - timing_s/stop_profile:0.00011890381574630737 - timing_per_token_ms/gen:0.23167570304340204 - timing_per_token_ms/update_actor:0.13706272718295018 - timing_per_token_ms/adv:0.004198602744726237 - perf/total_num_tokens:119107 - perf/time_per_step:30.537147894501686 - perf/throughput:487.54962485153044 (TaskRunner pid=2122883) Training Progress: 68%|██████▊ | 68/100 [54:28<17:47, 33.35s/it] (TaskRunner pid=2122883) step:69 - global_seqlen/min:15468 - global_seqlen/max:22188 - global_seqlen/minmax_diff:6720 - global_seqlen/balanced_min:18204 - global_seqlen/balanced_max:18567 - global_seqlen/mean:18252.5 - actor/entropy:0.19695989787578583 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3735440969467163 - training/rollout_probs_diff_mean:0.004728264175355434 - training/rollout_probs_diff_std:0.013597971759736538 - training/rollout_actor_probs_pearson_corr:0.9979573488235474 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58984375 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58984375 - rollout_corr/rollout_is_mean:1.000138759613037 - rollout_corr/rollout_is_ratio_fraction_high:3.193867814843543e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5969339074217714e-05 - rollout_corr/rollout_is_max:2.512885093688965 - rollout_corr/rollout_is_min:0.2698124945163727 - rollout_corr/rollout_is_std:0.040441952645778656 - rollout_corr/rollout_is_eff_sample_size:0.9983677133097597 - rollout_corr/rollout_is_seq_mean:1.0000832080841064 - rollout_corr/rollout_is_seq_std:0.0029280909802764654 - rollout_corr/rollout_is_seq_max:1.0104328393936157 - rollout_corr/rollout_is_seq_min:0.9879934787750244 - rollout_corr/rollout_is_seq_max_deviation:0.012006521224975586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2370114037767053) - rollout_corr/training_log_ppl:np.float64(0.2087440811592387) - rollout_corr/kl:np.float64(0.0012408357466497932) - rollout_corr/k3_kl:np.float64(0.0011961429782729738) - rollout_corr/rollout_ppl:np.float64(1.2353982599452138) - rollout_corr/rollout_log_ppl:np.float64(0.20750324439723045) - rollout_corr/log_ppl_diff:np.float64(0.0012408367620082572) - rollout_corr/log_ppl_abs_diff:np.float64(0.002828049662639387) - rollout_corr/log_ppl_diff_max:np.float64(0.013582497835159302) - rollout_corr/log_ppl_diff_min:np.float64(-0.010901011526584625) - rollout_corr/ppl_ratio:np.float64(1.001248066779226) - rollout_corr/chi2_token:np.float64(0.002271374687552452) - rollout_corr/chi2_seq:np.float64(0.7177229449152946) - actor/pg_loss:np.float64(0.004629563074558973) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001182094197247352) - actor/ppo_kl:np.float64(0.0004323655259614867) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(164067.67598723504) - self_distillation/token_reweight_w_std:np.float64(2016421.0568843854) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9979195562191308) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08755953633226454) - self_distillation/empty_target_batch:np.float64(0.41015625) - actor/grad_norm:np.float64(0.5471128672361374) - perf/mfu/actor:np.float64(0.03103007215439442) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.71195602416992) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:4 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02166440524160862 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02166440524160862 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:244.609375 - response_length/max:1402.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:244.609375 - response_length_non_aborted/max:1402.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:325.78125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011947937309741974 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1571588460355997) - timing_s/agent_loop/generate_sequences/max:np.float64(8.519991947337985) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.450770689632918) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.519991947337985) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:1402 - timing_s/gen:14.34707204811275 - timing_s/reward:0.05564185231924057 - timing_s/old_log_prob:2.2638629749417305 - timing_s/adv:0.49436523206532 - timing_s/update_actor:16.777688885107636 - timing_s/step:34.02742971852422 - timing_s/stop_profile:0.00012484006583690643 - timing_per_token_ms/gen:0.2291132553195904 - timing_per_token_ms/update_actor:0.11489993757778137 - timing_per_token_ms/adv:0.0033855994525771814 - perf/total_num_tokens:146020 - perf/time_per_step:34.02742971852422 - perf/throughput:536.40548672013 (TaskRunner pid=2122883) Training Progress: 69%|██████▉ | 69/100 [55:02<17:20, 33.56s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques, which are not provided in the question. Since no specific method or data is given, the score cannot be calculated directly from the sequence alone. However, the question provides four possible scores, and without additional context, the most reasonable assumption is that the score is a typical value within the range of possible stability scores. Among the options, 1.69 is a plausible value for a folding stability score, as it is within the typical range of stability scores for proteins. Therefore, the most reasonable answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2122883) step:70 - global_seqlen/min:10998 - global_seqlen/max:22982 - global_seqlen/minmax_diff:11984 - global_seqlen/balanced_min:17336 - global_seqlen/balanced_max:17343 - global_seqlen/mean:17340.25 - actor/entropy:0.199662446975708 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34822899103164673 - training/rollout_probs_diff_mean:0.004988952539861202 - training/rollout_probs_diff_std:0.014087354764342308 - training/rollout_actor_probs_pearson_corr:0.9978179335594177 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999821662902832 - rollout_corr/rollout_is_ratio_fraction_high:1.793078627088107e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.793078627088107e-05 - rollout_corr/rollout_is_max:2.048713207244873 - rollout_corr/rollout_is_min:0.19275815784931183 - rollout_corr/rollout_is_std:0.04096545651555061 - rollout_corr/rollout_is_eff_sample_size:0.9983241081129971 - rollout_corr/rollout_is_seq_mean:0.9997748732566833 - rollout_corr/rollout_is_seq_std:0.00317583535797894 - rollout_corr/rollout_is_seq_max:1.0109431743621826 - rollout_corr/rollout_is_seq_min:0.9912058115005493 - rollout_corr/rollout_is_seq_max_deviation:0.010943174362182617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.235567000694573) - rollout_corr/training_log_ppl:np.float64(0.20803631153830793) - rollout_corr/kl:np.float64(0.0014276731483562344) - rollout_corr/k3_kl:np.float64(0.0011787888146841397) - rollout_corr/rollout_ppl:np.float64(1.233756847679615) - rollout_corr/rollout_log_ppl:np.float64(0.20660863688681275) - rollout_corr/log_ppl_diff:np.float64(0.001427674651495181) - rollout_corr/log_ppl_abs_diff:np.float64(0.003025784491910599) - rollout_corr/log_ppl_diff_max:np.float64(0.014226198196411133) - rollout_corr/log_ppl_diff_min:np.float64(-0.008730649948120117) - rollout_corr/ppl_ratio:np.float64(1.0014357783365995) - rollout_corr/chi2_token:np.float64(0.0017687426880002022) - rollout_corr/chi2_seq:np.float64(0.5918401211965829) - actor/pg_loss:np.float64(0.0053908772533759475) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016089424170786515) - actor/ppo_kl:np.float64(0.0003056624431962973) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(589973.9683744449) - self_distillation/token_reweight_w_std:np.float64(5563161.509655834) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001278449082747) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10386245278641582) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.6108624935150146) - perf/mfu/actor:np.float64(0.029783237475419166) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.7671127319336) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.42) - val-aux/sciknoweval/reward/std@16:np.float64(0.23113610685340358) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.51528) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21100947357072017) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.32514000000000004) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18040222263814762) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.41974000000000006) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.22844473537910776) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.59892) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16675057279932926) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.25292000000000003) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11798816134028049) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.41842) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18877181836586193) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.6667599999999999) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1245056095938742) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.20698) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06619895954904491) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.41896000000000005) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13423860559458853) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7202199999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07823132973605224) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.18576) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.026280958267470154) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.42282) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.09698912402724233) - val-aux/sciknoweval/score/mean@16:np.float64(0.42) - val-aux/sciknoweval/score/std@16:np.float64(0.23113610685340358) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.51528) - val-aux/sciknoweval/score/best@2/std:np.float64(0.21100947357072017) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.32514000000000004) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18040222263814762) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.41974000000000006) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.22844473537910776) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.59892) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16675057279932926) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.25292000000000003) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11798816134028049) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.41842) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18877181836586193) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.6667599999999999) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1245056095938742) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.20698) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06619895954904491) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.41896000000000005) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13423860559458853) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7202199999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07823132973605224) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.18576) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.026280958267470154) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.42282) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.09698912402724233) - val-core/sciknoweval/acc/mean@16:np.float64(0.42) - val-aux/sciknoweval/acc/std@16:np.float64(0.23113610685340358) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.51528) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.21100947357072017) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.32514000000000004) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18040222263814762) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.41974000000000006) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.22844473537910776) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.59892) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16675057279932926) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.25292000000000003) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11798816134028049) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.41842) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18877181836586193) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.6667599999999999) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1245056095938742) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.20698) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06619895954904491) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.41896000000000005) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13423860559458853) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7202199999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07823132973605224) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.18576) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.026280958267470154) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.42282) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.09698912402724233) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99875) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.004841229182759271) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99982) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.001888809148643663) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9976999999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.006380438856379709) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9986799999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.004965641952456903) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.99564) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.008257747876994065) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.002431049156228644) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.99176) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.00984390166549829) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9868600000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.009494229826584146) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:4 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01323067955672741 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01323067955672741 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:217.8515625 - response_length/max:860.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:217.8515625 - response_length_non_aborted/max:860.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:324.03125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.492171764373779e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1253771558403969) - timing_s/agent_loop/generate_sequences/max:np.float64(5.687919491901994) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.208337127987761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.687919491901994) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:860 - timing_s/gen:11.549281872808933 - timing_s/reward:0.05496336705982685 - timing_s/old_log_prob:2.313216272741556 - timing_s/adv:0.5179897584021091 - timing_s/update_actor:16.506855254992843 - timing_s/step:31.032124953344464 - timing_s/testing:66.01478779688478 - timing_s/save_checkpoint:6.830698998644948 - timing_s/stop_profile:9.629130363464355e-05 - timing_per_token_ms/gen:0.2070877151301584 - timing_per_token_ms/update_actor:0.11899233903052756 - timing_per_token_ms/adv:0.003734013050576759 - perf/total_num_tokens:138722 - perf/time_per_step:31.032124953344464 - perf/throughput:558.7838417791356 (TaskRunner pid=2122883) Training Progress: 70%|███████ | 70/100 [56:46<27:20, 54.67s/it] (TaskRunner pid=2122883) step:71 - global_seqlen/min:9804 - global_seqlen/max:21681 - global_seqlen/minmax_diff:11877 - global_seqlen/balanced_min:17495 - global_seqlen/balanced_max:17500 - global_seqlen/mean:17498.375 - actor/entropy:0.19690172374248505 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3381648659706116 - training/rollout_probs_diff_mean:0.004845378454774618 - training/rollout_probs_diff_std:0.013365901075303555 - training/rollout_actor_probs_pearson_corr:0.9980552792549133 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:1.0001564025878906 - rollout_corr/rollout_is_ratio_fraction_high:3.4841385058825836e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.968277011765167e-05 - rollout_corr/rollout_is_max:3.150454521179199 - rollout_corr/rollout_is_min:0.3398284912109375 - rollout_corr/rollout_is_std:0.04114330932497978 - rollout_corr/rollout_is_eff_sample_size:0.9983106827566944 - rollout_corr/rollout_is_seq_mean:1.0002278089523315 - rollout_corr/rollout_is_seq_std:0.003083531279116869 - rollout_corr/rollout_is_seq_max:1.010101556777954 - rollout_corr/rollout_is_seq_min:0.9893363118171692 - rollout_corr/rollout_is_seq_max_deviation:0.01066368818283081 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2388491262681782) - rollout_corr/training_log_ppl:np.float64(0.2105677341169212) - rollout_corr/kl:np.float64(0.0012890833140133395) - rollout_corr/k3_kl:np.float64(0.0013068942296285968) - rollout_corr/rollout_ppl:np.float64(1.2372192768380046) - rollout_corr/rollout_log_ppl:np.float64(0.2092786498251371) - rollout_corr/log_ppl_diff:np.float64(0.0012890842917840928) - rollout_corr/log_ppl_abs_diff:np.float64(0.003031075408216566) - rollout_corr/log_ppl_diff_max:np.float64(0.01689492166042328) - rollout_corr/log_ppl_diff_min:np.float64(-0.01494649052619934) - rollout_corr/ppl_ratio:np.float64(1.0012982161715627) - rollout_corr/chi2_token:np.float64(0.002663188148289919) - rollout_corr/chi2_seq:np.float64(0.42635189229622483) - actor/pg_loss:np.float64(0.006174354115501046) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00172065351762285) - actor/ppo_kl:np.float64(0.0006120751639642208) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(469061.6761071717) - self_distillation/token_reweight_w_std:np.float64(4943925.522530494) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003047524951398) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08837462472729385) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.6535552144050598) - perf/mfu/actor:np.float64(0.030004138816374676) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(143.94376373291016) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:5 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0025042244233191013 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0025042244233191013 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:224.23046875 - response_length/max:788.0 - response_length/min:95.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:224.23046875 - response_length_non_aborted/max:788.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:322.59375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002894662320613861 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0912248101085424) - timing_s/agent_loop/generate_sequences/max:np.float64(5.649241955950856) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.27755354999681) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.649241955950856) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:788 - timing_s/gen:11.400874027982354 - timing_s/reward:0.06556447967886925 - timing_s/old_log_prob:2.415134770795703 - timing_s/adv:0.5144570916891098 - timing_s/update_actor:16.434071181342006 - timing_s/step:30.942065130919218 - timing_s/stop_profile:5.480460822582245e-05 - timing_per_token_ms/gen:0.19861111837329676 - timing_per_token_ms/update_actor:0.11739712388537511 - timing_per_token_ms/adv:0.003675034765293276 - perf/total_num_tokens:139987 - perf/time_per_step:30.942065130919218 - perf/throughput:565.5205923057329 (TaskRunner pid=2122883) Training Progress: 71%|███████ | 71/100 [57:17<23:02, 47.68s/it] (TaskRunner pid=2122883) step:72 - global_seqlen/min:13771 - global_seqlen/max:21717 - global_seqlen/minmax_diff:7946 - global_seqlen/balanced_min:18281 - global_seqlen/balanced_max:18449 - global_seqlen/mean:18338.0 - actor/entropy:0.19176998734474182 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37767118215560913 - training/rollout_probs_diff_mean:0.004621072206646204 - training/rollout_probs_diff_std:0.01323121227324009 - training/rollout_actor_probs_pearson_corr:0.9980506896972656 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:1.0001182556152344 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.93550061644055e-05 - rollout_corr/rollout_is_max:1.8492262363433838 - rollout_corr/rollout_is_min:0.13725660741329193 - rollout_corr/rollout_is_std:0.03996754437685013 - rollout_corr/rollout_is_eff_sample_size:0.9984054996396693 - rollout_corr/rollout_is_seq_mean:1.0001261234283447 - rollout_corr/rollout_is_seq_std:0.0031107664108276367 - rollout_corr/rollout_is_seq_max:1.010321021080017 - rollout_corr/rollout_is_seq_min:0.9898220896720886 - rollout_corr/rollout_is_seq_max_deviation:0.01032102108001709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.239236116874963) - rollout_corr/training_log_ppl:np.float64(0.2116152676026104) - rollout_corr/kl:np.float64(0.0011325802346102698) - rollout_corr/k3_kl:np.float64(0.001299673892788178) - rollout_corr/rollout_ppl:np.float64(1.2377676661126316) - rollout_corr/rollout_log_ppl:np.float64(0.21048268360027578) - rollout_corr/log_ppl_diff:np.float64(0.0011325840023346245) - rollout_corr/log_ppl_abs_diff:np.float64(0.003151193872326985) - rollout_corr/log_ppl_diff_max:np.float64(0.0144386887550354) - rollout_corr/log_ppl_diff_min:np.float64(-0.009770482778549194) - rollout_corr/ppl_ratio:np.float64(1.0011412110179663) - rollout_corr/chi2_token:np.float64(0.0029832087457180023) - rollout_corr/chi2_seq:np.float64(0.8706808432471007) - actor/pg_loss:np.float64(0.0043286209693178535) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016793236786725174) - actor/ppo_kl:np.float64(0.0003311673048500552) - actor/pg_clipfrac_lower:np.float64(3.354357795615215e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(500642.83659131406) - self_distillation/token_reweight_w_std:np.float64(5350166.238907624) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019122541416436) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09215450899500865) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.5711423642933369) - perf/mfu/actor:np.float64(0.031247951307443515) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.2710838317871) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:5 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018253633752465248 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018253633752465248 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:246.125 - response_length/max:1318.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:246.125 - response_length_non_aborted/max:1318.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:326.9375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010432861745357513 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9296897631138563) - timing_s/agent_loop/generate_sequences/max:np.float64(8.103419492021203) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3991422778126434) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.103419492021203) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:1318 - timing_s/gen:13.928170634433627 - timing_s/reward:0.4275554921478033 - timing_s/old_log_prob:2.5062245801091194 - timing_s/adv:0.598910516127944 - timing_s/update_actor:16.280453188344836 - timing_s/step:33.830064464360476 - timing_s/stop_profile:0.00017435476183891296 - timing_per_token_ms/gen:0.2210540032128242 - timing_per_token_ms/update_actor:0.11097484177898922 - timing_per_token_ms/adv:0.004082441624822391 - perf/total_num_tokens:146704 - perf/time_per_step:33.830064464360476 - perf/throughput:542.0622245434631 (TaskRunner pid=2122883) Training Progress: 72%|███████▏ | 72/100 [57:51<20:19, 43.55s/it] (TaskRunner pid=2122883) step:73 - global_seqlen/min:10239 - global_seqlen/max:17583 - global_seqlen/minmax_diff:7344 - global_seqlen/balanced_min:14427 - global_seqlen/balanced_max:14431 - global_seqlen/mean:14428.5 - actor/entropy:0.19701342284679413 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3973785638809204 - training/rollout_probs_diff_mean:0.004991715308278799 - training/rollout_probs_diff_std:0.013931449502706528 - training/rollout_actor_probs_pearson_corr:0.9978450536727905 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000314712524414 - rollout_corr/rollout_is_ratio_fraction_high:4.2892683268291876e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00019301706925034523 - rollout_corr/rollout_is_max:2.332307815551758 - rollout_corr/rollout_is_min:0.33360418677330017 - rollout_corr/rollout_is_std:0.0416143424808979 - rollout_corr/rollout_is_eff_sample_size:0.9982712404358611 - rollout_corr/rollout_is_seq_mean:1.0000600814819336 - rollout_corr/rollout_is_seq_std:0.0033438869286328554 - rollout_corr/rollout_is_seq_max:1.0109838247299194 - rollout_corr/rollout_is_seq_min:0.9896968007087708 - rollout_corr/rollout_is_seq_max_deviation:0.010983824729919434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2266183192841709) - rollout_corr/training_log_ppl:np.float64(0.20162912894738838) - rollout_corr/kl:np.float64(0.0010475357565766785) - rollout_corr/k3_kl:np.float64(0.0011981667771010507) - rollout_corr/rollout_ppl:np.float64(1.2252924079075456) - rollout_corr/rollout_log_ppl:np.float64(0.20058159226027783) - rollout_corr/log_ppl_diff:np.float64(0.0010475366871105507) - rollout_corr/log_ppl_abs_diff:np.float64(0.003063296157051809) - rollout_corr/log_ppl_diff_max:np.float64(0.01853451132774353) - rollout_corr/log_ppl_diff_min:np.float64(-0.009390532970428467) - rollout_corr/ppl_ratio:np.float64(1.0010553821921349) - rollout_corr/chi2_token:np.float64(0.0026172325015068054) - rollout_corr/chi2_seq:np.float64(0.2984912055544555) - actor/pg_loss:np.float64(0.005422829883173108) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018017217225860804) - actor/ppo_kl:np.float64(0.00022023160384776475) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(578319.4778499373) - self_distillation/token_reweight_w_std:np.float64(6073846.084833732) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982847378123552) - self_distillation/token_reweight_w_clip_frac:np.float64(0.116363640059717) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.679996520280838) - perf/mfu/actor:np.float64(0.02498840296640397) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.2462501525879) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:5 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005806596949696541 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005806596949696541 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:182.140625 - response_length/max:722.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:182.140625 - response_length_non_aborted/max:722.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014832429587841034 - timing_s/agent_loop/generate_sequences/min:np.float64(1.21829747967422) - timing_s/agent_loop/generate_sequences/max:np.float64(4.604781422764063) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.9372474718547892) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.604781422764063) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:722 - timing_s/gen:11.13407732360065 - timing_s/reward:0.05069064348936081 - timing_s/old_log_prob:2.2318159490823746 - timing_s/adv:0.49003932252526283 - timing_s/update_actor:15.927147168666124 - timing_s/step:29.924155635759234 - timing_s/stop_profile:0.0001366306096315384 - timing_per_token_ms/gen:0.23878522183238934 - timing_per_token_ms/update_actor:0.13798339370573973 - timing_per_token_ms/adv:0.004245411187279194 - perf/total_num_tokens:115428 - perf/time_per_step:29.924155635759234 - perf/throughput:482.1689933585964 (TaskRunner pid=2122883) Training Progress: 73%|███████▎ | 73/100 [58:21<17:45, 39.47s/it] (TaskRunner pid=2122883) step:74 - global_seqlen/min:12820 - global_seqlen/max:22635 - global_seqlen/minmax_diff:9815 - global_seqlen/balanced_min:16510 - global_seqlen/balanced_max:16523 - global_seqlen/mean:16520.75 - actor/entropy:0.1998564451932907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40099841356277466 - training/rollout_probs_diff_mean:0.005098035093396902 - training/rollout_probs_diff_std:0.014329318888485432 - training/rollout_actor_probs_pearson_corr:0.997765839099884 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:1.0000135898590088 - rollout_corr/rollout_is_ratio_fraction_high:1.871047425083816e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.484189700335264e-05 - rollout_corr/rollout_is_max:2.0350987911224365 - rollout_corr/rollout_is_min:0.29247722029685974 - rollout_corr/rollout_is_std:0.04207443445920944 - rollout_corr/rollout_is_eff_sample_size:0.9982329891138849 - rollout_corr/rollout_is_seq_mean:1.000016212463379 - rollout_corr/rollout_is_seq_std:0.0031045295763760805 - rollout_corr/rollout_is_seq_max:1.0087541341781616 - rollout_corr/rollout_is_seq_min:0.9893158674240112 - rollout_corr/rollout_is_seq_max_deviation:0.01068413257598877 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2355217128060758) - rollout_corr/training_log_ppl:np.float64(0.20864850541693158) - rollout_corr/kl:np.float64(0.0012986420607177251) - rollout_corr/k3_kl:np.float64(0.001190482557717587) - rollout_corr/rollout_ppl:np.float64(1.233883251901716) - rollout_corr/rollout_log_ppl:np.float64(0.20734986252500676) - rollout_corr/log_ppl_diff:np.float64(0.0012986428919248283) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032230869110208005) - rollout_corr/log_ppl_diff_max:np.float64(0.015039652585983276) - rollout_corr/log_ppl_diff_min:np.float64(-0.009414374828338623) - rollout_corr/ppl_ratio:np.float64(1.0013071070425212) - rollout_corr/chi2_token:np.float64(0.002133794594556093) - rollout_corr/chi2_seq:np.float64(0.6196947358548641) - actor/pg_loss:np.float64(0.004897095495834947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011714055831362202) - actor/ppo_kl:np.float64(0.00039855819784406776) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(341515.478634842) - self_distillation/token_reweight_w_std:np.float64(3745655.63619092) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992651597131044) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08639141090679914) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.5267659574747086) - perf/mfu/actor:np.float64(0.02834496970975933) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.1980857849121) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:5 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009532986208796501 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009532986208796501 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:208.7734375 - response_length/max:963.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:208.7734375 - response_length_non_aborted/max:963.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.5 - prompt_length/max:498.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.530914783477783e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0088061578571796) - timing_s/agent_loop/generate_sequences/max:np.float64(6.282911816611886) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.159521869587479) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.282911816611886) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:963 - timing_s/gen:12.101818397641182 - timing_s/reward:0.05504104122519493 - timing_s/old_log_prob:2.226160455495119 - timing_s/adv:0.4680979307740927 - timing_s/update_actor:16.286116683855653 - timing_s/step:31.225198931992054 - timing_s/stop_profile:0.00012497790157794952 - timing_per_token_ms/gen:0.2264307599753243 - timing_per_token_ms/update_actor:0.12322470744257716 - timing_per_token_ms/adv:0.003541742435831399 - perf/total_num_tokens:132166 - perf/time_per_step:31.225198931992054 - perf/throughput:529.0838990644033 (TaskRunner pid=2122883) Training Progress: 74%|███████▍ | 74/100 [58:52<16:02, 37.01s/it] (TaskRunner pid=2122883) step:75 - global_seqlen/min:12126 - global_seqlen/max:21222 - global_seqlen/minmax_diff:9096 - global_seqlen/balanced_min:18203 - global_seqlen/balanced_max:18392 - global_seqlen/mean:18236.375 - actor/entropy:0.2161518633365631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3713438808917999 - training/rollout_probs_diff_mean:0.005161375738680363 - training/rollout_probs_diff_std:0.013850989751517773 - training/rollout_actor_probs_pearson_corr:0.9980016350746155 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.999866247177124 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.835853855591267e-05 - rollout_corr/rollout_is_max:1.6655349731445312 - rollout_corr/rollout_is_min:0.44153663516044617 - rollout_corr/rollout_is_std:0.04145936295390129 - rollout_corr/rollout_is_eff_sample_size:0.9982835955264238 - rollout_corr/rollout_is_seq_mean:0.9998180270195007 - rollout_corr/rollout_is_seq_std:0.002778091700747609 - rollout_corr/rollout_is_seq_max:1.0106350183486938 - rollout_corr/rollout_is_seq_min:0.9903151392936707 - rollout_corr/rollout_is_seq_max_deviation:0.010635018348693848 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2521987650543451) - rollout_corr/training_log_ppl:np.float64(0.2221425136376638) - rollout_corr/kl:np.float64(0.001645305610473713) - rollout_corr/k3_kl:np.float64(0.0012188858009380965) - rollout_corr/rollout_ppl:np.float64(1.2501694383099675) - rollout_corr/rollout_log_ppl:np.float64(0.2204972050822107) - rollout_corr/log_ppl_diff:np.float64(0.0016453085554530844) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030864700238453224) - rollout_corr/log_ppl_diff_max:np.float64(0.012482985854148865) - rollout_corr/log_ppl_diff_min:np.float64(-0.006725788116455078) - rollout_corr/ppl_ratio:np.float64(1.0016529564745724) - rollout_corr/chi2_token:np.float64(0.0015822243876755238) - rollout_corr/chi2_seq:np.float64(0.3376803044229746) - actor/pg_loss:np.float64(0.004625871777534485) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014200286204868462) - actor/ppo_kl:np.float64(0.0005539787339348123) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.484375) - self_distillation/token_reweight_w_mean:np.float64(701245.2609014655) - self_distillation/token_reweight_w_std:np.float64(7589330.247221366) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001619327813387) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10732259399082977) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.6458503305912018) - perf/mfu/actor:np.float64(0.0315188448474388) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.17803955078125) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:5 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006162949837744236 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006162949837744236 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:228.57421875 - response_length/max:1278.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:228.57421875 - response_length_non_aborted/max:1278.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:341.3125 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015654414892196655 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3382554687559605) - timing_s/agent_loop/generate_sequences/max:np.float64(7.776741812005639) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4855196749194874) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.776741812005639) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:1278 - timing_s/gen:13.650750419124961 - timing_s/reward:0.04579509235918522 - timing_s/old_log_prob:2.2064243238419294 - timing_s/adv:0.47517586685717106 - timing_s/update_actor:16.09972064755857 - timing_s/step:32.5119928792119 - timing_s/stop_profile:3.0023977160453796e-05 - timing_per_token_ms/gen:0.2332863439994012 - timing_per_token_ms/update_actor:0.11035444713901865 - timing_per_token_ms/adv:0.0032570608663808668 - perf/total_num_tokens:145891 - perf/time_per_step:32.5119928792119 - perf/throughput:560.9122476051075 (TaskRunner pid=2122883) Training Progress: 75%|███████▌ | 75/100 [59:25<14:51, 35.67s/it] (TaskRunner pid=2122883) step:76 - global_seqlen/min:11565 - global_seqlen/max:20025 - global_seqlen/minmax_diff:8460 - global_seqlen/balanced_min:15676 - global_seqlen/balanced_max:15680 - global_seqlen/mean:15677.5 - actor/entropy:0.20906737446784973 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4323614835739136 - training/rollout_probs_diff_mean:0.005196167156100273 - training/rollout_probs_diff_std:0.014064192771911621 - training/rollout_actor_probs_pearson_corr:0.9979039430618286 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.64453125 - self_distillation/correct_sample_fraction:0.46484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.64453125 - rollout_corr/rollout_is_mean:0.9995588064193726 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.81677445047535e-05 - rollout_corr/rollout_is_max:1.5994808673858643 - rollout_corr/rollout_is_min:0.36089852452278137 - rollout_corr/rollout_is_std:0.041405413299798965 - rollout_corr/rollout_is_eff_sample_size:0.9982870407504366 - rollout_corr/rollout_is_seq_mean:0.9995691776275635 - rollout_corr/rollout_is_seq_std:0.003029718529433012 - rollout_corr/rollout_is_seq_max:1.0080506801605225 - rollout_corr/rollout_is_seq_min:0.9909569621086121 - rollout_corr/rollout_is_seq_max_deviation:0.00904303789138794 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2378678624518216) - rollout_corr/training_log_ppl:np.float64(0.2106911562732421) - rollout_corr/kl:np.float64(0.001780368542952715) - rollout_corr/k3_kl:np.float64(0.001284653402535696) - rollout_corr/rollout_ppl:np.float64(1.2356635369360447) - rollout_corr/rollout_log_ppl:np.float64(0.20891078899148852) - rollout_corr/log_ppl_diff:np.float64(0.0017803672817535698) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032782099151518196) - rollout_corr/log_ppl_diff_max:np.float64(0.015327438712120056) - rollout_corr/log_ppl_diff_min:np.float64(-0.010779500007629395) - rollout_corr/ppl_ratio:np.float64(1.0017898133955896) - rollout_corr/chi2_token:np.float64(0.001633466687053442) - rollout_corr/chi2_seq:np.float64(0.331364804180339) - actor/pg_loss:np.float64(0.0036399068776518106) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001190648112242343) - actor/ppo_kl:np.float64(0.00045568517619187787) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.64453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.46484375) - self_distillation/token_reweight_w_mean:np.float64(496084.83509531594) - self_distillation/token_reweight_w_std:np.float64(5315430.144841015) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001681232592091) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07428947099833749) - self_distillation/empty_target_batch:np.float64(0.35546875) - actor/grad_norm:np.float64(0.5466546416282654) - perf/mfu/actor:np.float64(0.026923698700232014) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.2565155029297) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:5 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004592355340719223 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004592355340719223 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:199.890625 - response_length/max:674.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:199.890625 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.03125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.350844442844391e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1626148615032434) - timing_s/agent_loop/generate_sequences/max:np.float64(4.779665656387806) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1255838525030413) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.779665656387806) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:412 - timing_s/agent_loop/slowest/response_length:674 - timing_s/gen:10.666637863963842 - timing_s/reward:0.053339336067438126 - timing_s/old_log_prob:2.288154808804393 - timing_s/adv:0.4929979033768177 - timing_s/update_actor:16.330744432285428 - timing_s/step:29.92172957211733 - timing_s/stop_profile:0.00012400373816490173 - timing_per_token_ms/gen:0.2084467651052107 - timing_per_token_ms/update_actor:0.13020845504931772 - timing_per_token_ms/adv:0.003930775820258473 - perf/total_num_tokens:125420 - perf/time_per_step:29.92172957211733 - perf/throughput:523.9503272099997 (TaskRunner pid=2122883) Training Progress: 76%|███████▌ | 76/100 [59:55<13:34, 33.96s/it] (TaskRunner pid=2122883) step:77 - global_seqlen/min:15406 - global_seqlen/max:21723 - global_seqlen/minmax_diff:6317 - global_seqlen/balanced_min:17270 - global_seqlen/balanced_max:17277 - global_seqlen/mean:17272.75 - actor/entropy:0.22195269167423248 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3425203263759613 - training/rollout_probs_diff_mean:0.005554586183279753 - training/rollout_probs_diff_std:0.014829994179308414 - training/rollout_actor_probs_pearson_corr:0.9977590441703796 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9998087286949158 - rollout_corr/rollout_is_ratio_fraction_high:3.536693111527711e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.073386223055422e-05 - rollout_corr/rollout_is_max:6.2831339836120605 - rollout_corr/rollout_is_min:0.22866636514663696 - rollout_corr/rollout_is_std:0.043667059391736984 - rollout_corr/rollout_is_eff_sample_size:0.9980959261529178 - rollout_corr/rollout_is_seq_mean:0.9998674392700195 - rollout_corr/rollout_is_seq_std:0.0035912925377488136 - rollout_corr/rollout_is_seq_max:1.0285019874572754 - rollout_corr/rollout_is_seq_min:0.9909583330154419 - rollout_corr/rollout_is_seq_max_deviation:0.02850198745727539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2535131475888193) - rollout_corr/training_log_ppl:np.float64(0.2237606028211303) - rollout_corr/kl:np.float64(0.0013143475887176237) - rollout_corr/k3_kl:np.float64(0.0014206709299742215) - rollout_corr/rollout_ppl:np.float64(1.2518676673062146) - rollout_corr/rollout_log_ppl:np.float64(0.22244625524035655) - rollout_corr/log_ppl_diff:np.float64(0.001314347580773756) - rollout_corr/log_ppl_abs_diff:np.float64(0.002847455471055582) - rollout_corr/log_ppl_diff_max:np.float64(0.017824441194534302) - rollout_corr/log_ppl_diff_min:np.float64(-0.01252920925617218) - rollout_corr/ppl_ratio:np.float64(1.00132129737176) - rollout_corr/chi2_token:np.float64(0.004566720454022288) - rollout_corr/chi2_seq:np.float64(1.2924350867979228) - actor/pg_loss:np.float64(0.0070024499436840415) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012036074376737815) - actor/ppo_kl:np.float64(0.00015001226471511586) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(294809.86716741277) - self_distillation/token_reweight_w_std:np.float64(3251414.243952886) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996227624360472) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10142341829487123) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.8022593855857849) - perf/mfu/actor:np.float64(0.029566345858852098) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.2310447692871) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:5 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00545755960047245 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00545755960047245 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:220.8984375 - response_length/max:607.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:220.8984375 - response_length_non_aborted/max:607.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:318.875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.929388761520386e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.813449727371335) - timing_s/agent_loop/generate_sequences/max:np.float64(4.553269684314728) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.197502375282056) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.553269684314728) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:607 - timing_s/gen:10.332580428570509 - timing_s/reward:0.05444280430674553 - timing_s/old_log_prob:2.2673390675336123 - timing_s/adv:0.5042030327022076 - timing_s/update_actor:16.172887090593576 - timing_s/step:29.42025664076209 - timing_s/stop_profile:0.00011752359569072723 - timing_per_token_ms/gen:0.1827158342806456 - timing_per_token_ms/update_actor:0.11704047626024791 - timing_per_token_ms/adv:0.003648832935564745 - perf/total_num_tokens:138182 - perf/time_per_step:29.42025664076209 - perf/throughput:587.1039879396706 (TaskRunner pid=2122883) Training Progress: 77%|███████▋ | 77/100 [1:00:24<12:30, 32.61s/it] (TaskRunner pid=2122883) step:78 - global_seqlen/min:14370 - global_seqlen/max:23360 - global_seqlen/minmax_diff:8990 - global_seqlen/balanced_min:18531 - global_seqlen/balanced_max:18674 - global_seqlen/mean:18562.25 - actor/entropy:0.20804251730442047 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41408851742744446 - training/rollout_probs_diff_mean:0.004874464590102434 - training/rollout_probs_diff_std:0.013253279961645603 - training/rollout_actor_probs_pearson_corr:0.9981226921081543 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000364780426025 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.655927114072256e-05 - rollout_corr/rollout_is_max:1.8679466247558594 - rollout_corr/rollout_is_min:0.29214736819267273 - rollout_corr/rollout_is_std:0.04069319739937782 - rollout_corr/rollout_is_eff_sample_size:0.9983468013504078 - rollout_corr/rollout_is_seq_mean:1.0000871419906616 - rollout_corr/rollout_is_seq_std:0.002924885367974639 - rollout_corr/rollout_is_seq_max:1.0078153610229492 - rollout_corr/rollout_is_seq_min:0.9887160658836365 - rollout_corr/rollout_is_seq_max_deviation:0.011283934116363525 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2604153575375676) - rollout_corr/training_log_ppl:np.float64(0.2286913382849889) - rollout_corr/kl:np.float64(0.00165782988653973) - rollout_corr/k3_kl:np.float64(0.0014850372907062592) - rollout_corr/rollout_ppl:np.float64(1.258268583100289) - rollout_corr/rollout_log_ppl:np.float64(0.22703350719530135) - rollout_corr/log_ppl_diff:np.float64(0.0016578310896875337) - rollout_corr/log_ppl_abs_diff:np.float64(0.003049075006856583) - rollout_corr/log_ppl_diff_max:np.float64(0.027070119976997375) - rollout_corr/log_ppl_diff_min:np.float64(-0.00748780369758606) - rollout_corr/ppl_ratio:np.float64(1.0016673954669386) - rollout_corr/chi2_token:np.float64(0.0024911961518228054) - rollout_corr/chi2_seq:np.float64(0.23507609497755766) - actor/pg_loss:np.float64(0.0077421864261850715) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020995262079850363) - actor/ppo_kl:np.float64(0.0008263197890130414) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(516053.4604536833) - self_distillation/token_reweight_w_std:np.float64(6114605.236582575) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00137659907341) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12383084223256446) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.6829441338777542) - perf/mfu/actor:np.float64(0.03151336998012594) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.21386337280273) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:5 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009750287048518658 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009750287048518658 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:251.6953125 - response_length/max:1327.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:251.6953125 - response_length_non_aborted/max:1327.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:328.375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.014535367488861e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2290885895490646) - timing_s/agent_loop/generate_sequences/max:np.float64(8.05762130022049) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.65355857443501) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.05762130022049) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:1327 - timing_s/gen:13.922783764079213 - timing_s/reward:0.055119847878813744 - timing_s/old_log_prob:2.244201695546508 - timing_s/adv:0.5187073163688183 - timing_s/update_actor:16.548424838110805 - timing_s/step:33.38240977190435 - timing_s/stop_profile:0.00013656727969646454 - timing_per_token_ms/gen:0.2160782159120839 - timing_per_token_ms/update_actor:0.11143870515502434 - timing_per_token_ms/adv:0.003493025605522083 - perf/total_num_tokens:148498 - perf/time_per_step:33.38240977190435 - perf/throughput:556.0488331079847 (TaskRunner pid=2122883) Training Progress: 78%|███████▊ | 78/100 [1:00:58<12:02, 32.86s/it] (TaskRunner pid=2122883) step:79 - global_seqlen/min:12701 - global_seqlen/max:19610 - global_seqlen/minmax_diff:6909 - global_seqlen/balanced_min:17271 - global_seqlen/balanced_max:17283 - global_seqlen/mean:17280.5 - actor/entropy:0.21525056660175323 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7200708985328674 - training/rollout_probs_diff_mean:0.005195227451622486 - training/rollout_probs_diff_std:0.014740017242729664 - training/rollout_actor_probs_pearson_corr:0.9977768659591675 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999945342540741 - rollout_corr/rollout_is_ratio_fraction_high:1.6934227460296825e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015240804350469261 - rollout_corr/rollout_is_max:3.029350996017456 - rollout_corr/rollout_is_min:0.015028837136924267 - rollout_corr/rollout_is_std:0.04251272976398468 - rollout_corr/rollout_is_eff_sample_size:0.9981956909582614 - rollout_corr/rollout_is_seq_mean:0.9999911785125732 - rollout_corr/rollout_is_seq_std:0.0029612616635859013 - rollout_corr/rollout_is_seq_max:1.0107840299606323 - rollout_corr/rollout_is_seq_min:0.9903897047042847 - rollout_corr/rollout_is_seq_max_deviation:0.010784029960632324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2490549199283123) - rollout_corr/training_log_ppl:np.float64(0.2196736648620572) - rollout_corr/kl:np.float64(0.001692611791760612) - rollout_corr/k3_kl:np.float64(0.0015163807295834886) - rollout_corr/rollout_ppl:np.float64(1.246921275742352) - rollout_corr/rollout_log_ppl:np.float64(0.21798105229390785) - rollout_corr/log_ppl_diff:np.float64(0.0016926125681493431) - rollout_corr/log_ppl_abs_diff:np.float64(0.003021725657163188) - rollout_corr/log_ppl_diff_max:np.float64(0.033561140298843384) - rollout_corr/log_ppl_diff_min:np.float64(-0.007312938570976257) - rollout_corr/ppl_ratio:np.float64(1.00170242250897) - rollout_corr/chi2_token:np.float64(0.002288506831973791) - rollout_corr/chi2_seq:np.float64(0.5073147353250533) - actor/pg_loss:np.float64(0.006525568896904588) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014381463884092227) - actor/ppo_kl:np.float64(0.0006511699603919396) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(377157.56747266906) - self_distillation/token_reweight_w_std:np.float64(4524650.8311734535) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991592520382255) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11779780086362734) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.6401597559452057) - perf/mfu/actor:np.float64(0.0296049883148389) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.26931762695312) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:5 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018056120024994016 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018056120024994016 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:230.671875 - response_length/max:890.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:230.671875 - response_length_non_aborted/max:890.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.34375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012810342013835907 - timing_s/agent_loop/generate_sequences/min:np.float64(1.271697724238038) - timing_s/agent_loop/generate_sequences/max:np.float64(5.964426325634122) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.5344342128373682) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.964426325634122) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:890 - timing_s/gen:11.82465054281056 - timing_s/reward:0.06565968319773674 - timing_s/old_log_prob:2.223105376586318 - timing_s/adv:0.4894870724529028 - timing_s/update_actor:16.2752229552716 - timing_s/step:30.97196850553155 - timing_s/stop_profile:0.00010983459651470184 - timing_per_token_ms/gen:0.20024132193339023 - timing_per_token_ms/update_actor:0.11772824104678396 - timing_per_token_ms/adv:0.003540747319615338 - perf/total_num_tokens:138244 - perf/time_per_step:30.97196850553155 - perf/throughput:557.939996513742 (TaskRunner pid=2122883) Training Progress: 79%|███████▉ | 79/100 [1:01:29<11:18, 32.30s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques, which are not provided in the question. Without access to specific computational tools or experimental data, it is not possible to calculate the exact folding stability score for the given sequence. However, the question provides four possible scores: -0.01, 1.69, 2.49, and 0.45. Among these, 1.69 is a reasonable value for a folding stability score, as it is positive and within a typical range for such scores. Therefore, the most plausible answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2122883) step:80 - global_seqlen/min:12832 - global_seqlen/max:21001 - global_seqlen/minmax_diff:8169 - global_seqlen/balanced_min:16760 - global_seqlen/balanced_max:16771 - global_seqlen/mean:16769.125 - actor/entropy:0.21624459326267242 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6118733286857605 - training/rollout_probs_diff_mean:0.005380045156925917 - training/rollout_probs_diff_std:0.014546380378305912 - training/rollout_actor_probs_pearson_corr:0.9978134632110596 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999927282333374 - rollout_corr/rollout_is_ratio_fraction_high:3.5480494261719286e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.096098852343857e-05 - rollout_corr/rollout_is_max:3.9595537185668945 - rollout_corr/rollout_is_min:0.3052869737148285 - rollout_corr/rollout_is_std:0.04341449216008186 - rollout_corr/rollout_is_eff_sample_size:0.9981184902564971 - rollout_corr/rollout_is_seq_mean:0.9999736547470093 - rollout_corr/rollout_is_seq_std:0.003248573513701558 - rollout_corr/rollout_is_seq_max:1.0156736373901367 - rollout_corr/rollout_is_seq_min:0.990421712398529 - rollout_corr/rollout_is_seq_max_deviation:0.01567363739013672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2467931918799877) - rollout_corr/training_log_ppl:np.float64(0.2184930287185125) - rollout_corr/kl:np.float64(0.0014139293027497501) - rollout_corr/k3_kl:np.float64(0.0013873711505993924) - rollout_corr/rollout_ppl:np.float64(1.2450102255679667) - rollout_corr/rollout_log_ppl:np.float64(0.2170790982490871) - rollout_corr/log_ppl_diff:np.float64(0.0014139304694253951) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031045038776937872) - rollout_corr/log_ppl_diff_max:np.float64(0.015252634882926941) - rollout_corr/log_ppl_diff_min:np.float64(-0.009716227650642395) - rollout_corr/ppl_ratio:np.float64(1.001422454835847) - rollout_corr/chi2_token:np.float64(0.0026412673760205507) - rollout_corr/chi2_seq:np.float64(0.6153503498062491) - actor/pg_loss:np.float64(0.005080795381218195) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020866218960691185) - actor/ppo_kl:np.float64(0.00041786666335319467) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(607829.0349954877) - self_distillation/token_reweight_w_std:np.float64(6253481.655585986) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0039331214502454) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09547229323652573) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.5971960201859474) - perf/mfu/actor:np.float64(0.028664539556949087) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.34428787231445) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.41875) - val-aux/sciknoweval/reward/std@16:np.float64(0.2211101008128732) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5120999999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.21421579131747706) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.32600000000000007) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15530304468803183) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.41967999999999994) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.22011101001975603) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.60526) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.18121537035652593) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.27272) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09037255406946425) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.40104) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.18007103708696343) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.68068) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1255793418054099) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.24458) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.05154055880198174) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.38406000000000007) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.13114827437959176) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7286199999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06724812379449635) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.22526) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03666116598309839) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.37333999999999995) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.10244541245169461) - val-aux/sciknoweval/score/mean@16:np.float64(0.41875) - val-aux/sciknoweval/score/std@16:np.float64(0.2211101008128732) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5120999999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.21421579131747706) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.32600000000000007) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15530304468803183) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.41967999999999994) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.22011101001975603) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.60526) - val-aux/sciknoweval/score/best@4/std:np.float64(0.18121537035652593) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.27272) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09037255406946425) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.40104) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.18007103708696343) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.68068) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1255793418054099) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.24458) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.05154055880198174) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.38406000000000007) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.13114827437959176) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7286199999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06724812379449635) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.22526) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03666116598309839) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.37333999999999995) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.10244541245169461) - val-core/sciknoweval/acc/mean@16:np.float64(0.41875) - val-aux/sciknoweval/acc/std@16:np.float64(0.2211101008128732) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5120999999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.21421579131747706) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.32600000000000007) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15530304468803183) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.41967999999999994) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.22011101001975603) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.60526) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.18121537035652593) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.27272) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09037255406946425) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.40104) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.18007103708696343) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.68068) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1255793418054099) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.24458) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.05154055880198174) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.38406000000000007) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.13114827437959176) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7286199999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06724812379449635) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.22526) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03666116598309839) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.37333999999999995) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.10244541245169461) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9975) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.009682458365518542) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9998600000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0023042059141141357) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.99542) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.01273201501325385) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.99774) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.009234474228192518) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.99092) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.016755431405841622) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99928) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.00529873114696611) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.98448) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.019491610038772544) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.99994) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0015256715413187116) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9750200000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.019365214541818408) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:5 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009311412461102009 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.009311412461102009 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:220.19140625 - response_length/max:872.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:220.19140625 - response_length_non_aborted/max:872.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.84375 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014083273708820343 - timing_s/agent_loop/generate_sequences/min:np.float64(1.258571656420827) - timing_s/agent_loop/generate_sequences/max:np.float64(5.787867229431868) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4260267040735926) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.787867229431868) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:414 - timing_s/agent_loop/slowest/response_length:872 - timing_s/gen:11.63143178448081 - timing_s/reward:0.053980035707354546 - timing_s/old_log_prob:2.230306738987565 - timing_s/adv:0.5261808633804321 - timing_s/update_actor:16.25501248985529 - timing_s/step:30.786901891231537 - timing_s/testing:70.0108205396682 - timing_s/save_checkpoint:7.13819195330143 - timing_s/stop_profile:8.743070065975189e-05 - timing_per_token_ms/gen:0.20634447629868918 - timing_per_token_ms/update_actor:0.12116771514506042 - timing_per_token_ms/adv:0.003922244477428251 - perf/total_num_tokens:134153 - perf/time_per_step:30.786901891231537 - perf/throughput:544.6837443807894 (TaskRunner pid=2122883) Training Progress: 80%|████████ | 80/100 [1:03:17<18:20, 55.01s/it] (TaskRunner pid=2122883) step:81 - global_seqlen/min:15445 - global_seqlen/max:22337 - global_seqlen/minmax_diff:6892 - global_seqlen/balanced_min:18825 - global_seqlen/balanced_max:18829 - global_seqlen/mean:18827.5 - actor/entropy:0.20360738039016724 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4294052720069885 - training/rollout_probs_diff_mean:0.005188526585698128 - training/rollout_probs_diff_std:0.014588381163775921 - training/rollout_actor_probs_pearson_corr:0.9977055191993713 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0000635385513306 - rollout_corr/rollout_is_ratio_fraction_high:6.641429354203865e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.3207146771019325e-05 - rollout_corr/rollout_is_max:2.8543355464935303 - rollout_corr/rollout_is_min:0.44702544808387756 - rollout_corr/rollout_is_std:0.0434611439704895 - rollout_corr/rollout_is_eff_sample_size:0.9981148086752116 - rollout_corr/rollout_is_seq_mean:1.000145435333252 - rollout_corr/rollout_is_seq_std:0.0028170603327453136 - rollout_corr/rollout_is_seq_max:1.0175964832305908 - rollout_corr/rollout_is_seq_min:0.9926503300666809 - rollout_corr/rollout_is_seq_max_deviation:0.01759648323059082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2413170905783772) - rollout_corr/training_log_ppl:np.float64(0.21387588272045832) - rollout_corr/kl:np.float64(0.0007690461568188312) - rollout_corr/k3_kl:np.float64(0.0012659859104928728) - rollout_corr/rollout_ppl:np.float64(1.2403291380032897) - rollout_corr/rollout_log_ppl:np.float64(0.21310683467891067) - rollout_corr/log_ppl_diff:np.float64(0.0007690480415476486) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026080097450176254) - rollout_corr/log_ppl_diff_max:np.float64(0.010331928730010986) - rollout_corr/log_ppl_diff_min:np.float64(-0.011125713586807251) - rollout_corr/ppl_ratio:np.float64(1.0007748361676931) - rollout_corr/chi2_token:np.float64(0.004497667541727424) - rollout_corr/chi2_seq:np.float64(1.3236201561521739) - actor/pg_loss:np.float64(0.005069697159342468) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008628873297311657) - actor/ppo_kl:np.float64(-7.736450616491197e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(291387.3354909057) - self_distillation/token_reweight_w_std:np.float64(3353028.3190113385) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000224343733862) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06953210350184236) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.531838670372963) - perf/mfu/actor:np.float64(0.03232239126793497) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.74435806274414) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:5 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002285066759213805 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.002285066759213805 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:235.265625 - response_length/max:684.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:235.265625 - response_length_non_aborted/max:684.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:353.09375 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.15045428276062e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9042285159230232) - timing_s/agent_loop/generate_sequences/max:np.float64(4.9942101035267115) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3118886794691207) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.9942101035267115) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:684 - timing_s/gen:10.56088761612773 - timing_s/reward:0.052522532641887665 - timing_s/old_log_prob:2.370040809735656 - timing_s/adv:0.5358876064419746 - timing_s/update_actor:16.438335929065943 - timing_s/step:30.05812031030655 - timing_s/stop_profile:0.00012102536857128143 - timing_per_token_ms/gen:0.17534846941833915 - timing_per_token_ms/update_actor:0.1091378032735755 - timing_per_token_ms/adv:0.0035578781466071878 - perf/total_num_tokens:150620 - perf/time_per_step:30.05812031030655 - perf/throughput:626.3698396850281 (TaskRunner pid=2122883) Training Progress: 81%|████████ | 81/100 [1:03:47<15:03, 47.54s/it] (TaskRunner pid=2122883) step:82 - global_seqlen/min:12582 - global_seqlen/max:21105 - global_seqlen/minmax_diff:8523 - global_seqlen/balanced_min:16604 - global_seqlen/balanced_max:16611 - global_seqlen/mean:16609.0 - actor/entropy:0.21828316152095795 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311316251754761 - training/rollout_probs_diff_mean:0.005371991079300642 - training/rollout_probs_diff_std:0.014509456232190132 - training/rollout_actor_probs_pearson_corr:0.9978258609771729 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9998401999473572 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.5041833547875285e-05 - rollout_corr/rollout_is_max:1.7522820234298706 - rollout_corr/rollout_is_min:0.2179795652627945 - rollout_corr/rollout_is_std:0.04170876741409302 - rollout_corr/rollout_is_eff_sample_size:0.9982629246817137 - rollout_corr/rollout_is_seq_mean:0.9997910857200623 - rollout_corr/rollout_is_seq_std:0.0029545440338552 - rollout_corr/rollout_is_seq_max:1.0073826313018799 - rollout_corr/rollout_is_seq_min:0.9908558130264282 - rollout_corr/rollout_is_seq_max_deviation:0.009144186973571777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.250613727606833) - rollout_corr/training_log_ppl:np.float64(0.2206964968208922) - rollout_corr/kl:np.float64(0.0011574556569300398) - rollout_corr/k3_kl:np.float64(0.001129698138868207) - rollout_corr/rollout_ppl:np.float64(1.2491793450899422) - rollout_corr/rollout_log_ppl:np.float64(0.21953904036490712) - rollout_corr/log_ppl_diff:np.float64(0.0011574564559850842) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027988772781100124) - rollout_corr/log_ppl_diff_max:np.float64(0.010943189263343811) - rollout_corr/log_ppl_diff_min:np.float64(-0.008710235357284546) - rollout_corr/ppl_ratio:np.float64(1.0011638076975942) - rollout_corr/chi2_token:np.float64(0.0022077024914324284) - rollout_corr/chi2_seq:np.float64(0.42010653484612703) - actor/pg_loss:np.float64(0.0061133516719564795) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015116771783141303) - actor/ppo_kl:np.float64(1.9086709578175487e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(585352.785423923) - self_distillation/token_reweight_w_std:np.float64(5886800.302368791) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995204512961209) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11297732920502312) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6384810656309128) - perf/mfu/actor:np.float64(0.02862514647106433) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.69329833984375) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:5 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005249614827334881 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005249614827334881 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:212.90625 - response_length/max:694.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:212.90625 - response_length_non_aborted/max:694.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:306.125 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016188248991966248 - timing_s/agent_loop/generate_sequences/min:np.float64(1.191643027588725) - timing_s/agent_loop/generate_sequences/max:np.float64(4.798683322966099) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.316618954995647) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.798683322966099) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:694 - timing_s/gen:10.743289474397898 - timing_s/reward:0.0457973200827837 - timing_s/old_log_prob:2.278144048526883 - timing_s/adv:0.49894460663199425 - timing_s/update_actor:16.217082366347313 - timing_s/step:29.820195823907852 - timing_s/stop_profile:0.00011817924678325653 - timing_per_token_ms/gen:0.19711011071477136 - timing_per_token_ms/update_actor:0.12205041217372593 - timing_per_token_ms/adv:0.003755077116563266 - perf/total_num_tokens:132872 - perf/time_per_step:29.820195823907852 - perf/throughput:556.9715268832678 (TaskRunner pid=2122883) Training Progress: 82%|████████▏ | 82/100 [1:04:17<12:40, 42.24s/it] (TaskRunner pid=2122883) step:83 - global_seqlen/min:12171 - global_seqlen/max:22671 - global_seqlen/minmax_diff:10500 - global_seqlen/balanced_min:16335 - global_seqlen/balanced_max:16338 - global_seqlen/mean:16336.875 - actor/entropy:0.21306166052818298 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3262504041194916 - training/rollout_probs_diff_mean:0.005117357242852449 - training/rollout_probs_diff_std:0.013851228170096874 - training/rollout_actor_probs_pearson_corr:0.997997522354126 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:0.9997509717941284 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.668184530804865e-05 - rollout_corr/rollout_is_max:1.9102360010147095 - rollout_corr/rollout_is_min:0.4035520553588867 - rollout_corr/rollout_is_std:0.04130885377526283 - rollout_corr/rollout_is_eff_sample_size:0.9982955945128663 - rollout_corr/rollout_is_seq_mean:0.9998254776000977 - rollout_corr/rollout_is_seq_std:0.003131643170490861 - rollout_corr/rollout_is_seq_max:1.0085992813110352 - rollout_corr/rollout_is_seq_min:0.9895705580711365 - rollout_corr/rollout_is_seq_max_deviation:0.010429441928863525 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2474157405085862) - rollout_corr/training_log_ppl:np.float64(0.21835851964715403) - rollout_corr/kl:np.float64(0.0012202397511362761) - rollout_corr/k3_kl:np.float64(0.0013064491049590288) - rollout_corr/rollout_ppl:np.float64(1.2458532857708633) - rollout_corr/rollout_log_ppl:np.float64(0.21713827828352805) - rollout_corr/log_ppl_diff:np.float64(0.0012202413636259735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031315083615481853) - rollout_corr/log_ppl_diff_max:np.float64(0.012149259448051453) - rollout_corr/log_ppl_diff_min:np.float64(-0.011597350239753723) - rollout_corr/ppl_ratio:np.float64(1.0012280030641705) - rollout_corr/chi2_token:np.float64(0.0028531376738101244) - rollout_corr/chi2_seq:np.float64(0.5433944647666067) - actor/pg_loss:np.float64(0.0065868336241692305) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018259112548548728) - actor/ppo_kl:np.float64(0.00016781720140102152) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(497521.2222965432) - self_distillation/token_reweight_w_std:np.float64(5335673.747744209) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009391447529197) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10771104626473971) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.696645587682724) - perf/mfu/actor:np.float64(0.02794429017338062) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.5435447692871) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:5 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006303493399173021 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006303493399173021 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:206.74609375 - response_length/max:518.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:206.74609375 - response_length_non_aborted/max:518.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:303.78125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013703666627407074 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2017349246889353) - timing_s/agent_loop/generate_sequences/max:np.float64(4.064564555883408) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2316053204122) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.064564555883408) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:518 - timing_s/gen:9.984482478350401 - timing_s/reward:0.052736496552824974 - timing_s/old_log_prob:2.2688260599970818 - timing_s/adv:0.47870176285505295 - timing_s/update_actor:16.14213657937944 - timing_s/step:29.018089989200234 - timing_s/stop_profile:0.00011230818927288055 - timing_per_token_ms/gen:0.18864629543239558 - timing_per_token_ms/update_actor:0.12350997803572776 - timing_per_token_ms/adv:0.003662739682888044 - perf/total_num_tokens:130695 - perf/time_per_step:29.018089989200234 - perf/throughput:562.98932859055 (TaskRunner pid=2122883) Training Progress: 83%|████████▎ | 83/100 [1:04:46<10:50, 38.29s/it] (TaskRunner pid=2122883) step:84 - global_seqlen/min:13107 - global_seqlen/max:21342 - global_seqlen/minmax_diff:8235 - global_seqlen/balanced_min:17248 - global_seqlen/balanced_max:17255 - global_seqlen/mean:17251.75 - actor/entropy:0.2078612893819809 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41484442353248596 - training/rollout_probs_diff_mean:0.00501943938434124 - training/rollout_probs_diff_std:0.013952094130218029 - training/rollout_actor_probs_pearson_corr:0.9979199767112732 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.999704897403717 - rollout_corr/rollout_is_ratio_fraction_high:1.8811844711308368e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.64355323149357e-05 - rollout_corr/rollout_is_max:3.0805070400238037 - rollout_corr/rollout_is_min:0.05248083919286728 - rollout_corr/rollout_is_std:0.040534697473049164 - rollout_corr/rollout_is_eff_sample_size:0.9983585642197468 - rollout_corr/rollout_is_seq_mean:0.9996325969696045 - rollout_corr/rollout_is_seq_std:0.0031430358067154884 - rollout_corr/rollout_is_seq_max:1.0117285251617432 - rollout_corr/rollout_is_seq_min:0.9894457459449768 - rollout_corr/rollout_is_seq_max_deviation:0.011728525161743164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.242810423951596) - rollout_corr/training_log_ppl:np.float64(0.21454886808351148) - rollout_corr/kl:np.float64(0.001670566343833002) - rollout_corr/k3_kl:np.float64(0.0014176062694559732) - rollout_corr/rollout_ppl:np.float64(1.240722066257149) - rollout_corr/rollout_log_ppl:np.float64(0.2128783009829931) - rollout_corr/log_ppl_diff:np.float64(0.001670567100518383) - rollout_corr/log_ppl_abs_diff:np.float64(0.003282473553554155) - rollout_corr/log_ppl_diff_max:np.float64(0.01952880620956421) - rollout_corr/log_ppl_diff_min:np.float64(-0.013135656714439392) - rollout_corr/ppl_ratio:np.float64(1.0016799475997686) - rollout_corr/chi2_token:np.float64(0.002261385088786483) - rollout_corr/chi2_seq:np.float64(1.0813027566764504) - actor/pg_loss:np.float64(0.0058089441154152155) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014233192905521719) - actor/ppo_kl:np.float64(0.0003872091626533347) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(483881.97194674145) - self_distillation/token_reweight_w_std:np.float64(5088252.776621332) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999582192394882) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1075905231264187) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.6635904759168625) - perf/mfu/actor:np.float64(0.02916947468869229) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.59191131591797) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:5 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.013916061259806156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.013916061259806156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:207.6484375 - response_length/max:750.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:207.6484375 - response_length_non_aborted/max:750.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:331.46875 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.707920253276825e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1966691762208939) - timing_s/agent_loop/generate_sequences/max:np.float64(4.983985688537359) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2186228269565618) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.983985688537359) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:750 - timing_s/gen:10.676102293655276 - timing_s/reward:0.05822275020182133 - timing_s/old_log_prob:2.3356290757656097 - timing_s/adv:0.47087168879806995 - timing_s/update_actor:16.726612301543355 - timing_s/step:30.356266921386123 - timing_s/stop_profile:0.00011728145182132721 - timing_per_token_ms/gen:0.20083717020307906 - timing_per_token_ms/update_actor:0.12119504036940712 - timing_per_token_ms/adv:0.0034117675655953015 - perf/total_num_tokens:138014 - perf/time_per_step:30.356266921386123 - perf/throughput:568.3093393755233 (TaskRunner pid=2122883) Training Progress: 84%|████████▍ | 84/100 [1:05:16<09:34, 35.92s/it] (TaskRunner pid=2122883) step:85 - global_seqlen/min:12069 - global_seqlen/max:20047 - global_seqlen/minmax_diff:7978 - global_seqlen/balanced_min:16063 - global_seqlen/balanced_max:16314 - global_seqlen/mean:16095.0 - actor/entropy:0.2045360505580902 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7543164491653442 - training/rollout_probs_diff_mean:0.005047583021223545 - training/rollout_probs_diff_std:0.01423710584640503 - training/rollout_actor_probs_pearson_corr:0.9978507161140442 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9999943375587463 - rollout_corr/rollout_is_ratio_fraction_high:3.8331800169544294e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9165900084772147e-05 - rollout_corr/rollout_is_max:4.387180328369141 - rollout_corr/rollout_is_min:0.49413782358169556 - rollout_corr/rollout_is_std:0.04079852253198624 - rollout_corr/rollout_is_eff_sample_size:0.9983380090836461 - rollout_corr/rollout_is_seq_mean:1.0000009536743164 - rollout_corr/rollout_is_seq_std:0.0031892997212707996 - rollout_corr/rollout_is_seq_max:1.0172944068908691 - rollout_corr/rollout_is_seq_min:0.9912353754043579 - rollout_corr/rollout_is_seq_max_deviation:0.01729440689086914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.241943544242531) - rollout_corr/training_log_ppl:np.float64(0.2141833759524161) - rollout_corr/kl:np.float64(0.0017040160140098237) - rollout_corr/k3_kl:np.float64(0.0013909810037660009) - rollout_corr/rollout_ppl:np.float64(1.2398182358592749) - rollout_corr/rollout_log_ppl:np.float64(0.21247935874271207) - rollout_corr/log_ppl_diff:np.float64(0.001704017209704034) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031017301516840234) - rollout_corr/log_ppl_diff_max:np.float64(0.01839999109506607) - rollout_corr/log_ppl_diff_min:np.float64(-0.006871715188026428) - rollout_corr/ppl_ratio:np.float64(1.0017130349297076) - rollout_corr/chi2_token:np.float64(0.002452671527862549) - rollout_corr/chi2_seq:np.float64(0.1490656854584813) - actor/pg_loss:np.float64(0.0054126320173963904) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001665425015744404) - actor/ppo_kl:np.float64(0.0007683562842331071) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(373973.35678189015) - self_distillation/token_reweight_w_std:np.float64(3960859.6874851873) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998681743163615) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10099843548960052) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.6237643733620644) - perf/mfu/actor:np.float64(0.02787586130815002) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.19374084472656) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:6 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011717552319169044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011717552319169044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:203.8125 - response_length/max:1094.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:203.8125 - response_length_non_aborted/max:1094.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.15625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002668313682079315 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2239922620356083) - timing_s/agent_loop/generate_sequences/max:np.float64(6.708017874509096) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2544095085613662) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.708017874509096) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:1094 - timing_s/gen:12.766073238104582 - timing_s/reward:0.051905008032917976 - timing_s/old_log_prob:2.5455926675349474 - timing_s/adv:0.4925172030925751 - timing_s/update_actor:16.307795433327556 - timing_s/step:32.2666488904506 - timing_s/stop_profile:0.00014187954366207123 - timing_per_token_ms/gen:0.24467328346566586 - timing_per_token_ms/update_actor:0.1266526517033827 - timing_per_token_ms/adv:0.0038250792411663177 - perf/total_num_tokens:128760 - perf/time_per_step:32.2666488904506 - perf/throughput:498.81225827462237 (TaskRunner pid=2122883) Training Progress: 85%|████████▌ | 85/100 [1:05:49<08:44, 34.97s/it] (TaskRunner pid=2122883) step:86 - global_seqlen/min:11119 - global_seqlen/max:22626 - global_seqlen/minmax_diff:11507 - global_seqlen/balanced_min:16850 - global_seqlen/balanced_max:16852 - global_seqlen/mean:16851.25 - actor/entropy:0.22577793896198273 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311284065246582 - training/rollout_probs_diff_mean:0.005342109594494104 - training/rollout_probs_diff_std:0.014018124900758266 - training/rollout_actor_probs_pearson_corr:0.9979961514472961 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000851154327393 - rollout_corr/rollout_is_ratio_fraction_high:1.8723085304372944e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.744617060874589e-05 - rollout_corr/rollout_is_max:2.115358352661133 - rollout_corr/rollout_is_min:0.21797996759414673 - rollout_corr/rollout_is_std:0.04185427725315094 - rollout_corr/rollout_is_eff_sample_size:0.9982517580297928 - rollout_corr/rollout_is_seq_mean:1.0001436471939087 - rollout_corr/rollout_is_seq_std:0.003316018031910062 - rollout_corr/rollout_is_seq_max:1.0115185976028442 - rollout_corr/rollout_is_seq_min:0.9917247891426086 - rollout_corr/rollout_is_seq_max_deviation:0.011518597602844238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2645139200612903) - rollout_corr/training_log_ppl:np.float64(0.23199850719538517) - rollout_corr/kl:np.float64(0.001092571028761924) - rollout_corr/k3_kl:np.float64(0.001342841520084903) - rollout_corr/rollout_ppl:np.float64(1.2630862044170499) - rollout_corr/rollout_log_ppl:np.float64(0.2309059344988782) - rollout_corr/log_ppl_diff:np.float64(0.001092572696506977) - rollout_corr/log_ppl_abs_diff:np.float64(0.003120153967756778) - rollout_corr/log_ppl_diff_max:np.float64(0.03792935609817505) - rollout_corr/log_ppl_diff_min:np.float64(-0.010497242212295532) - rollout_corr/ppl_ratio:np.float64(1.0011034170165658) - rollout_corr/chi2_token:np.float64(0.002893829019740224) - rollout_corr/chi2_seq:np.float64(1.1480373400263488) - actor/pg_loss:np.float64(0.005574887851253152) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017103734035117668) - actor/ppo_kl:np.float64(0.0003303216700523137) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(513197.1782494064) - self_distillation/token_reweight_w_std:np.float64(5438486.467104235) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009826868772507) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10202174982987344) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5823733061552048) - perf/mfu/actor:np.float64(0.02872267604930534) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.13408279418945) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:6 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024995319545269012 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024995319545269012 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:208.6328125 - response_length/max:542.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:208.6328125 - response_length_non_aborted/max:542.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.96875 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010875426232814789 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3264490310102701) - timing_s/agent_loop/generate_sequences/max:np.float64(4.1357088424265385) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3096187077389914) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.1357088424265385) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:542 - timing_s/gen:9.943797947838902 - timing_s/reward:0.04411541484296322 - timing_s/old_log_prob:2.2926547210663557 - timing_s/adv:0.47092016600072384 - timing_s/update_actor:16.34498623199761 - timing_s/step:29.13274565897882 - timing_s/stop_profile:3.097020089626312e-05 - timing_per_token_ms/gen:0.18617857981349753 - timing_per_token_ms/update_actor:0.12124461265482983 - timing_per_token_ms/adv:0.003493213901051286 - perf/total_num_tokens:134810 - perf/time_per_step:29.13274565897882 - perf/throughput:578.4298602423827 (TaskRunner pid=2122883) Training Progress: 86%|████████▌ | 86/100 [1:06:18<07:45, 33.22s/it] (TaskRunner pid=2122883) step:87 - global_seqlen/min:13699 - global_seqlen/max:20034 - global_seqlen/minmax_diff:6335 - global_seqlen/balanced_min:16196 - global_seqlen/balanced_max:16205 - global_seqlen/mean:16202.875 - actor/entropy:0.22808320820331573 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4770777225494385 - training/rollout_probs_diff_mean:0.0052826725877821445 - training/rollout_probs_diff_std:0.014059673063457012 - training/rollout_actor_probs_pearson_corr:0.998014509677887 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.69921875 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.69921875 - rollout_corr/rollout_is_mean:0.9995889067649841 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9377215721760876e-05 - rollout_corr/rollout_is_max:1.8165900707244873 - rollout_corr/rollout_is_min:0.13343973457813263 - rollout_corr/rollout_is_std:0.04224056005477905 - rollout_corr/rollout_is_eff_sample_size:0.998217546864214 - rollout_corr/rollout_is_seq_mean:0.9996139407157898 - rollout_corr/rollout_is_seq_std:0.0032783818896859884 - rollout_corr/rollout_is_seq_max:1.0129916667938232 - rollout_corr/rollout_is_seq_min:0.9896868467330933 - rollout_corr/rollout_is_seq_max_deviation:0.012991666793823242 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2670027003623545) - rollout_corr/training_log_ppl:np.float64(0.23412725170783233) - rollout_corr/kl:np.float64(0.0013061097060056426) - rollout_corr/k3_kl:np.float64(0.0011628404061525544) - rollout_corr/rollout_ppl:np.float64(1.2652938198298216) - rollout_corr/rollout_log_ppl:np.float64(0.2328211401036242) - rollout_corr/log_ppl_diff:np.float64(0.0013061116042081267) - rollout_corr/log_ppl_abs_diff:np.float64(0.00299576812540181) - rollout_corr/log_ppl_diff_max:np.float64(0.020071491599082947) - rollout_corr/log_ppl_diff_min:np.float64(-0.008261770009994507) - rollout_corr/ppl_ratio:np.float64(1.0013143399264663) - rollout_corr/chi2_token:np.float64(0.0019626670982688665) - rollout_corr/chi2_seq:np.float64(0.6271165478974581) - actor/pg_loss:np.float64(0.005690383608452976) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016033555998546944) - actor/ppo_kl:np.float64(-2.0786709365339107e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.69921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(740095.8522027656) - self_distillation/token_reweight_w_std:np.float64(7703203.598722717) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025711173657328) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11661678130622022) - self_distillation/empty_target_batch:np.float64(0.30078125) - actor/grad_norm:np.float64(0.6615225672721863) - perf/mfu/actor:np.float64(0.027610136886118748) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.15999221801758) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:6 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002850872930139303 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002850872930139303 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:201.58984375 - response_length/max:720.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:201.58984375 - response_length_non_aborted/max:720.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.75 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.329904913902283e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2359547521919012) - timing_s/agent_loop/generate_sequences/max:np.float64(4.947052989155054) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.24328743157821) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.947052989155054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:720 - timing_s/gen:10.758620804175735 - timing_s/reward:0.0536897424608469 - timing_s/old_log_prob:2.2264872901141644 - timing_s/adv:0.4786405097693205 - timing_s/update_actor:16.508141493424773 - timing_s/step:30.114757101982832 - timing_s/stop_profile:0.00012442097067832947 - timing_per_token_ms/gen:0.20847212207986773 - timing_per_token_ms/update_actor:0.12735503339241316 - timing_per_token_ms/adv:0.0036925584947834914 - perf/total_num_tokens:129623 - perf/time_per_step:30.114757101982832 - perf/throughput:538.0377117148709 (TaskRunner pid=2122883) Training Progress: 87%|████████▋ | 87/100 [1:06:48<06:59, 32.30s/it] (TaskRunner pid=2122883) step:88 - global_seqlen/min:12296 - global_seqlen/max:20557 - global_seqlen/minmax_diff:8261 - global_seqlen/balanced_min:17022 - global_seqlen/balanced_max:17025 - global_seqlen/mean:17023.375 - actor/entropy:0.22276757657527924 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6303352117538452 - training/rollout_probs_diff_mean:0.005168667994439602 - training/rollout_probs_diff_std:0.01409972459077835 - training/rollout_actor_probs_pearson_corr:0.9979792833328247 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.703125 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.703125 - rollout_corr/rollout_is_mean:1.0003288984298706 - rollout_corr/rollout_is_ratio_fraction_high:5.5634885939070955e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.27248111111112e-05 - rollout_corr/rollout_is_max:3.249321699142456 - rollout_corr/rollout_is_min:0.06875494867563248 - rollout_corr/rollout_is_std:0.04171448200941086 - rollout_corr/rollout_is_eff_sample_size:0.9982639938423429 - rollout_corr/rollout_is_seq_mean:1.000346064567566 - rollout_corr/rollout_is_seq_std:0.0032712207175791264 - rollout_corr/rollout_is_seq_max:1.0110112428665161 - rollout_corr/rollout_is_seq_min:0.9913240671157837 - rollout_corr/rollout_is_seq_max_deviation:0.011011242866516113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2595614441670477) - rollout_corr/training_log_ppl:np.float64(0.22807635995559394) - rollout_corr/kl:np.float64(0.001190670725655707) - rollout_corr/k3_kl:np.float64(0.001538011453078525) - rollout_corr/rollout_ppl:np.float64(1.257966611534357) - rollout_corr/rollout_log_ppl:np.float64(0.22688568860758096) - rollout_corr/log_ppl_diff:np.float64(0.0011906713480129838) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034372524241916835) - rollout_corr/log_ppl_diff_max:np.float64(0.021827280521392822) - rollout_corr/log_ppl_diff_min:np.float64(-0.014090389013290405) - rollout_corr/ppl_ratio:np.float64(1.0012007460463792) - rollout_corr/chi2_token:np.float64(0.004329530056566) - rollout_corr/chi2_seq:np.float64(1.7654117406345904) - actor/pg_loss:np.float64(0.0044011540012434125) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018381910886091646) - actor/ppo_kl:np.float64(0.0006000098311353952) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(616948.9494683831) - self_distillation/token_reweight_w_std:np.float64(6217318.756260075) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004455572925508) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10639075534709264) - self_distillation/empty_target_batch:np.float64(0.296875) - actor/grad_norm:np.float64(0.5419183969497681) - perf/mfu/actor:np.float64(0.028824236807819127) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.21399307250977) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:6 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034400904551148415 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034400904551148415 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:210.63671875 - response_length/max:547.0 - response_length/min:106.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:210.63671875 - response_length_non_aborted/max:547.0 - response_length_non_aborted/min:106.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:321.34375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016992166638374329 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9193975105881691) - timing_s/agent_loop/generate_sequences/max:np.float64(4.1702308394014835) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.1946431579781347) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.1702308394014835) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:539 - timing_s/gen:9.90405653603375 - timing_s/reward:0.054226161912083626 - timing_s/old_log_prob:2.2448495719581842 - timing_s/adv:0.4909304156899452 - timing_s/update_actor:16.746504409238696 - timing_s/step:29.53132553398609 - timing_s/stop_profile:0.00011452659964561462 - timing_per_token_ms/gen:0.18367035469157408 - timing_per_token_ms/update_actor:0.12296698223206838 - timing_per_token_ms/adv:0.003604825832788337 - perf/total_num_tokens:136187 - perf/time_per_step:29.53132553398609 - perf/throughput:576.4514356258295 (TaskRunner pid=2122883) Training Progress: 88%|████████▊ | 88/100 [1:07:18<06:17, 31.49s/it] (TaskRunner pid=2122883) step:89 - global_seqlen/min:15072 - global_seqlen/max:22879 - global_seqlen/minmax_diff:7807 - global_seqlen/balanced_min:18089 - global_seqlen/balanced_max:18166 - global_seqlen/mean:18099.375 - actor/entropy:0.20623931288719177 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7721544504165649 - training/rollout_probs_diff_mean:0.005006530787795782 - training/rollout_probs_diff_std:0.014681383036077023 - training/rollout_actor_probs_pearson_corr:0.997765302658081 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000381469726562 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00016286940081045032 - rollout_corr/rollout_is_max:1.7593015432357788 - rollout_corr/rollout_is_min:0.009484831243753433 - rollout_corr/rollout_is_std:0.0410359650850296 - rollout_corr/rollout_is_eff_sample_size:0.9983191181153939 - rollout_corr/rollout_is_seq_mean:1.000022530555725 - rollout_corr/rollout_is_seq_std:0.002867963397875428 - rollout_corr/rollout_is_seq_max:1.0078660249710083 - rollout_corr/rollout_is_seq_min:0.9917351603507996 - rollout_corr/rollout_is_seq_max_deviation:0.00826483964920044 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2432606685906649) - rollout_corr/training_log_ppl:np.float64(0.2151060424221214) - rollout_corr/kl:np.float64(0.001223206437249047) - rollout_corr/k3_kl:np.float64(0.0014821982162231961) - rollout_corr/rollout_ppl:np.float64(1.241701312828809) - rollout_corr/rollout_log_ppl:np.float64(0.21388283489795867) - rollout_corr/log_ppl_diff:np.float64(0.0012232075241627172) - rollout_corr/log_ppl_abs_diff:np.float64(0.003019628828042187) - rollout_corr/log_ppl_diff_max:np.float64(0.012458458542823792) - rollout_corr/log_ppl_diff_min:np.float64(-0.009425677359104156) - rollout_corr/ppl_ratio:np.float64(1.001230750232935) - rollout_corr/chi2_token:np.float64(0.0032338621094822884) - rollout_corr/chi2_seq:np.float64(0.811216983012855) - actor/pg_loss:np.float64(0.005030228639952838) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001661099498051044) - actor/ppo_kl:np.float64(0.00023485885652974048) - actor/pg_clipfrac_lower:np.float64(3.753240162041038e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(451622.0916177754) - self_distillation/token_reweight_w_std:np.float64(4642653.067052306) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9989522572141141) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08174082619370893) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.5944810956716537) - perf/mfu/actor:np.float64(0.03101760938380116) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.39448928833008) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:6 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021557575091719627 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021557575091719627 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:215.85546875 - response_length/max:907.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:215.85546875 - response_length_non_aborted/max:907.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.75 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000102972611784935 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3130767196416855) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7279154025018215) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.337402616853069) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7279154025018215) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:907 - timing_s/gen:11.618766691535711 - timing_s/reward:0.05591783672571182 - timing_s/old_log_prob:2.1951356530189514 - timing_s/adv:0.5187619235366583 - timing_s/update_actor:16.619402937591076 - timing_s/step:31.09566144272685 - timing_s/stop_profile:3.01264226436615e-05 - timing_per_token_ms/gen:0.21026016923099786 - timing_per_token_ms/update_actor:0.11477884552360977 - timing_per_token_ms/adv:0.0035827336823554563 - perf/total_num_tokens:144795 - perf/time_per_step:31.09566144272685 - perf/throughput:582.0546713031367 (TaskRunner pid=2122883) Training Progress: 89%|████████▉ | 89/100 [1:07:49<05:45, 31.37s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The question asks for the folding stability score of a specific protein sequence. Without access to the actual computational tool or database that calculates this score, it is not possible to determine the exact value. However, the options provided are numerical scores that are typically used in protein folding stability assessments. Among the options, 1.69 is a reasonable value for a folding stability score, as it falls within the typical range of scores used to indicate the stability of a protein structure. Therefore, the most plausible answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2122883) step:90 - global_seqlen/min:13820 - global_seqlen/max:20696 - global_seqlen/minmax_diff:6876 - global_seqlen/balanced_min:16907 - global_seqlen/balanced_max:16910 - global_seqlen/mean:16908.375 - actor/entropy:0.19698114693164825 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6937196254730225 - training/rollout_probs_diff_mean:0.004946956876665354 - training/rollout_probs_diff_std:0.014074870385229588 - training/rollout_actor_probs_pearson_corr:0.9977890253067017 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.9999498724937439 - rollout_corr/rollout_is_ratio_fraction_high:1.8322065443499014e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010993239266099408 - rollout_corr/rollout_is_max:3.0224504470825195 - rollout_corr/rollout_is_min:0.00395188108086586 - rollout_corr/rollout_is_std:0.04045668616890907 - rollout_corr/rollout_is_eff_sample_size:0.9983656933663717 - rollout_corr/rollout_is_seq_mean:0.9999589920043945 - rollout_corr/rollout_is_seq_std:0.0028546203393489122 - rollout_corr/rollout_is_seq_max:1.0088039636611938 - rollout_corr/rollout_is_seq_min:0.99063640832901 - rollout_corr/rollout_is_seq_max_deviation:0.00936359167098999 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2240748791955411) - rollout_corr/training_log_ppl:np.float64(0.199976087838877) - rollout_corr/kl:np.float64(0.0012380354552803396) - rollout_corr/k3_kl:np.float64(0.0013727707471673511) - rollout_corr/rollout_ppl:np.float64(1.2225207276642323) - rollout_corr/rollout_log_ppl:np.float64(0.19873805253882892) - rollout_corr/log_ppl_diff:np.float64(0.0012380353000480682) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029817646427545696) - rollout_corr/log_ppl_diff_max:np.float64(0.014439418911933899) - rollout_corr/log_ppl_diff_min:np.float64(-0.009225308895111084) - rollout_corr/ppl_ratio:np.float64(1.0012455948162824) - rollout_corr/chi2_token:np.float64(0.0027791301254183054) - rollout_corr/chi2_seq:np.float64(0.5106464147102088) - actor/pg_loss:np.float64(0.007799276732839644) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020063032634425326) - actor/ppo_kl:np.float64(0.00030233424429404465) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(428857.22522661276) - self_distillation/token_reweight_w_std:np.float64(5102986.168928434) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9979278270620853) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11433671996928751) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.6825058907270432) - perf/mfu/actor:np.float64(0.02891552234793523) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.10916900634766) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.485) - val-aux/sciknoweval/reward/std@16:np.float64(0.1826775697855999) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.55618) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14939994447654767) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.41332) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15483870511911316) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4847600000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18236506835118788) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.61418) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11578798271076449) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.35528) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1237239235875391) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.48751999999999995) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13784588373033047) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.66276) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08530648056930885) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3057) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09776880099258937) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.49102) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08840624761811913) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.6975800000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.05027353975557356) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.26202) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06976567401063935) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.49556) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.058256449010532264) - val-aux/sciknoweval/score/mean@16:np.float64(0.485) - val-aux/sciknoweval/score/std@16:np.float64(0.1826775697855999) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.55618) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14939994447654767) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.41332) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15483870511911316) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4847600000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18236506835118788) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.61418) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11578798271076449) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.35528) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1237239235875391) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.48751999999999995) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13784588373033047) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.66276) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08530648056930885) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3057) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09776880099258937) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.49102) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08840624761811913) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.6975800000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.05027353975557356) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.26202) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06976567401063935) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.49556) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.058256449010532264) - val-core/sciknoweval/acc/mean@16:np.float64(0.485) - val-aux/sciknoweval/acc/std@16:np.float64(0.1826775697855999) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.55618) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14939994447654767) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.41332) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15483870511911316) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4847600000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18236506835118788) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.61418) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11578798271076449) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.35528) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1237239235875391) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.48751999999999995) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13784588373033047) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.66276) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08530648056930885) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3057) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09776880099258937) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.49102) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08840624761811913) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.6975800000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.05027353975557356) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.26202) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06976567401063935) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.49556) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.058256449010532264) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:6 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017451767344027758 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017451767344027758 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:213.19921875 - response_length/max:599.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:213.19921875 - response_length_non_aborted/max:599.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.1875 - prompt_length/max:499.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5389716029167175e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3019966781139374) - timing_s/agent_loop/generate_sequences/max:np.float64(4.4793809447437525) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3214208512436016) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.4793809447437525) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:421 - timing_s/agent_loop/slowest/response_length:599 - timing_s/gen:10.459255415946245 - timing_s/reward:0.0545695535838604 - timing_s/old_log_prob:2.228510320186615 - timing_s/adv:0.479594849050045 - timing_s/update_actor:16.379232734441757 - timing_s/step:29.687829732894897 - timing_s/testing:21.612145015969872 - timing_s/save_checkpoint:6.945257265120745 - timing_s/stop_profile:0.00011808052659034729 - timing_per_token_ms/gen:0.19163516033540823 - timing_per_token_ms/update_actor:0.12108816440404353 - timing_per_token_ms/adv:0.0035455421429472452 - perf/total_num_tokens:135267 - perf/time_per_step:29.687829732894897 - perf/throughput:569.5389374072391 (TaskRunner pid=2122883) Training Progress: 90%|█████████ | 90/100 [1:08:47<06:34, 39.46s/it] (TaskRunner pid=2122883) step:91 - global_seqlen/min:12564 - global_seqlen/max:21781 - global_seqlen/minmax_diff:9217 - global_seqlen/balanced_min:17651 - global_seqlen/balanced_max:17661 - global_seqlen/mean:17654.0 - actor/entropy:0.1927318572998047 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102362394332886 - training/rollout_probs_diff_mean:0.0049341288395226 - training/rollout_probs_diff_std:0.013917961157858372 - training/rollout_actor_probs_pearson_corr:0.9978261590003967 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0002244710922241 - rollout_corr/rollout_is_ratio_fraction_high:1.766285095072817e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.065140380291268e-05 - rollout_corr/rollout_is_max:2.8016161918640137 - rollout_corr/rollout_is_min:0.35093075037002563 - rollout_corr/rollout_is_std:0.04169304296374321 - rollout_corr/rollout_is_eff_sample_size:0.9982654193934108 - rollout_corr/rollout_is_seq_mean:1.0002824068069458 - rollout_corr/rollout_is_seq_std:0.003071394981816411 - rollout_corr/rollout_is_seq_max:1.009526014328003 - rollout_corr/rollout_is_seq_min:0.9910033941268921 - rollout_corr/rollout_is_seq_max_deviation:0.00952601432800293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2255855221301317) - rollout_corr/training_log_ppl:np.float64(0.20155303589854157) - rollout_corr/kl:np.float64(0.0010046164143293623) - rollout_corr/k3_kl:np.float64(0.0012420805354622644) - rollout_corr/rollout_ppl:np.float64(1.2243025470525026) - rollout_corr/rollout_log_ppl:np.float64(0.20054841847741045) - rollout_corr/log_ppl_diff:np.float64(0.0010046174211311154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029608946133521385) - rollout_corr/log_ppl_diff_max:np.float64(0.015554353594779968) - rollout_corr/log_ppl_diff_min:np.float64(-0.006916940212249756) - rollout_corr/ppl_ratio:np.float64(1.0010123364627361) - rollout_corr/chi2_token:np.float64(0.003038359573110938) - rollout_corr/chi2_seq:np.float64(0.7696262006647885) - actor/pg_loss:np.float64(0.005795407225377858) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001670339756856265) - actor/ppo_kl:np.float64(0.0003866044301399363) - actor/pg_clipfrac_lower:np.float64(3.50374812114751e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(411195.32281198865) - self_distillation/token_reweight_w_std:np.float64(4754109.018892683) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010015049483627) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08646299615793396) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.5738260895013809) - perf/mfu/actor:np.float64(0.029971853940342928) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(145.91704177856445) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:6 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023601986467838287 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023601986467838287 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:221.15625 - response_length/max:894.0 - response_length/min:132.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:221.15625 - response_length_non_aborted/max:894.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:330.53125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.253699004650116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5865748934447765) - timing_s/agent_loop/generate_sequences/max:np.float64(6.002601191401482) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.461126289665117) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.002601191401482) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:894 - timing_s/gen:11.83268902078271 - timing_s/reward:0.051725566387176514 - timing_s/old_log_prob:2.437912594527006 - timing_s/adv:0.4970137644559145 - timing_s/update_actor:16.579867849126458 - timing_s/step:31.503263341262937 - timing_s/stop_profile:2.8558075428009033e-05 - timing_per_token_ms/gen:0.2089990289102499 - timing_per_token_ms/update_actor:0.11739455540618599 - timing_per_token_ms/adv:0.0035191299737730437 - perf/total_num_tokens:141232 - perf/time_per_step:31.503263341262937 - perf/throughput:560.3863894594314 (TaskRunner pid=2122883) Training Progress: 91%|█████████ | 91/100 [1:09:19<05:33, 37.08s/it] (TaskRunner pid=2122883) step:92 - global_seqlen/min:15984 - global_seqlen/max:22004 - global_seqlen/minmax_diff:6020 - global_seqlen/balanced_min:18842 - global_seqlen/balanced_max:18845 - global_seqlen/mean:18843.5 - actor/entropy:0.20552825927734375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8458645343780518 - training/rollout_probs_diff_mean:0.0050735012628138065 - training/rollout_probs_diff_std:0.01476234756410122 - training/rollout_actor_probs_pearson_corr:0.9976810216903687 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:1.0001269578933716 - rollout_corr/rollout_is_ratio_fraction_high:6.938180740689859e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.938180740689859e-05 - rollout_corr/rollout_is_max:3.0634891986846924 - rollout_corr/rollout_is_min:0.11200299113988876 - rollout_corr/rollout_is_std:0.04258697107434273 - rollout_corr/rollout_is_eff_sample_size:0.9981899895749777 - rollout_corr/rollout_is_seq_mean:1.0000636577606201 - rollout_corr/rollout_is_seq_std:0.0031039409805089235 - rollout_corr/rollout_is_seq_max:1.0127332210540771 - rollout_corr/rollout_is_seq_min:0.991115391254425 - rollout_corr/rollout_is_seq_max_deviation:0.012733221054077148 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.235127116087824) - rollout_corr/training_log_ppl:np.float64(0.20900695942691527) - rollout_corr/kl:np.float64(0.0012782263143336081) - rollout_corr/k3_kl:np.float64(0.0013150399036589988) - rollout_corr/rollout_ppl:np.float64(1.2335282661952078) - rollout_corr/rollout_log_ppl:np.float64(0.20772873278474435) - rollout_corr/log_ppl_diff:np.float64(0.001278226642170921) - rollout_corr/log_ppl_abs_diff:np.float64(0.002990237466292456) - rollout_corr/log_ppl_diff_max:np.float64(0.012402921915054321) - rollout_corr/log_ppl_diff_min:np.float64(-0.011149108409881592) - rollout_corr/ppl_ratio:np.float64(1.0012855627574027) - rollout_corr/chi2_token:np.float64(0.002650786191225052) - rollout_corr/chi2_seq:np.float64(0.7602505653630942) - actor/pg_loss:np.float64(0.004597345599904656) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014476663550340163) - actor/ppo_kl:np.float64(0.0003580096706059521) - actor/pg_clipfrac_lower:np.float64(2.2321428332361393e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(553014.0691977837) - self_distillation/token_reweight_w_std:np.float64(5880688.194196964) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0030879531987011) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08305959589779377) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.5153934583067894) - perf/mfu/actor:np.float64(0.03218116200910849) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(145.88633728027344) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:6 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007577794138342142 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007577794138342142 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:225.203125 - response_length/max:638.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:225.203125 - response_length_non_aborted/max:638.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:363.65625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5453045964241028e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4698635023087263) - timing_s/agent_loop/generate_sequences/max:np.float64(4.719432448968291) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.4977113083878066) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.719432448968291) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:630 - timing_s/gen:10.855900479480624 - timing_s/reward:0.046163953840732574 - timing_s/old_log_prob:2.2303929030895233 - timing_s/adv:0.48601470328867435 - timing_s/update_actor:16.612354155629873 - timing_s/step:30.2675437964499 - timing_s/stop_profile:0.00011699646711349487 - timing_per_token_ms/gen:0.1883005009276456 - timing_per_token_ms/update_actor:0.11019949953319363 - timing_per_token_ms/adv:0.0032240209043481465 - perf/total_num_tokens:150748 - perf/time_per_step:30.2675437964499 - perf/throughput:622.5645571613964 (TaskRunner pid=2122883) Training Progress: 92%|█████████▏| 92/100 [1:09:49<04:40, 35.05s/it] (TaskRunner pid=2122883) step:93 - global_seqlen/min:14142 - global_seqlen/max:22580 - global_seqlen/minmax_diff:8438 - global_seqlen/balanced_min:17968 - global_seqlen/balanced_max:19159 - global_seqlen/mean:18118.25 - actor/entropy:0.2024667114019394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3206155598163605 - training/rollout_probs_diff_mean:0.004839439410716295 - training/rollout_probs_diff_std:0.013173987157642841 - training/rollout_actor_probs_pearson_corr:0.9981030821800232 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.9996916651725769 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.3325748012866825e-05 - rollout_corr/rollout_is_max:1.922481656074524 - rollout_corr/rollout_is_min:0.0006340847467072308 - rollout_corr/rollout_is_std:0.0401468463242054 - rollout_corr/rollout_is_eff_sample_size:0.9983899332265034 - rollout_corr/rollout_is_seq_mean:0.9997001886367798 - rollout_corr/rollout_is_seq_std:0.0026396142784506083 - rollout_corr/rollout_is_seq_max:1.0092061758041382 - rollout_corr/rollout_is_seq_min:0.991854190826416 - rollout_corr/rollout_is_seq_max_deviation:0.009206175804138184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2384646688587964) - rollout_corr/training_log_ppl:np.float64(0.21190124790882692) - rollout_corr/kl:np.float64(0.001526296815164585) - rollout_corr/k3_kl:np.float64(0.0011333852019390633) - rollout_corr/rollout_ppl:np.float64(1.2365091615356505) - rollout_corr/rollout_log_ppl:np.float64(0.21037495086784475) - rollout_corr/log_ppl_diff:np.float64(0.001526297040982172) - rollout_corr/log_ppl_abs_diff:np.float64(0.002726003556745127) - rollout_corr/log_ppl_diff_max:np.float64(0.024714887142181396) - rollout_corr/log_ppl_diff_min:np.float64(-0.008926495909690857) - rollout_corr/ppl_ratio:np.float64(1.0015337793156505) - rollout_corr/chi2_token:np.float64(0.0012352506164461374) - rollout_corr/chi2_seq:np.float64(0.21279096603393555) - actor/pg_loss:np.float64(0.003292814362794161) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011469970941107022) - actor/ppo_kl:np.float64(0.0002611581061984225) - actor/pg_clipfrac_lower:np.float64(2.1701389414374717e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(562577.1132855881) - self_distillation/token_reweight_w_std:np.float64(5476452.781596074) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991126488894224) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08483730700390879) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.5270627364516258) - perf/mfu/actor:np.float64(0.030530903588503785) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.33957290649414) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:6 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0026640654541552067 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0026640654541552067 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:219.7578125 - response_length/max:1916.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:219.7578125 - response_length_non_aborted/max:1916.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:346.4375 - prompt_length/max:500.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001993868499994278 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4296222887933254) - timing_s/agent_loop/generate_sequences/max:np.float64(10.560190688818693) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.3909345842766925) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.560190688818693) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:1916 - timing_s/gen:16.355938605964184 - timing_s/reward:0.056257134303450584 - timing_s/old_log_prob:2.2599684596061707 - timing_s/adv:0.6293122116476297 - timing_s/update_actor:16.973596734926105 - timing_s/step:36.360969342291355 - timing_s/stop_profile:3.0001625418663025e-05 - timing_per_token_ms/gen:0.2907308934900669 - timing_per_token_ms/update_actor:0.11710289856171335 - timing_per_token_ms/adv:0.004341701127644983 - perf/total_num_tokens:144946 - perf/time_per_step:36.360969342291355 - perf/throughput:498.2884210110072 (TaskRunner pid=2122883) Training Progress: 93%|█████████▎| 93/100 [1:10:26<04:08, 35.45s/it] (TaskRunner pid=2122883) step:94 - global_seqlen/min:12367 - global_seqlen/max:21204 - global_seqlen/minmax_diff:8837 - global_seqlen/balanced_min:15481 - global_seqlen/balanced_max:15486 - global_seqlen/mean:15484.25 - actor/entropy:0.18188761174678802 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26072946190834045 - training/rollout_probs_diff_mean:0.004726748447865248 - training/rollout_probs_diff_std:0.013647111132740974 - training/rollout_actor_probs_pearson_corr:0.9978026747703552 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9999868273735046 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.8747504226630554e-05 - rollout_corr/rollout_is_max:1.9659937620162964 - rollout_corr/rollout_is_min:0.06315071135759354 - rollout_corr/rollout_is_std:0.03998841717839241 - rollout_corr/rollout_is_eff_sample_size:0.9984032418855266 - rollout_corr/rollout_is_seq_mean:1.0000756978988647 - rollout_corr/rollout_is_seq_std:0.003055985551327467 - rollout_corr/rollout_is_seq_max:1.0124971866607666 - rollout_corr/rollout_is_seq_min:0.9919678568840027 - rollout_corr/rollout_is_seq_max_deviation:0.012497186660766602 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.208876115269959) - rollout_corr/training_log_ppl:np.float64(0.1878806567110587) - rollout_corr/kl:np.float64(0.001109215704858002) - rollout_corr/k3_kl:np.float64(0.0013251136373924055) - rollout_corr/rollout_ppl:np.float64(1.2075008023530245) - rollout_corr/rollout_log_ppl:np.float64(0.1867714399995748) - rollout_corr/log_ppl_diff:np.float64(0.0011092167114838958) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029893830069340765) - rollout_corr/log_ppl_diff_max:np.float64(0.017061367630958557) - rollout_corr/log_ppl_diff_min:np.float64(-0.011037014424800873) - rollout_corr/ppl_ratio:np.float64(1.0011169675271958) - rollout_corr/chi2_token:np.float64(0.0031342231668531895) - rollout_corr/chi2_seq:np.float64(0.7604034042451531) - actor/pg_loss:np.float64(0.0038497489877045155) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0018158378306907252) - actor/ppo_kl:np.float64(0.0003219292054641443) - actor/pg_clipfrac_lower:np.float64(2.0559211407089606e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(568107.0682997338) - self_distillation/token_reweight_w_std:np.float64(5762513.378730269) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993454308714718) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07923695075442083) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.571109913289547) - perf/mfu/actor:np.float64(0.02667527205188385) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.28662490844727) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:6 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017134688096120954 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017134688096120954 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:199.4765625 - response_length/max:551.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:199.4765625 - response_length_non_aborted/max:551.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:284.40625 - prompt_length/max:500.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010585226118564606 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4458684269338846) - timing_s/agent_loop/generate_sequences/max:np.float64(4.342792140319943) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.248531175951939) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.342792140319943) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:551 - timing_s/gen:10.355708943679929 - timing_s/reward:0.052927035838365555 - timing_s/old_log_prob:2.2116340044885874 - timing_s/adv:0.4717257060110569 - timing_s/update_actor:16.339361615478992 - timing_s/step:29.518693948164582 - timing_s/stop_profile:0.00011059083044528961 - timing_per_token_ms/gen:0.20279068154309968 - timing_per_token_ms/update_actor:0.13190307583091684 - timing_per_token_ms/adv:0.0038081090948145447 - perf/total_num_tokens:123874 - perf/time_per_step:29.518693948164582 - perf/throughput:524.5574220590739 (TaskRunner pid=2122883) Training Progress: 94%|█████████▍| 94/100 [1:10:55<03:22, 33.68s/it] (TaskRunner pid=2122883) step:95 - global_seqlen/min:12550 - global_seqlen/max:19516 - global_seqlen/minmax_diff:6966 - global_seqlen/balanced_min:15669 - global_seqlen/balanced_max:15675 - global_seqlen/mean:15672.625 - actor/entropy:0.1723659634590149 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42921364307403564 - training/rollout_probs_diff_mean:0.004754330962896347 - training/rollout_probs_diff_std:0.01394499558955431 - training/rollout_actor_probs_pearson_corr:0.997614324092865 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999802708625793 - rollout_corr/rollout_is_ratio_fraction_high:7.880686462158337e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.9105146647198126e-05 - rollout_corr/rollout_is_max:3.042388916015625 - rollout_corr/rollout_is_min:0.31485360860824585 - rollout_corr/rollout_is_std:0.04127493128180504 - rollout_corr/rollout_is_eff_sample_size:0.9982989210156206 - rollout_corr/rollout_is_seq_mean:0.9999309778213501 - rollout_corr/rollout_is_seq_std:0.0030664117075502872 - rollout_corr/rollout_is_seq_max:1.0131924152374268 - rollout_corr/rollout_is_seq_min:0.9899721145629883 - rollout_corr/rollout_is_seq_max_deviation:0.013192415237426758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.194142870604992) - rollout_corr/training_log_ppl:np.float64(0.17597143551392946) - rollout_corr/kl:np.float64(0.0014179566909646724) - rollout_corr/k3_kl:np.float64(0.001468020923283575) - rollout_corr/rollout_ppl:np.float64(1.1923948260955513) - rollout_corr/rollout_log_ppl:np.float64(0.1745534775982378) - rollout_corr/log_ppl_diff:np.float64(0.0014179579156916589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032777310407254845) - rollout_corr/log_ppl_diff_max:np.float64(0.028223499655723572) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074967145919799805) - rollout_corr/ppl_ratio:np.float64(1.0014282818883657) - rollout_corr/chi2_token:np.float64(0.0027583797927945852) - rollout_corr/chi2_seq:np.float64(0.5167441787198186) - actor/pg_loss:np.float64(0.0040054775308817625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016601379911662661) - actor/ppo_kl:np.float64(0.0004547775269259091) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(408989.465876037) - self_distillation/token_reweight_w_std:np.float64(4584876.381394251) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018037902191281) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06804537688731216) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.555400162935257) - perf/mfu/actor:np.float64(0.026826703716015997) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.3315315246582) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:6 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007434935308992863 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007434935308992863 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:198.26953125 - response_length/max:567.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:198.26953125 - response_length_non_aborted/max:567.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.5 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013350322842597961 - timing_s/agent_loop/generate_sequences/min:np.float64(1.496675157919526) - timing_s/agent_loop/generate_sequences/max:np.float64(4.148937383666635) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.279242074866488) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.148937383666635) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:567 - timing_s/gen:9.941075744107366 - timing_s/reward:0.05165614001452923 - timing_s/old_log_prob:2.20885256677866 - timing_s/adv:0.4986916873604059 - timing_s/update_actor:16.47932105138898 - timing_s/step:29.26573464088142 - timing_s/stop_profile:0.00011342950165271759 - timing_per_token_ms/gen:0.19585625123839798 - timing_per_token_ms/update_actor:0.1314339577080178 - timing_per_token_ms/adv:0.003977410352129955 - perf/total_num_tokens:125381 - perf/time_per_step:29.26573464088142 - perf/throughput:535.528159204548 (TaskRunner pid=2122883) Training Progress: 95%|█████████▌| 95/100 [1:11:24<02:41, 32.37s/it] (TaskRunner pid=2122883) step:96 - global_seqlen/min:11753 - global_seqlen/max:19731 - global_seqlen/minmax_diff:7978 - global_seqlen/balanced_min:15977 - global_seqlen/balanced_max:16380 - global_seqlen/mean:16066.125 - actor/entropy:0.18051546812057495 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5147483348846436 - training/rollout_probs_diff_mean:0.005008666310459375 - training/rollout_probs_diff_std:0.014469319954514503 - training/rollout_actor_probs_pearson_corr:0.9975249171257019 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9997128248214722 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.8940052036195993e-05 - rollout_corr/rollout_is_max:1.883395791053772 - rollout_corr/rollout_is_min:0.0843614861369133 - rollout_corr/rollout_is_std:0.0420609749853611 - rollout_corr/rollout_is_eff_sample_size:0.9982329891138849 - rollout_corr/rollout_is_seq_mean:0.9996994733810425 - rollout_corr/rollout_is_seq_std:0.0030340866651386023 - rollout_corr/rollout_is_seq_max:1.0078694820404053 - rollout_corr/rollout_is_seq_min:0.9882951378822327 - rollout_corr/rollout_is_seq_max_deviation:0.011704862117767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2039276850409806) - rollout_corr/training_log_ppl:np.float64(0.1833886240492575) - rollout_corr/kl:np.float64(0.0019757910764717224) - rollout_corr/k3_kl:np.float64(0.0021378641860110292) - rollout_corr/rollout_ppl:np.float64(1.2014858215115964) - rollout_corr/rollout_log_ppl:np.float64(0.18141283292789012) - rollout_corr/log_ppl_diff:np.float64(0.001975791121367365) - rollout_corr/log_ppl_abs_diff:np.float64(0.004001759923994541) - rollout_corr/log_ppl_diff_max:np.float64(0.03609415888786316) - rollout_corr/log_ppl_diff_min:np.float64(-0.013453051447868347) - rollout_corr/ppl_ratio:np.float64(1.001994721358642) - rollout_corr/chi2_token:np.float64(0.0038919271901249886) - rollout_corr/chi2_seq:np.float64(1.5032801111228764) - actor/pg_loss:np.float64(0.008404427906498313) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0034021124438368133) - actor/ppo_kl:np.float64(0.0007476098585712521) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(604270.4651900218) - self_distillation/token_reweight_w_std:np.float64(6237989.264843742) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000877343583852) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1141639132401906) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.7972646504640579) - perf/mfu/actor:np.float64(0.02722053833241182) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.33768463134766) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:6 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006266914773732424 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.006266914773732424 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:200.62890625 - response_length/max:991.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:200.62890625 - response_length_non_aborted/max:991.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:301.4375 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001253075897693634 - timing_s/agent_loop/generate_sequences/min:np.float64(1.300127375870943) - timing_s/agent_loop/generate_sequences/max:np.float64(6.019844556227326) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2045263498512213) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.019844556227326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:991 - timing_s/gen:11.907243676483631 - timing_s/reward:0.051400572061538696 - timing_s/old_log_prob:2.1595465298742056 - timing_s/adv:0.5041366368532181 - timing_s/update_actor:16.78061268851161 - timing_s/step:31.484902096912265 - timing_s/stop_profile:0.00011850707232952118 - timing_per_token_ms/gen:0.2318343427208121 - timing_per_token_ms/update_actor:0.1305589609233061 - timing_per_token_ms/adv:0.003922357108926531 - perf/total_num_tokens:128529 - perf/time_per_step:31.484902096912265 - perf/throughput:510.28029086917854 (TaskRunner pid=2122883) Training Progress: 96%|█████████▌| 96/100 [1:11:56<02:08, 32.12s/it] (TaskRunner pid=2122883) step:97 - global_seqlen/min:10953 - global_seqlen/max:18766 - global_seqlen/minmax_diff:7813 - global_seqlen/balanced_min:15532 - global_seqlen/balanced_max:15577 - global_seqlen/mean:15539.5 - actor/entropy:0.16762813925743103 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4621080160140991 - training/rollout_probs_diff_mean:0.0049015823751688 - training/rollout_probs_diff_std:0.014530841261148453 - training/rollout_actor_probs_pearson_corr:0.9972710013389587 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0001134872436523 - rollout_corr/rollout_is_ratio_fraction_high:2.096787648042664e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.290362944127992e-05 - rollout_corr/rollout_is_max:2.718266487121582 - rollout_corr/rollout_is_min:0.27240684628486633 - rollout_corr/rollout_is_std:0.04075905680656433 - rollout_corr/rollout_is_eff_sample_size:0.9983419299402324 - rollout_corr/rollout_is_seq_mean:1.0001165866851807 - rollout_corr/rollout_is_seq_std:0.0031043454073369503 - rollout_corr/rollout_is_seq_max:1.0120599269866943 - rollout_corr/rollout_is_seq_min:0.9912907481193542 - rollout_corr/rollout_is_seq_max_deviation:0.012059926986694336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.180992522276938) - rollout_corr/training_log_ppl:np.float64(0.16471268837631214) - rollout_corr/kl:np.float64(0.0010781966158646128) - rollout_corr/k3_kl:np.float64(0.00135383781866949) - rollout_corr/rollout_ppl:np.float64(1.179666185285896) - rollout_corr/rollout_log_ppl:np.float64(0.16363449190976098) - rollout_corr/log_ppl_diff:np.float64(0.0010781964665511623) - rollout_corr/log_ppl_abs_diff:np.float64(0.002958475160994567) - rollout_corr/log_ppl_diff_max:np.float64(0.040684640407562256) - rollout_corr/log_ppl_diff_min:np.float64(-0.00859902799129486) - rollout_corr/ppl_ratio:np.float64(1.0010882455389947) - rollout_corr/chi2_token:np.float64(0.0030083218589425087) - rollout_corr/chi2_seq:np.float64(0.46528330561704934) - actor/pg_loss:np.float64(0.0045534297823905945) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000986409560937318) - actor/ppo_kl:np.float64(0.00034714695744098023) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(418781.28417293704) - self_distillation/token_reweight_w_std:np.float64(4127527.157774403) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994778318796307) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05477544415043667) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.47031004168093204) - perf/mfu/actor:np.float64(0.026484544790594824) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.26701736450195) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:6 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008989977068267763 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008989977068267763 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:186.296875 - response_length/max:521.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:186.296875 - response_length_non_aborted/max:521.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.3125 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.993595838546753e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3136495407670736) - timing_s/agent_loop/generate_sequences/max:np.float64(3.662762999534607) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.092001289151085) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.662762999534607) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:499 - timing_s/agent_loop/slowest/response_length:521 - timing_s/gen:9.951778454706073 - timing_s/reward:0.05435767211019993 - timing_s/old_log_prob:2.201238000765443 - timing_s/adv:0.47390690073370934 - timing_s/update_actor:16.638015927746892 - timing_s/step:29.408174956217408 - timing_s/stop_profile:0.00012020952999591827 - timing_per_token_ms/gen:0.20866766868040915 - timing_per_token_ms/update_actor:0.13383648064405942 - timing_per_token_ms/adv:0.003812115099695207 - perf/total_num_tokens:124316 - perf/time_per_step:29.408174956217408 - perf/throughput:528.4074929211027 (TaskRunner pid=2122883) Training Progress: 97%|█████████▋| 97/100 [1:12:25<01:33, 31.32s/it] (TaskRunner pid=2122883) step:98 - global_seqlen/min:10609 - global_seqlen/max:19875 - global_seqlen/minmax_diff:9266 - global_seqlen/balanced_min:15622 - global_seqlen/balanced_max:15661 - global_seqlen/mean:15639.625 - actor/entropy:0.15689386427402496 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29895806312561035 - training/rollout_probs_diff_mean:0.004634734708815813 - training/rollout_probs_diff_std:0.014165589585900307 - training/rollout_actor_probs_pearson_corr:0.9973638653755188 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.6171875 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6171875 - rollout_corr/rollout_is_mean:0.9999297857284546 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00020426088303793222 - rollout_corr/rollout_is_max:1.9731019735336304 - rollout_corr/rollout_is_min:0.3392069637775421 - rollout_corr/rollout_is_std:0.04116937890648842 - rollout_corr/rollout_is_eff_sample_size:0.9983079502062758 - rollout_corr/rollout_is_seq_mean:0.9998367428779602 - rollout_corr/rollout_is_seq_std:0.0030175577849149704 - rollout_corr/rollout_is_seq_max:1.0067346096038818 - rollout_corr/rollout_is_seq_min:0.9912088513374329 - rollout_corr/rollout_is_seq_max_deviation:0.008791148662567139 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1760511882603168) - rollout_corr/training_log_ppl:np.float64(0.1607860552467173) - rollout_corr/kl:np.float64(0.0013580936059085502) - rollout_corr/k3_kl:np.float64(0.0012840381255614375) - rollout_corr/rollout_ppl:np.float64(1.1744432025589049) - rollout_corr/rollout_log_ppl:np.float64(0.15942796152376104) - rollout_corr/log_ppl_diff:np.float64(0.001358093722956255) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031231173197738826) - rollout_corr/log_ppl_diff_max:np.float64(0.01717827469110489) - rollout_corr/log_ppl_diff_min:np.float64(-0.00918780267238617) - rollout_corr/ppl_ratio:np.float64(1.0013662145938724) - rollout_corr/chi2_token:np.float64(0.002376826712861657) - rollout_corr/chi2_seq:np.float64(0.4572152856271714) - actor/pg_loss:np.float64(0.005094089079648256) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014830221407464705) - actor/ppo_kl:np.float64(0.00030168789419349196) - actor/pg_clipfrac_lower:np.float64(2.0777924873982556e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(541957.2093263853) - self_distillation/token_reweight_w_std:np.float64(5558886.31772876) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006034357938915) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06585665716556832) - self_distillation/empty_target_batch:np.float64(0.3828125) - actor/grad_norm:np.float64(0.5607671290636063) - perf/mfu/actor:np.float64(0.02655483396971807) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.38301467895508) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:6 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006357620004564524 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006357620004564524 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:191.23828125 - response_length/max:619.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:191.23828125 - response_length_non_aborted/max:619.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:297.5 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.298697113990784e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1248416900634766) - timing_s/agent_loop/generate_sequences/max:np.float64(4.359633307904005) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.9661425068916287) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.359633307904005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:619 - timing_s/gen:10.306377716362476 - timing_s/reward:0.053195398300886154 - timing_s/old_log_prob:2.2464902866631746 - timing_s/adv:0.48585028015077114 - timing_s/update_actor:16.741798777133226 - timing_s/step:29.920553406700492 - timing_s/stop_profile:0.0001219678670167923 - timing_per_token_ms/gen:0.2105189802553767 - timing_per_token_ms/update_actor:0.13380914485747922 - timing_per_token_ms/adv:0.0038831675963359987 - perf/total_num_tokens:125117 - perf/time_per_step:29.920553406700492 - perf/throughput:522.7050712403474 (TaskRunner pid=2122883) Training Progress: 98%|█████████▊| 98/100 [1:12:55<01:01, 30.91s/it] (TaskRunner pid=2122883) step:99 - global_seqlen/min:11254 - global_seqlen/max:19365 - global_seqlen/minmax_diff:8111 - global_seqlen/balanced_min:14846 - global_seqlen/balanced_max:14884 - global_seqlen/mean:14854.875 - actor/entropy:0.1654287576675415 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3766273558139801 - training/rollout_probs_diff_mean:0.004851612728089094 - training/rollout_probs_diff_std:0.01478587370365858 - training/rollout_actor_probs_pearson_corr:0.9972561001777649 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9998522400856018 - rollout_corr/rollout_is_ratio_fraction_high:8.606036863056943e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017212073726113886 - rollout_corr/rollout_is_max:2.241464614868164 - rollout_corr/rollout_is_min:0.0024131345562636852 - rollout_corr/rollout_is_std:0.04235963150858879 - rollout_corr/rollout_is_eff_sample_size:0.9982082817437392 - rollout_corr/rollout_is_seq_mean:0.9998134970664978 - rollout_corr/rollout_is_seq_std:0.0033946263138204813 - rollout_corr/rollout_is_seq_max:1.0130257606506348 - rollout_corr/rollout_is_seq_min:0.9866660833358765 - rollout_corr/rollout_is_seq_max_deviation:0.013333916664123535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1866260999813676) - rollout_corr/training_log_ppl:np.float64(0.16976790214539506) - rollout_corr/kl:np.float64(0.0013737595602520969) - rollout_corr/k3_kl:np.float64(0.0014210123007671882) - rollout_corr/rollout_ppl:np.float64(1.1849938463419676) - rollout_corr/rollout_log_ppl:np.float64(0.1683941402588971) - rollout_corr/log_ppl_diff:np.float64(0.0013737618864979595) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031606208940502256) - rollout_corr/log_ppl_diff_max:np.float64(0.019837364554405212) - rollout_corr/log_ppl_diff_min:np.float64(-0.011420145630836487) - rollout_corr/ppl_ratio:np.float64(1.0013831653632224) - rollout_corr/chi2_token:np.float64(0.002982073463499546) - rollout_corr/chi2_seq:np.float64(0.4137629037722945) - actor/pg_loss:np.float64(0.005757042206823826) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0016545631406188477) - actor/ppo_kl:np.float64(0.0001789195605663707) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(424982.73119985964) - self_distillation/token_reweight_w_std:np.float64(4268973.158323454) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9983152812346816) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08449886360904202) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.6708255037665367) - perf/mfu/actor:np.float64(0.025329531080435078) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.14479446411133) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:7 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00973826926201582 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00973826926201582 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:181.55859375 - response_length/max:564.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:181.55859375 - response_length_non_aborted/max:564.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.65625 - prompt_length/max:499.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00039384886622428894 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4838687852025032) - timing_s/agent_loop/generate_sequences/max:np.float64(4.118193060159683) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.065903241862543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.118193060159683) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:564 - timing_s/gen:9.949486400932074 - timing_s/reward:0.04738493822515011 - timing_s/old_log_prob:2.369191888719797 - timing_s/adv:0.5069726314395666 - timing_s/update_actor:16.423191968351603 - timing_s/step:29.40855737030506 - timing_s/stop_profile:3.053061664104462e-05 - timing_per_token_ms/gen:0.2140641236027469 - timing_per_token_ms/update_actor:0.13819698893756766 - timing_per_token_ms/adv:0.004266045922967769 - perf/total_num_tokens:118839 - perf/time_per_step:29.40855737030506 - perf/throughput:505.1208331286434 (TaskRunner pid=2122883) Training Progress: 99%|█████████▉| 99/100 [1:13:25<00:30, 30.61s/it] (TaskRunner pid=2122883) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2122883) validation generation end (TaskRunner pid=2122883) [prompt] system (TaskRunner pid=2122883) (TaskRunner pid=2122883) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ... (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2122883) (TaskRunner pid=2122883) A (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) user (TaskRunner pid=2122883) What is the folding stability score of the protein sequence 'GSSTTRYRFLDEEEARRAAKEWARRGYQVHVTQNGTYWEVEVR'? (TaskRunner pid=2122883) (TaskRunner pid=2122883) A: -0.01 (TaskRunner pid=2122883) B: 1.69 (TaskRunner pid=2122883) C: 2.49 (TaskRunner pid=2122883) D: 0.45 (TaskRunner pid=2122883) Please reason step by step. (TaskRunner pid=2122883) assistant (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) [response] (TaskRunner pid=2122883) The folding stability score of a protein sequence is typically determined through computational methods or experimental techniques, which assess the structural stability based on factors like secondary structure, hydrogen bonding, and van der Waals interactions. Without access to specific computational tools or databases that can calculate this score for the given sequence, it is not possible to determine the exact value. However, the provided options suggest a range of possible scores. Among the options, 1.69 is a reasonable value for a protein sequence that may have moderate stability, as it falls within the typical range of stability scores for proteins. Therefore, the most plausible answer is 1.69. (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) B (TaskRunner pid=2122883) (TaskRunner pid=2122883) [ground_truth] B (TaskRunner pid=2122883) [score] 1.0 (TaskRunner pid=2122883) [acc] 1.0 (TaskRunner pid=2122883) [pred] B (TaskRunner pid=2122883) [incorrect_format] 1 (TaskRunner pid=2122883) [feedback] (TaskRunner pid=2122883) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2123259) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/biology/qwen3gen-biology-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2122883) step:100 - global_seqlen/min:12732 - global_seqlen/max:21577 - global_seqlen/minmax_diff:8845 - global_seqlen/balanced_min:17451 - global_seqlen/balanced_max:17456 - global_seqlen/mean:17454.0 - actor/entropy:0.17713677883148193 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6802612543106079 - training/rollout_probs_diff_mean:0.004852317273616791 - training/rollout_probs_diff_std:0.015326019376516342 - training/rollout_actor_probs_pearson_corr:0.9972407817840576 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.51953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:1.0001729726791382 - rollout_corr/rollout_is_ratio_fraction_high:3.995525185018778e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00019977624469902366 - rollout_corr/rollout_is_max:3.1677703857421875 - rollout_corr/rollout_is_min:0.004871013108640909 - rollout_corr/rollout_is_std:0.04112447053194046 - rollout_corr/rollout_is_eff_sample_size:0.9983123460555775 - rollout_corr/rollout_is_seq_mean:1.0002431869506836 - rollout_corr/rollout_is_seq_std:0.0035587374586611986 - rollout_corr/rollout_is_seq_max:1.0158462524414062 - rollout_corr/rollout_is_seq_min:0.9900786280632019 - rollout_corr/rollout_is_seq_max_deviation:0.01584625244140625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.203045655041933) - rollout_corr/training_log_ppl:np.float64(0.18334656501247082) - rollout_corr/kl:np.float64(0.0016552262983289268) - rollout_corr/k3_kl:np.float64(0.0015354104443474625) - rollout_corr/rollout_ppl:np.float64(1.2010509767569602) - rollout_corr/rollout_log_ppl:np.float64(0.18169133765331935) - rollout_corr/log_ppl_diff:np.float64(0.0016552273591514677) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034326322202105075) - rollout_corr/log_ppl_diff_max:np.float64(0.025318950414657593) - rollout_corr/log_ppl_diff_min:np.float64(-0.009318530559539795) - rollout_corr/ppl_ratio:np.float64(1.001667067874223) - rollout_corr/chi2_token:np.float64(0.0024400001857429743) - rollout_corr/chi2_seq:np.float64(0.4870492776390165) - actor/pg_loss:np.float64(0.004485586890950799) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013001139122934546) - actor/ppo_kl:np.float64(0.0008072886606438345) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51953125) - self_distillation/token_reweight_w_mean:np.float64(702577.8307879148) - self_distillation/token_reweight_w_std:np.float64(6412720.680917468) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022917841561139) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05805497569963336) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.551053985953331) - perf/mfu/actor:np.float64(0.030039900561703775) - perf/max_memory_allocated_gb:np.float64(127.15039157867432) - perf/max_memory_reserved_gb:np.float64(133.6796875) - perf/cpu_memory_used_gb:np.float64(144.17816162109375) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.42875) - val-aux/sciknoweval/reward/std@16:np.float64(0.1440962083400905) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.49282000000000004) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11460243726780435) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.36628) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13169012287904225) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.4294) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.14357431870086687) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.53776) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07143174787390709) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.30992) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09737417027288235) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.43699999999999994) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12461646331648187) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.56108) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.03374430360648781) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.27224) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.058556923969906534) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.44081999999999993) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10054251250479307) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.5715600000000001) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.016366340783720925) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.25117999999999996) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.027867680697389763) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.44438) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.08103965668240862) - val-aux/sciknoweval/score/mean@16:np.float64(0.42875) - val-aux/sciknoweval/score/std@16:np.float64(0.1440962083400905) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.49282000000000004) - val-aux/sciknoweval/score/best@2/std:np.float64(0.11460243726780435) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.36628) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13169012287904225) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.4294) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.14357431870086687) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.53776) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07143174787390709) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.30992) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09737417027288235) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.43699999999999994) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12461646331648187) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.56108) - val-aux/sciknoweval/score/best@8/std:np.float64(0.03374430360648781) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.27224) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.058556923969906534) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.44081999999999993) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10054251250479307) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.5715600000000001) - val-aux/sciknoweval/score/best@16/std:np.float64(0.016366340783720925) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.25117999999999996) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.027867680697389763) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.44438) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.08103965668240862) - val-core/sciknoweval/acc/mean@16:np.float64(0.42875) - val-aux/sciknoweval/acc/std@16:np.float64(0.1440962083400905) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.49282000000000004) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.11460243726780435) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.36628) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13169012287904225) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.4294) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.14357431870086687) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.53776) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07143174787390709) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.30992) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09737417027288235) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.43699999999999994) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12461646331648187) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.56108) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.03374430360648781) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.27224) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.058556923969906534) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.44081999999999993) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10054251250479307) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.5715600000000001) - val-core/sciknoweval/acc/best@16/std:np.float64(0.016366340783720925) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.25117999999999996) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.027867680697389763) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.44438) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.08103965668240862) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:7 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009639204363338649 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0009639204363338649 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:195.53125 - response_length/max:558.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:195.53125 - response_length_non_aborted/max:558.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.90625 - prompt_length/max:499.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7704983949661255e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.320506438612938) - timing_s/agent_loop/generate_sequences/max:np.float64(4.294556196779013) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.2014299611910246) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.294556196779013) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:558 - timing_s/gen:10.153399815782905 - timing_s/reward:0.05525374785065651 - timing_s/old_log_prob:2.262016834691167 - timing_s/adv:0.46920773200690746 - timing_s/update_actor:16.347947159782052 - timing_s/step:29.376724991947412 - timing_s/testing:21.447576792910695 - timing_s/save_checkpoint:6.84916939586401 - timing_s/stop_profile:0.00012588128447532654 - timing_per_token_ms/gen:0.2028408146033024 - timing_per_token_ms/update_actor:0.11707880113284957 - timing_per_token_ms/adv:0.0033603166323400613 - perf/total_num_tokens:139632 - perf/time_per_step:29.376724991947412 - perf/throughput:594.1438334186128 (TaskRunner pid=2122883) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2122883) "np.float64(0.42875), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2122883) "np.float64(0.1440962083400905), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2122883) "np.float64(0.49282000000000004), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2122883) "np.float64(0.11460243726780435), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2122883) "np.float64(0.36628), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2122883) "np.float64(0.13169012287904225), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2122883) "np.float64(0.4294), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2122883) "np.float64(0.14357431870086687), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2122883) "np.float64(0.53776), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2122883) "np.float64(0.07143174787390709), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2122883) "np.float64(0.30992), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2122883) Training Progress: 100%|██████████| 100/100 [1:14:23<00:00, 38.74s/it] (TaskRunner pid=2122883) (TaskRunner pid=2122883) "np.float64(0.09737417027288235), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2122883) "np.float64(0.43699999999999994), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2122883) "np.float64(0.12461646331648187), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2122883) "np.float64(0.56108), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2122883) "np.float64(0.03374430360648781), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2122883) "np.float64(0.27224), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2122883) "np.float64(0.058556923969906534), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2122883) "np.float64(0.44081999999999993), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "np.float64(0.10054251250479307), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2122883) "np.float64(0.5715600000000001), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2122883) 'np.float64(0.016366340783720925), ' (TaskRunner pid=2122883) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.25117999999999996), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.027867680697389763), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.44438), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.08103965668240862), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/mean@16': np.float64(0.42875), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/std@16': np.float64(0.1440962083400905), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.49282000000000004), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.11460243726780435), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.36628), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.13169012287904225), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.4294), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.14357431870086687), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.53776), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.07143174787390709), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.30992), " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.09737417027288235), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.43699999999999994), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.12461646331648187), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.56108), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.03374430360648781), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.27224), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.058556923969906534), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.44081999999999993), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.10054251250479307), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.5715600000000001), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.016366340783720925), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.25117999999999996), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.027867680697389763), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.44438), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.08103965668240862), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/mean@16': np.float64(0.42875), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/std@16': np.float64(0.1440962083400905), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.49282000000000004), " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.11460243726780435), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.36628), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.13169012287904225), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.4294), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.14357431870086687), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.53776), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.07143174787390709), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.30992), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.09737417027288235), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.43699999999999994), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.12461646331648187), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.56108), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.03374430360648781), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.27224), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.058556923969906534), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.44081999999999993), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.10054251250479307), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.5715600000000001), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.016366340783720925), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.25117999999999996), " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.027867680697389763), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.44438), " (TaskRunner pid=2122883) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.08103965668240862), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/std@16': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@2/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@2/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@2/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@4/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@4/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@8/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2122883) (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/worst@16/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2122883) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2122883) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2122883) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2122883) Training Progress: 100%|██████████| 100/100 [1:14:23<00:00, 44.64s/it] (TaskRunner pid=2122883) Traceback (most recent call last): (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in (TaskRunner pid=2122883) worker.main_loop() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop (TaskRunner pid=2122883) self.core_worker.run_task_loop() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/signal_handling.py", line 73, in handler (TaskRunner pid=2122883) _error_if_any_worker_fails() (TaskRunner pid=2122883) RuntimeError: DataLoader worker (pid 2143352) is killed by signal: Killed. (TaskRunner pid=2122883) Traceback (most recent call last): (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/workers/default_worker.py", line 322, in (TaskRunner pid=2122883) worker.main_loop() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py", line 1031, in main_loop (TaskRunner pid=2122883) self.core_worker.run_task_loop() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/signal_handling.py", line 73, in handler (TaskRunner pid=2122883) wandb: ERROR Problem finishing run (TaskRunner pid=2122883) Exception ignored in: (TaskRunner pid=2122883) Traceback (most recent call last): (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/verl/utils/tracking.py", line 169, in __del__ (TaskRunner pid=2122883) self.logger["wandb"].finish(exit_code=0) (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 4097, in finish (TaskRunner pid=2122883) wandb.run.finish(exit_code=exit_code, quiet=quiet) (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2122883) return func(self, *args, **kwargs) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 445, in wrapper (TaskRunner pid=2122883) return func(self, *args, **kwargs) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2275, in finish (TaskRunner pid=2122883) return self._finish(exit_code) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 400, in wrapper (TaskRunner pid=2122883) return func(self, *args, **kwargs) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2303, in _finish (TaskRunner pid=2122883) self._atexit_cleanup(exit_code=exit_code) (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2498, in _atexit_cleanup (TaskRunner pid=2122883) self._on_finish() (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/wandb_run.py", line 2711, in _on_finish (TaskRunner pid=2122883) exit_handle = self._backend.interface.deliver_exit(self._exit_code) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface.py", line 1038, in deliver_exit (TaskRunner pid=2122883) return self._deliver_exit(exit_data) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_shared.py", line 460, in _deliver_exit (TaskRunner pid=2122883) return self._deliver(record) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_sock.py", line 50, in _deliver (TaskRunner pid=2122883) return self._asyncer.run(lambda: self.deliver_async(record)) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2122883) return future.result() (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2122883) return self.__get_result() (TaskRunner pid=2122883) (TaskRunner pid=2122883) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2122883) raise self._exception (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2122883) return await fn() (TaskRunner pid=2122883) ^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/interface/interface_sock.py", line 58, in deliver_async (TaskRunner pid=2122883) handle = await self._client.deliver(request) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 53, in deliver (TaskRunner pid=2122883) handle = self._mailbox.require_response(request) (TaskRunner pid=2122883) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2122883) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/mailbox/mailbox.py", line 76, in require_response (TaskRunner pid=2122883) raise MailboxClosedError() (TaskRunner pid=2122883) wandb.sdk.mailbox.mailbox.MailboxClosedError: main_ppo exit code: 0 [2026-07-02 12:53:36] Finished biology rlsd_tr [2026-07-02 12:53:36] Starting material grpo Experiment: qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: material Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet Ray tmp: /tmp/ray_q3g_material_grpo_Qwen3_4B 2026-07-02 12:53:38,592 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/material Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_material_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/material +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_material_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/material', 'EXPERIMENT': 'qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 12:53:48,296 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2146199) TaskRunner hostname: mole-workspace-rw, PID: 2146199 (TaskRunner pid=2146199) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2146199) 'calculate_entropy': False, (TaskRunner pid=2146199) 'calculate_sum_pi_squared': False, (TaskRunner pid=2146199) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2146199) 'async_save': False, (TaskRunner pid=2146199) 'load_contents': ['model', (TaskRunner pid=2146199) 'optimizer', (TaskRunner pid=2146199) 'extra'], (TaskRunner pid=2146199) 'save_contents': ['model', (TaskRunner pid=2146199) 'optimizer', (TaskRunner pid=2146199) 'extra']}, (TaskRunner pid=2146199) 'clip_ratio': 0.2, (TaskRunner pid=2146199) 'clip_ratio_c': 3.0, (TaskRunner pid=2146199) 'clip_ratio_high': 0.28, (TaskRunner pid=2146199) 'clip_ratio_low': 0.2, (TaskRunner pid=2146199) 'data_loader_seed': 42, (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_coeff': 0, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'freeze_vision_tower': False, (TaskRunner pid=2146199) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'forward_only': False, (TaskRunner pid=2146199) 'forward_prefetch': False, (TaskRunner pid=2146199) 'fsdp_size': -1, (TaskRunner pid=2146199) 'full_determinism': False, (TaskRunner pid=2146199) 'model_dtype': 'fp32', (TaskRunner pid=2146199) 'offload_policy': False, (TaskRunner pid=2146199) 'optimizer_offload': False, (TaskRunner pid=2146199) 'param_offload': False, (TaskRunner pid=2146199) 'reshard_after_forward': True, (TaskRunner pid=2146199) 'seed': 42, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_orig_params': False, (TaskRunner pid=2146199) 'use_torch_compile': True, (TaskRunner pid=2146199) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2146199) 'grad_clip': 1.0, (TaskRunner pid=2146199) 'kl_loss_coef': 0.0, (TaskRunner pid=2146199) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2146199) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2146199) 'loss_scale_factor': None, (TaskRunner pid=2146199) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2146199) 'betas': [0.9, 0.999], (TaskRunner pid=2146199) 'clip_grad': 1.0, (TaskRunner pid=2146199) 'lr': 1e-06, (TaskRunner pid=2146199) 'lr_scheduler_type': 'constant', (TaskRunner pid=2146199) 'lr_warmup_steps': 10, (TaskRunner pid=2146199) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2146199) 'min_lr_ratio': 0.0, (TaskRunner pid=2146199) 'num_cycles': 0.5, (TaskRunner pid=2146199) 'optimizer': 'AdamW', (TaskRunner pid=2146199) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2146199) 'override_optimizer_config': None, (TaskRunner pid=2146199) 'total_training_steps': -1, (TaskRunner pid=2146199) 'warmup_style': None, (TaskRunner pid=2146199) 'weight_decay': 0.01}, (TaskRunner pid=2146199) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2146199) 'clip_cov_lb': 1.0, (TaskRunner pid=2146199) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2146199) 'clip_cov_ub': 5.0, (TaskRunner pid=2146199) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2146199) 'loss_mode': 'vanilla', (TaskRunner pid=2146199) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2146199) 'ppo_epochs': 1, (TaskRunner pid=2146199) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2146199) 'ppo_micro_batch_size': None, (TaskRunner pid=2146199) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2146199) 'ppo_mini_batch_size': 8, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'rollout_n': 8, (TaskRunner pid=2146199) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2146199) 'mode': 'disabled', (TaskRunner pid=2146199) 'record_file': None, (TaskRunner pid=2146199) 'replay_file': None}, (TaskRunner pid=2146199) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2146199) 'alpha': 0.5, (TaskRunner pid=2146199) 'distillation_add_tail': True, (TaskRunner pid=2146199) 'distillation_topk': 100, (TaskRunner pid=2146199) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2146199) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2146199) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'loss_weight': 0.0, (TaskRunner pid=2146199) 'mask': 'all'}, (TaskRunner pid=2146199) 'feedback_template': '\n' (TaskRunner pid=2146199) 'The ' (TaskRunner pid=2146199) 'following ' (TaskRunner pid=2146199) 'is ' (TaskRunner pid=2146199) 'feedback ' (TaskRunner pid=2146199) 'from ' (TaskRunner pid=2146199) 'your ' (TaskRunner pid=2146199) 'unsuccessful ' (TaskRunner pid=2146199) 'earlier ' (TaskRunner pid=2146199) 'attempt:\n' (TaskRunner pid=2146199) '\n' (TaskRunner pid=2146199) '{feedback_raw}', (TaskRunner pid=2146199) 'full_logit_distillation': True, (TaskRunner pid=2146199) 'include_environment_feedback': True, (TaskRunner pid=2146199) 'is_clip': 2, (TaskRunner pid=2146199) 'max_reprompt_len': 22528, (TaskRunner pid=2146199) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2146199) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2146199) '\n' (TaskRunner pid=2146199) 'Correctly ' (TaskRunner pid=2146199) 'solve ' (TaskRunner pid=2146199) 'the ' (TaskRunner pid=2146199) 'original ' (TaskRunner pid=2146199) 'question.', (TaskRunner pid=2146199) 'reprompt_truncation': 'right', (TaskRunner pid=2146199) 'solution_template': '\n' (TaskRunner pid=2146199) 'Correct ' (TaskRunner pid=2146199) 'solution:\n' (TaskRunner pid=2146199) '\n' (TaskRunner pid=2146199) '{successful_previous_attempt}', (TaskRunner pid=2146199) 'srpo_dynamic_weighting': False, (TaskRunner pid=2146199) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2146199) 'success_reward_threshold': 0.5, (TaskRunner pid=2146199) 'teacher_regularization': 'ema', (TaskRunner pid=2146199) 'teacher_update_rate': 0.0, (TaskRunner pid=2146199) 'token_reweight_decay_steps': None, (TaskRunner pid=2146199) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2146199) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2146199) 'shuffle': False, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2146199) 'tau_neg': 1.05, (TaskRunner pid=2146199) 'tau_pos': 1.0, (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_dynamic_bsz': False, (TaskRunner pid=2146199) 'use_fused_kernels': False, (TaskRunner pid=2146199) 'use_kl_loss': False, (TaskRunner pid=2146199) 'use_prefix_grouper': False, (TaskRunner pid=2146199) 'use_remove_padding': True, (TaskRunner pid=2146199) 'use_torch_compile': True}, (TaskRunner pid=2146199) 'hybrid_engine': True, (TaskRunner pid=2146199) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2146199) 'custom_chat_template': None, (TaskRunner pid=2146199) 'enable_activation_offload': False, (TaskRunner pid=2146199) 'enable_gradient_checkpointing': True, (TaskRunner pid=2146199) 'exclude_modules': None, (TaskRunner pid=2146199) 'external_lib': None, (TaskRunner pid=2146199) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2146199) 'hf_config_path': None, (TaskRunner pid=2146199) 'lora_adapter_path': None, (TaskRunner pid=2146199) 'lora_alpha': 16, (TaskRunner pid=2146199) 'lora_rank': 0, (TaskRunner pid=2146199) 'override_config': {}, (TaskRunner pid=2146199) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2146199) 'target_modules': 'all-linear', (TaskRunner pid=2146199) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2146199) 'num_shards': 4}, (TaskRunner pid=2146199) 'tokenizer_path': None, (TaskRunner pid=2146199) 'trust_remote_code': True, (TaskRunner pid=2146199) 'use_fused_kernels': False, (TaskRunner pid=2146199) 'use_liger': False, (TaskRunner pid=2146199) 'use_remove_padding': True, (TaskRunner pid=2146199) 'use_shm': False}, (TaskRunner pid=2146199) 'nccl_timeout': 600, (TaskRunner pid=2146199) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'forward_only': True, (TaskRunner pid=2146199) 'forward_prefetch': False, (TaskRunner pid=2146199) 'fsdp_size': -1, (TaskRunner pid=2146199) 'full_determinism': False, (TaskRunner pid=2146199) 'model_dtype': 'fp32', (TaskRunner pid=2146199) 'offload_policy': False, (TaskRunner pid=2146199) 'optimizer_offload': False, (TaskRunner pid=2146199) 'param_offload': False, (TaskRunner pid=2146199) 'reshard_after_forward': True, (TaskRunner pid=2146199) 'seed': 42, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_orig_params': False, (TaskRunner pid=2146199) 'use_torch_compile': True, (TaskRunner pid=2146199) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2146199) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2146199) 'log_prob_micro_batch_size': None, (TaskRunner pid=2146199) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2146199) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'rollout_n': 8, (TaskRunner pid=2146199) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2146199) 'mode': 'disabled', (TaskRunner pid=2146199) 'record_file': None, (TaskRunner pid=2146199) 'replay_file': None}, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_torch_compile': True}, (TaskRunner pid=2146199) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2146199) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2146199) 'agent_loop_config_path': None, (TaskRunner pid=2146199) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2146199) 'name': None, (TaskRunner pid=2146199) 'path': None}, (TaskRunner pid=2146199) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2146199) 'num_workers': 8}, (TaskRunner pid=2146199) 'calculate_log_probs': True, (TaskRunner pid=2146199) 'cudagraph_capture_sizes': None, (TaskRunner pid=2146199) 'data_parallel_size': 1, (TaskRunner pid=2146199) 'disable_log_stats': True, (TaskRunner pid=2146199) 'do_sample': True, (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'enable_chunked_prefill': True, (TaskRunner pid=2146199) 'enable_prefix_caching': True, (TaskRunner pid=2146199) 'enable_rollout_routing_replay': False, (TaskRunner pid=2146199) 'enforce_eager': False, (TaskRunner pid=2146199) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2146199) 'expert_parallel_size': 1, (TaskRunner pid=2146199) 'free_cache_engine': True, (TaskRunner pid=2146199) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2146199) 'ignore_eos': False, (TaskRunner pid=2146199) 'layered_summon': False, (TaskRunner pid=2146199) 'load_format': 'dummy', (TaskRunner pid=2146199) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2146199) 'log_prob_micro_batch_size': None, (TaskRunner pid=2146199) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2146199) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2146199) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2146199) 'max_model_len': 10240, (TaskRunner pid=2146199) 'max_num_batched_tokens': 10240, (TaskRunner pid=2146199) 'max_num_seqs': 1024, (TaskRunner pid=2146199) 'mode': 'async', (TaskRunner pid=2146199) 'multi_stage_wake_up': False, (TaskRunner pid=2146199) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'format': 'hermes', (TaskRunner pid=2146199) 'interaction_config_path': None, (TaskRunner pid=2146199) 'max_assistant_turns': None, (TaskRunner pid=2146199) 'max_parallel_calls': 1, (TaskRunner pid=2146199) 'max_tool_response_length': 256, (TaskRunner pid=2146199) 'max_user_turns': None, (TaskRunner pid=2146199) 'num_repeat_rollouts': None, (TaskRunner pid=2146199) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2146199) 'tool_config_path': None, (TaskRunner pid=2146199) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2146199) 'use_inference_chat_template': False}, (TaskRunner pid=2146199) 'n': 8, (TaskRunner pid=2146199) 'name': 'vllm', (TaskRunner pid=2146199) 'over_sample_rate': 0, (TaskRunner pid=2146199) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2146199) 'port': 9090, (TaskRunner pid=2146199) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2146199) 'prompt_length': 2048, (TaskRunner pid=2146199) 'quantization': None, (TaskRunner pid=2146199) 'quantization_config_file': None, (TaskRunner pid=2146199) 'response_length': 8192, (TaskRunner pid=2146199) 'scheduling_policy': 'fcfs', (TaskRunner pid=2146199) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2146199) 'skip_rollout': False, (TaskRunner pid=2146199) 'skip_tokenizer_init': True, (TaskRunner pid=2146199) 'temperature': 1.0, (TaskRunner pid=2146199) 'tensor_model_parallel_size': 2, (TaskRunner pid=2146199) 'top_k': -1, (TaskRunner pid=2146199) 'top_p': 1.0, (TaskRunner pid=2146199) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2146199) 'backend': None, (TaskRunner pid=2146199) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2146199) 'token2text': False}, (TaskRunner pid=2146199) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2146199) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2146199) 'do_sample': True, (TaskRunner pid=2146199) 'n': 16, (TaskRunner pid=2146199) 'temperature': 0.6, (TaskRunner pid=2146199) 'top_k': -1, (TaskRunner pid=2146199) 'top_p': 0.95}}}, (TaskRunner pid=2146199) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2146199) 'adv_estimator': 'grpo', (TaskRunner pid=2146199) 'gamma': 1.0, (TaskRunner pid=2146199) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2146199) 'horizon': 10000, (TaskRunner pid=2146199) 'kl_coef': 0.001, (TaskRunner pid=2146199) 'target_kl': 0.1, (TaskRunner pid=2146199) 'type': 'fixed'}, (TaskRunner pid=2146199) 'kl_penalty': 'kl', (TaskRunner pid=2146199) 'lam': 1.0, (TaskRunner pid=2146199) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2146199) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2146199) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2146199) 'loss_type': 'ppo_clip', (TaskRunner pid=2146199) 'rollout_is': 'token', (TaskRunner pid=2146199) 'rollout_is_batch_normalize': False, (TaskRunner pid=2146199) 'rollout_is_threshold': 2.0, (TaskRunner pid=2146199) 'rollout_rs': None, (TaskRunner pid=2146199) 'rollout_rs_threshold': None}, (TaskRunner pid=2146199) 'use_kl_in_reward': False, (TaskRunner pid=2146199) 'use_pf_ppo': False}, (TaskRunner pid=2146199) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2146199) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2146199) 'async_save': False, (TaskRunner pid=2146199) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2146199) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2146199) 'cliprange_value': 0.5, (TaskRunner pid=2146199) 'data_loader_seed': 42, (TaskRunner pid=2146199) 'enable': None, (TaskRunner pid=2146199) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2146199) 'forward_micro_batch_size': None, (TaskRunner pid=2146199) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2146199) 'grad_clip': 1.0, (TaskRunner pid=2146199) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2146199) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2146199) 'enable_activation_offload': False, (TaskRunner pid=2146199) 'enable_gradient_checkpointing': True, (TaskRunner pid=2146199) 'external_lib': None, (TaskRunner pid=2146199) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'entropy_checkpointing': False, (TaskRunner pid=2146199) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2146199) 'forward_only': False, (TaskRunner pid=2146199) 'forward_prefetch': False, (TaskRunner pid=2146199) 'fsdp_size': -1, (TaskRunner pid=2146199) 'full_determinism': False, (TaskRunner pid=2146199) 'model_dtype': 'fp32', (TaskRunner pid=2146199) 'offload_policy': False, (TaskRunner pid=2146199) 'optimizer_offload': False, (TaskRunner pid=2146199) 'param_offload': False, (TaskRunner pid=2146199) 'reshard_after_forward': True, (TaskRunner pid=2146199) 'seed': 42, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_orig_params': False, (TaskRunner pid=2146199) 'use_torch_compile': True, (TaskRunner pid=2146199) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2146199) 'lora_alpha': 16, (TaskRunner pid=2146199) 'lora_rank': 0, (TaskRunner pid=2146199) 'override_config': {}, (TaskRunner pid=2146199) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2146199) 'target_modules': 'all-linear', (TaskRunner pid=2146199) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2146199) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2146199) 'trust_remote_code': True, (TaskRunner pid=2146199) 'use_remove_padding': False, (TaskRunner pid=2146199) 'use_shm': False}, (TaskRunner pid=2146199) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2146199) 'betas': [0.9, 0.999], (TaskRunner pid=2146199) 'clip_grad': 1.0, (TaskRunner pid=2146199) 'lr': 1e-05, (TaskRunner pid=2146199) 'lr_scheduler_type': 'constant', (TaskRunner pid=2146199) 'lr_warmup_steps': -1, (TaskRunner pid=2146199) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2146199) 'min_lr_ratio': 0.0, (TaskRunner pid=2146199) 'num_cycles': 0.5, (TaskRunner pid=2146199) 'optimizer': 'AdamW', (TaskRunner pid=2146199) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2146199) 'override_optimizer_config': None, (TaskRunner pid=2146199) 'total_training_steps': -1, (TaskRunner pid=2146199) 'warmup_style': None, (TaskRunner pid=2146199) 'weight_decay': 0.01}, (TaskRunner pid=2146199) 'ppo_epochs': 1, (TaskRunner pid=2146199) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2146199) 'ppo_micro_batch_size': None, (TaskRunner pid=2146199) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2146199) 'ppo_mini_batch_size': 8, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'rollout_n': 8, (TaskRunner pid=2146199) 'shuffle': False, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_dynamic_bsz': False}, (TaskRunner pid=2146199) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2146199) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2146199) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2146199) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2146199) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2146199) 'dataloader_num_workers': 8, (TaskRunner pid=2146199) 'filter_overlong_prompts': True, (TaskRunner pid=2146199) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2146199) 'image_key': 'images', (TaskRunner pid=2146199) 'image_patch_size': 14, (TaskRunner pid=2146199) 'max_prompt_length': 2048, (TaskRunner pid=2146199) 'max_response_length': 8192, (TaskRunner pid=2146199) 'prompt_key': 'prompt', (TaskRunner pid=2146199) 'return_full_prompt': False, (TaskRunner pid=2146199) 'return_multi_modal_inputs': True, (TaskRunner pid=2146199) 'return_raw_chat': True, (TaskRunner pid=2146199) 'return_raw_input_ids': False, (TaskRunner pid=2146199) 'reward_fn_key': 'data_source', (TaskRunner pid=2146199) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2146199) 'seed': None, (TaskRunner pid=2146199) 'shuffle': True, (TaskRunner pid=2146199) 'tokenizer': None, (TaskRunner pid=2146199) 'tool_config_path': None, (TaskRunner pid=2146199) 'train_batch_size': 32, (TaskRunner pid=2146199) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet'], (TaskRunner pid=2146199) 'train_max_samples': 3200, (TaskRunner pid=2146199) 'truncation': 'error', (TaskRunner pid=2146199) 'trust_remote_code': True, (TaskRunner pid=2146199) 'use_shm': False, (TaskRunner pid=2146199) 'val_batch_size': None, (TaskRunner pid=2146199) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet'], (TaskRunner pid=2146199) 'val_max_samples': -1, (TaskRunner pid=2146199) 'validation_shuffle': False, (TaskRunner pid=2146199) 'video_key': 'videos'}, (TaskRunner pid=2146199) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2146199) 'cuda-memory-usage': 'true', (TaskRunner pid=2146199) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2146199) 'capture-range-end': None, (TaskRunner pid=2146199) 'cuda-graph-trace': 'graph', (TaskRunner pid=2146199) 'cuda-memory-usage': 'true', (TaskRunner pid=2146199) 'kill': 'none', (TaskRunner pid=2146199) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2146199) 'torch_memory': {'context': 'all', (TaskRunner pid=2146199) 'kw_args': {}, (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'stacks': 'all', (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2146199) 'profile_continuous_steps': False, (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'steps': None, (TaskRunner pid=2146199) 'tool': None}, (TaskRunner pid=2146199) 'max_model_len': 10240, (TaskRunner pid=2146199) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_material_grpo_Qwen3_4B', (TaskRunner pid=2146199) 'include_dashboard': False, (TaskRunner pid=2146199) 'num_cpus': None, (TaskRunner pid=2146199) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2146199) 'TASK': 'datasets/sciknoweval/material', (TaskRunner pid=2146199) 'USER': 'root'}}}, (TaskRunner pid=2146199) 'timeline_json_file': None}, (TaskRunner pid=2146199) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2146199) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2146199) 'name': 'custom_reward_manager', (TaskRunner pid=2146199) 'path': None}, (TaskRunner pid=2146199) 'name': 'naive', (TaskRunner pid=2146199) 'source': 'register'}, (TaskRunner pid=2146199) 'reward_model': {'enable': False, (TaskRunner pid=2146199) 'enable_resource_pool': False, (TaskRunner pid=2146199) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2146199) 'launch_reward_fn_async': False, (TaskRunner pid=2146199) 'max_length': None, (TaskRunner pid=2146199) 'micro_batch_size': None, (TaskRunner pid=2146199) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2146199) 'model': {'external_lib': None, (TaskRunner pid=2146199) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2146199) 'forward_prefetch': False, (TaskRunner pid=2146199) 'fsdp_size': -1, (TaskRunner pid=2146199) 'param_offload': False, (TaskRunner pid=2146199) 'reshard_after_forward': True, (TaskRunner pid=2146199) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2146199) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2146199) 'override_config': {}, (TaskRunner pid=2146199) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2146199) 'trust_remote_code': False, (TaskRunner pid=2146199) 'use_fused_kernels': False, (TaskRunner pid=2146199) 'use_remove_padding': False, (TaskRunner pid=2146199) 'use_shm': False}, (TaskRunner pid=2146199) 'n_gpus_per_node': 8, (TaskRunner pid=2146199) 'nnodes': 0, (TaskRunner pid=2146199) 'num_workers': 1, (TaskRunner pid=2146199) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2146199) 'all_ranks': False, (TaskRunner pid=2146199) 'enable': False, (TaskRunner pid=2146199) 'ranks': [], (TaskRunner pid=2146199) 'save_path': 'outputs/profile', (TaskRunner pid=2146199) 'tool': None, (TaskRunner pid=2146199) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2146199) 'analysis': True, (TaskRunner pid=2146199) 'contents': [], (TaskRunner pid=2146199) 'discrete': False, (TaskRunner pid=2146199) 'level': 'level0'}, (TaskRunner pid=2146199) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2146199) 'discrete': False}, (TaskRunner pid=2146199) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2146199) 'step_end': None, (TaskRunner pid=2146199) 'step_start': 0}, (TaskRunner pid=2146199) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2146199) 'stack_depth': 32, (TaskRunner pid=2146199) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2146199) 'reward_loop_class_name': None, (TaskRunner pid=2146199) 'reward_loop_module_path': None, (TaskRunner pid=2146199) 'reward_loop_source': 'register', (TaskRunner pid=2146199) 'reward_manager': 'naive', (TaskRunner pid=2146199) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2146199) 'cudagraph_capture_sizes': None, (TaskRunner pid=2146199) 'data_parallel_size': 1, (TaskRunner pid=2146199) 'disable_log_stats': True, (TaskRunner pid=2146199) 'dtype': 'bfloat16', (TaskRunner pid=2146199) 'enable_chunked_prefill': True, (TaskRunner pid=2146199) 'enable_prefix_caching': True, (TaskRunner pid=2146199) 'enforce_eager': True, (TaskRunner pid=2146199) 'engine_kwargs': {}, (TaskRunner pid=2146199) 'expert_parallel_size': 1, (TaskRunner pid=2146199) 'free_cache_engine': True, (TaskRunner pid=2146199) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2146199) 'limit_images': None, (TaskRunner pid=2146199) 'load_format': 'auto', (TaskRunner pid=2146199) 'max_model_len': None, (TaskRunner pid=2146199) 'max_num_batched_tokens': 8192, (TaskRunner pid=2146199) 'max_num_seqs': 1024, (TaskRunner pid=2146199) 'name': '???', (TaskRunner pid=2146199) 'prompt_length': 2048, (TaskRunner pid=2146199) 'response_length': 2048, (TaskRunner pid=2146199) 'skip_tokenizer_init': False, (TaskRunner pid=2146199) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2146199) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2146199) 'memory_limit_mb': 1024, (TaskRunner pid=2146199) 'url': None}, (TaskRunner pid=2146199) 'strategy': 'fsdp', (TaskRunner pid=2146199) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2146199) 'use_dynamic_bsz': False, (TaskRunner pid=2146199) 'use_reward_loop': False}, (TaskRunner pid=2146199) 'trainer': {'balance_batch': True, (TaskRunner pid=2146199) 'critic_warmup': 0, (TaskRunner pid=2146199) 'default_hdfs_dir': None, (TaskRunner pid=2146199) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2146199) 'del_local_ckpt_after_load': False, (TaskRunner pid=2146199) 'device': 'cuda', (TaskRunner pid=2146199) 'esi_redundant_time': 0, (TaskRunner pid=2146199) 'experiment_name': 'qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2146199) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2146199) 'log_val_generations': 0, (TaskRunner pid=2146199) 'logger': ['console', 'wandb'], (TaskRunner pid=2146199) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2146199) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2146199) 'n_gpus_per_node': 8, (TaskRunner pid=2146199) 'nnodes': 1, (TaskRunner pid=2146199) 'project_name': 'SDPO-root', (TaskRunner pid=2146199) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2146199) 'resume_from_path': None, (TaskRunner pid=2146199) 'resume_mode': 'auto', (TaskRunner pid=2146199) 'rollout_data_dir': None, (TaskRunner pid=2146199) 'save_freq': 10, (TaskRunner pid=2146199) 'test_freq': 10, (TaskRunner pid=2146199) 'total_epochs': 30, (TaskRunner pid=2146199) 'total_training_steps': 100, (TaskRunner pid=2146199) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2146199) 'val_before_train': False, (TaskRunner pid=2146199) 'val_only': False, (TaskRunner pid=2146199) 'validation_data_dir': None}, (TaskRunner pid=2146199) 'transfer_queue': {'enable': False}, (TaskRunner pid=2146199) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/material', (TaskRunner pid=2146199) 'dir': '/users/root/SDPO', (TaskRunner pid=2146199) 'log_dir': '/users/root/output', (TaskRunner pid=2146199) 'task': 'datasets/sciknoweval/material'}} (TaskRunner pid=2146199) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2146199) use_critic=need_critic(config), (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2146199) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2146199) Using dataset class: RLHFDataset (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2146199) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2146199) dataset len: 841 (TaskRunner pid=2146199) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2146199) WARNING:2026-07-02 12:53:57,322:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2146199) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/841 [00:00 (TaskRunner pid=2146199) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2146199) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2146199) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2146199) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2146199) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2146199) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 94/94 [00:00<00:00, 149.22 examples/s] (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2146199) self.use_critic = need_critic(self.config) (WorkerDict pid=2146579) [W702 12:54:06.660994833 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:50537 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2146579) [Gloo] Rank 2 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2146579) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2146579) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2146583) [W702 12:54:06.312883846 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:50537 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2146577) Model config after override: Qwen3Config { (WorkerDict pid=2146577) "architectures": [ (WorkerDict pid=2146577) "Qwen3ForCausalLM" (WorkerDict pid=2146577) ], (WorkerDict pid=2146577) "attention_bias": false, (WorkerDict pid=2146577) "attention_dropout": 0.0, (WorkerDict pid=2146577) "dtype": "bfloat16", (WorkerDict pid=2146577) "eos_token_id": 151645, (WorkerDict pid=2146577) "head_dim": 128, (WorkerDict pid=2146577) "hidden_act": "silu", (WorkerDict pid=2146577) "hidden_size": 2560, (WorkerDict pid=2146577) "initializer_range": 0.02, (WorkerDict pid=2146577) "intermediate_size": 9728, (WorkerDict pid=2146577) "layer_types": [ (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention", (WorkerDict pid=2146577) "full_attention" (WorkerDict pid=2146577) ], (WorkerDict pid=2146577) "max_position_embeddings": 40960, (WorkerDict pid=2146577) "max_window_layers": 36, (WorkerDict pid=2146577) "model_type": "qwen3", (WorkerDict pid=2146577) "num_attention_heads": 32, (WorkerDict pid=2146577) "num_hidden_layers": 36, (WorkerDict pid=2146577) "num_key_value_heads": 8, (WorkerDict pid=2146577) "pad_token_id": 151643, (WorkerDict pid=2146577) "rms_norm_eps": 1e-06, (WorkerDict pid=2146577) "rope_scaling": null, (WorkerDict pid=2146577) "rope_theta": 1000000, (WorkerDict pid=2146577) "sliding_window": null, (WorkerDict pid=2146577) "tie_word_embeddings": true, (WorkerDict pid=2146577) "transformers_version": "4.57.1", (WorkerDict pid=2146577) "use_cache": true, (WorkerDict pid=2146577) "use_sliding_window": false, (WorkerDict pid=2146577) "vocab_size": 151936 (WorkerDict pid=2146577) } (WorkerDict pid=2146577) (WorkerDict pid=2146579) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2146579) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2146579) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2146577) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2146577) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2146579) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2146579) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2146584) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.91s/it] [repeated 13x across cluster] (WorkerDict pid=2146581) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.94s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.98s/it] (WorkerDict pid=2146584) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it] [repeated 5x across cluster] (WorkerDict pid=2146577) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2146577) Actor use_remove_padding=True (WorkerDict pid=2146577) Actor use_fused_kernels=False (WorkerDict pid=2146577) Actor use_prefix_grouper=False (WorkerDict pid=2146581) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2146581) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2146579) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2146579) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2146578) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2146581) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2146581) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2146579) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2146579) warnings.warn( (WorkerDict pid=2146581) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2146581) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2146199) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2146199) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2146584) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2146584) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2147446) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2147446) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2147445) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2147445) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2147446) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2147446) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2147445) ['serve', (vLLMHttpServer pid=2147445) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2147445) '--dtype', (vLLMHttpServer pid=2147445) 'bfloat16', (vLLMHttpServer pid=2147445) '--load_format', (vLLMHttpServer pid=2147445) 'dummy', (vLLMHttpServer pid=2147445) '--trust_remote_code', (vLLMHttpServer pid=2147445) '--max_model_len', (vLLMHttpServer pid=2147445) '40960', (vLLMHttpServer pid=2147445) '--max_num_seqs', (vLLMHttpServer pid=2147445) '1024', (vLLMHttpServer pid=2147445) '--enable_chunked_prefill', (vLLMHttpServer pid=2147445) '--max_num_batched_tokens', (vLLMHttpServer pid=2147445) '10240', (vLLMHttpServer pid=2147445) '--enable_prefix_caching', (vLLMHttpServer pid=2147445) '--enable_sleep_mode', (vLLMHttpServer pid=2147445) '--logprobs_mode', (vLLMHttpServer pid=2147445) 'processed_logprobs', (vLLMHttpServer pid=2147445) '--disable_custom_all_reduce', (vLLMHttpServer pid=2147445) '--gpu_memory_utilization', (vLLMHttpServer pid=2147445) '0.8', (vLLMHttpServer pid=2147445) '--disable_log_stats', (vLLMHttpServer pid=2147445) '--tensor_parallel_size', (vLLMHttpServer pid=2147445) '2', (vLLMHttpServer pid=2147445) '--seed', (vLLMHttpServer pid=2147445) '0', (vLLMHttpServer pid=2147445) '--override_generation_config', (vLLMHttpServer pid=2147445) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2147445) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2147445) '--hf_overrides', (vLLMHttpServer pid=2147445) '{}', (vLLMHttpServer pid=2147445) '--scheduling_policy', (vLLMHttpServer pid=2147445) 'fcfs'] (WorkerDict pid=2146583) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2147446) WARNING 07-02 12:54:49 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2146579) WARNING 07-02 12:54:57 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2147448) WARNING 07-02 12:54:49 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2146579) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.770000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/aotautograd/ayamvxbmfo3ie54kyeiv7wcyjafynjwj3y4mmjujg4hvg4mrrnne/.2146580.130521564378816.tmp' (vLLMHttpServer pid=2147448) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2147448) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2147448) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2147448) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2146578) [rank1]:W0702 12:55:06.925000 2146578 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2146582.128752478582464.tmp' (WorkerDict pid=2146579) 2026-07-02 12:55:11,940 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2146579) 2026-07-02 12:55:11,960 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2146579) [rank2]:W0702 12:55:07.327000 2146579 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2146580.130521564378816.tmp' (WorkerDict pid=2146577) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) The volume of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = a \times b \times c $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given the lattice parameters: (TaskRunner pid=2146199) - $ a = 7.82 $ Å (TaskRunner pid=2146199) - $ b = 7.82 $ Å (TaskRunner pid=2146199) - $ c = 17.88 $ Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Since all angles (α, β, γ) are 90.00º, the crystal structure is cubic, and the volume is simply the product of the three lattice parameters. (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Then multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 61.1524 \times 17.88 \approx 1094.31 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2146199) step:10 - global_seqlen/min:19757 - global_seqlen/max:25941 - global_seqlen/minmax_diff:6184 - global_seqlen/balanced_min:23088 - global_seqlen/balanced_max:23091 - global_seqlen/mean:23089.5 - actor/entropy:0.046847615391016006 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.999447226524353 - training/rollout_probs_diff_mean:0.0027974338736385107 - training/rollout_probs_diff_std:0.015953663736581802 - training/rollout_actor_probs_pearson_corr:0.9908416271209717 - rollout_corr/rollout_is_mean:1.000261902809143 - rollout_corr/rollout_is_ratio_fraction_high:0.00017077336087822914 - rollout_corr/rollout_is_ratio_fraction_low:0.0003415467217564583 - rollout_corr/rollout_is_max:3.556555986404419 - rollout_corr/rollout_is_min:0.0005527785397134721 - rollout_corr/rollout_is_std:0.03943456709384918 - rollout_corr/rollout_is_eff_sample_size:0.9984481612293591 - rollout_corr/rollout_is_seq_mean:1.0002124309539795 - rollout_corr/rollout_is_seq_std:0.0021660649217665195 - rollout_corr/rollout_is_seq_max:1.0074161291122437 - rollout_corr/rollout_is_seq_min:0.9950547218322754 - rollout_corr/rollout_is_seq_max_deviation:0.007416129112243652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0476945461705327) - rollout_corr/training_log_ppl:np.float64(0.04598778319450503) - rollout_corr/kl:np.float64(0.0008644373356485602) - rollout_corr/k3_kl:np.float64(0.001017599147822068) - rollout_corr/rollout_ppl:np.float64(1.0467740190215409) - rollout_corr/rollout_log_ppl:np.float64(0.04512334530954831) - rollout_corr/log_ppl_diff:np.float64(0.0008644378849567147) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019693393714987906) - rollout_corr/log_ppl_diff_max:np.float64(0.026914089918136597) - rollout_corr/log_ppl_diff_min:np.float64(-0.007021835073828697) - rollout_corr/ppl_ratio:np.float64(1.0008692720439285) - rollout_corr/chi2_token:np.float64(0.0024165299255400896) - rollout_corr/chi2_seq:np.float64(1.1769734171684831) - actor/pg_loss:np.float64(4.175701178610325e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004576832020575239) - actor/ppo_kl:np.float64(0.00016489524013607593) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.30771495401859283) - perf/mfu/actor:np.float64(0.06710769780779227) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(98.84016799926758) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6688829787234043) - val-aux/sciknoweval/reward/std@16:np.float64(0.11328073215884149) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7156382978723406) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.0834642720784752) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6216595744680851) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.10733205509043674) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6679893617021277) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.11290679974600436) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7462021276595743) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.051777462488278976) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5769893617021277) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.089712059129489) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6791702127659575) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.08965742400306344) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7647765957446808) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.03263714213393466) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5384255319148936) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06250559596458083) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.68581914893617) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.06258938352611139) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7768936170212765) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.01940928498500347) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5148829787234042) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03347496380735761) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6902234042553191) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04116517145682157) - val-aux/sciknoweval/score/mean@16:np.float64(0.6688829787234043) - val-aux/sciknoweval/score/std@16:np.float64(0.11328073215884149) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7156382978723406) - val-aux/sciknoweval/score/best@2/std:np.float64(0.0834642720784752) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6216595744680851) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.10733205509043674) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6679893617021277) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.11290679974600436) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7462021276595743) - val-aux/sciknoweval/score/best@4/std:np.float64(0.051777462488278976) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5769893617021277) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.089712059129489) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6791702127659575) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.08965742400306344) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7647765957446808) - val-aux/sciknoweval/score/best@8/std:np.float64(0.03263714213393466) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5384255319148936) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06250559596458083) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.68581914893617) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.06258938352611139) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7768936170212765) - val-aux/sciknoweval/score/best@16/std:np.float64(0.01940928498500347) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5148829787234042) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03347496380735761) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6902234042553191) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04116517145682157) - val-core/sciknoweval/acc/mean@16:np.float64(0.6688829787234043) - val-aux/sciknoweval/acc/std@16:np.float64(0.11328073215884149) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7156382978723406) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.0834642720784752) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6216595744680851) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.10733205509043674) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6679893617021277) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.11290679974600436) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7462021276595743) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.051777462488278976) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5769893617021277) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.089712059129489) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6791702127659575) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.08965742400306344) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7647765957446808) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.03263714213393466) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5384255319148936) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06250559596458083) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.68581914893617) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.06258938352611139) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7768936170212765) - val-core/sciknoweval/acc/best@16/std:np.float64(0.01940928498500347) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5148829787234042) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03347496380735761) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6902234042553191) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04116517145682157) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9993351063829787) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0025751219057230166) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999361702127659) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0008215631734599073) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9989255319148936) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.003205624725037656) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9995106382978725) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0022285647127782246) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9979148936170213) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.004223069283034246) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9998617021276596) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0012050425965809124) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9958723404255319) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0051839848095949674) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999361702127659) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0008215631734599076) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9933191489361702) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.00514189777423551) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009057087008841336 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009057087008841336 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:320.234375 - response_length/max:646.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:320.234375 - response_length_non_aborted/max:646.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.3125 - prompt_length/max:483.0 - prompt_length/min:267.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013000331819057465 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8450097162276506) - timing_s/agent_loop/generate_sequences/max:np.float64(5.59000326320529) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5512213698166306) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.59000326320529) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:646 - timing_s/gen:11.490527359768748 - timing_s/reward:0.05574657954275608 - timing_s/old_log_prob:2.4932925812900066 - timing_s/adv:0.14599293656647205 - timing_s/update_actor:9.96449064835906 - timing_s/step:24.248870246112347 - timing_s/testing:84.99297607503831 - timing_s/save_checkpoint:6.481567297130823 - timing_s/stop_profile:4.972517490386963e-05 - timing_per_token_ms/update_actor:0.05394492436150122 - timing_per_token_ms/adv:0.0007903643245115315 - timing_per_token_ms/gen:0.1401625684285039 - perf/total_num_tokens:184716 - perf/time_per_step:24.248870246112347 - perf/throughput:952.1886902628703 (TaskRunner pid=2146199) Training Progress: 10%|█ | 10/100 [06:39<1:26:19, 57.55s/it] (TaskRunner pid=2146199) step:11 - global_seqlen/min:22085 - global_seqlen/max:26112 - global_seqlen/minmax_diff:4027 - global_seqlen/balanced_min:24005 - global_seqlen/balanced_max:24008 - global_seqlen/mean:24007.125 - actor/entropy:0.04804360121488571 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6709709763526917 - training/rollout_probs_diff_mean:0.0024593675043433905 - training/rollout_probs_diff_std:0.01397766824811697 - training/rollout_actor_probs_pearson_corr:0.9930342435836792 - rollout_corr/rollout_is_mean:1.00017249584198 - rollout_corr/rollout_is_ratio_fraction_high:0.00015193445142358541 - rollout_corr/rollout_is_ratio_fraction_low:0.0003255738120060414 - rollout_corr/rollout_is_max:12.16672420501709 - rollout_corr/rollout_is_min:2.06115369216775e-09 - rollout_corr/rollout_is_std:0.036076683551073074 - rollout_corr/rollout_is_eff_sample_size:0.9987002836469919 - rollout_corr/rollout_is_seq_mean:1.0001469850540161 - rollout_corr/rollout_is_seq_std:0.003002496436238289 - rollout_corr/rollout_is_seq_max:1.029190182685852 - rollout_corr/rollout_is_seq_min:0.9930292367935181 - rollout_corr/rollout_is_seq_max_deviation:0.02919018268585205 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0474164304323494) - rollout_corr/training_log_ppl:np.float64(0.04559624723515299) - rollout_corr/kl:np.float64(0.0010769233614666973) - rollout_corr/k3_kl:np.float64(0.0011144494094139645) - rollout_corr/rollout_ppl:np.float64(1.046275925822556) - rollout_corr/rollout_log_ppl:np.float64(0.04451932340271014) - rollout_corr/log_ppl_diff:np.float64(0.001076923839718802) - rollout_corr/log_ppl_abs_diff:np.float64(0.001993083431443665) - rollout_corr/log_ppl_diff_max:np.float64(0.07666594535112381) - rollout_corr/log_ppl_diff_min:np.float64(-0.0072558969259262085) - rollout_corr/ppl_ratio:np.float64(1.0010912341531366) - rollout_corr/chi2_token:np.float64(0.0032416859176009893) - rollout_corr/chi2_seq:np.float64(1.461701161460951) - actor/pg_loss:np.float64(0.0001977116335183382) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005052614824307966) - actor/ppo_kl:np.float64(0.00013562464442529176) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25391409173607826) - perf/mfu/actor:np.float64(0.0668949207945355) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(107.56357479095459) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003453795565292239 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003453795565292239 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:359.94140625 - response_length/max:767.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:359.94140625 - response_length_non_aborted/max:767.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.28125 - prompt_length/max:481.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.8119574785232544e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.812249569222331) - timing_s/agent_loop/generate_sequences/max:np.float64(6.398797504603863) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9711650326280505) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.398797504603863) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:767 - timing_s/gen:12.201012717559934 - timing_s/reward:0.054907042533159256 - timing_s/old_log_prob:2.5618306677788496 - timing_s/adv:0.14669655822217464 - timing_s/update_actor:10.214757708832622 - timing_s/step:25.273304034024477 - timing_s/stop_profile:0.00012939982116222382 - timing_per_token_ms/update_actor:0.05318607345128072 - timing_per_token_ms/adv:0.0007638178156598023 - timing_per_token_ms/gen:0.13241101218253767 - perf/total_num_tokens:192057 - perf/time_per_step:25.273304034024477 - perf/throughput:949.900534084508 (TaskRunner pid=2146199) Training Progress: 11%|█ | 11/100 [07:05<1:10:44, 47.69s/it] (TaskRunner pid=2146199) step:12 - global_seqlen/min:20750 - global_seqlen/max:25023 - global_seqlen/minmax_diff:4273 - global_seqlen/balanced_min:23495 - global_seqlen/balanced_max:23502 - global_seqlen/mean:23500.0 - actor/entropy:0.048802800476551056 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9525734186172485 - training/rollout_probs_diff_mean:0.002639800775796175 - training/rollout_probs_diff_std:0.014638073742389679 - training/rollout_actor_probs_pearson_corr:0.9923166036605835 - rollout_corr/rollout_is_mean:1.0000332593917847 - rollout_corr/rollout_is_ratio_fraction_high:4.502070805756375e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0003376553358975798 - rollout_corr/rollout_is_max:5.481647968292236 - rollout_corr/rollout_is_min:7.654040956595054e-09 - rollout_corr/rollout_is_std:0.03588617965579033 - rollout_corr/rollout_is_eff_sample_size:0.9987138383870012 - rollout_corr/rollout_is_seq_mean:1.0000615119934082 - rollout_corr/rollout_is_seq_std:0.0023416769690811634 - rollout_corr/rollout_is_seq_max:1.0136011838912964 - rollout_corr/rollout_is_seq_min:0.9917080998420715 - rollout_corr/rollout_is_seq_max_deviation:0.013601183891296387 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0493773608468473) - rollout_corr/training_log_ppl:np.float64(0.04763505339360563) - rollout_corr/kl:np.float64(0.0011809221397331982) - rollout_corr/k3_kl:np.float64(0.0011246201967551883) - rollout_corr/rollout_ppl:np.float64(1.0481200786307454) - rollout_corr/rollout_log_ppl:np.float64(0.04645413088655914) - rollout_corr/log_ppl_diff:np.float64(0.0011809225034085102) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022081997303757817) - rollout_corr/log_ppl_diff_max:np.float64(0.05857003107666969) - rollout_corr/log_ppl_diff_min:np.float64(-0.006129670888185501) - rollout_corr/ppl_ratio:np.float64(1.0011915797367692) - rollout_corr/chi2_token:np.float64(0.001855870708823204) - rollout_corr/chi2_seq:np.float64(0.5780831542797387) - actor/pg_loss:np.float64(0.00019696413073688745) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009464220574955107) - actor/ppo_kl:np.float64(0.0002554697420098506) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4698943421244621) - perf/mfu/actor:np.float64(0.06694472114904917) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.3212513923645) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005194264464080334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005194264464080334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:347.0625 - response_length/max:708.0 - response_length/min:168.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.0625 - response_length_non_aborted/max:708.0 - response_length_non_aborted/min:168.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:387.3125 - prompt_length/max:482.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.843280375003815e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8421658035367727) - timing_s/agent_loop/generate_sequences/max:np.float64(5.9758670050650835) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6000157960515935) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.9758670050650835) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:685 - timing_s/gen:12.484461918473244 - timing_s/reward:0.05170908384025097 - timing_s/old_log_prob:2.4932120461016893 - timing_s/adv:0.12115662358701229 - timing_s/update_actor:9.90009731054306 - timing_s/step:25.11627039499581 - timing_s/stop_profile:6.029941141605377e-05 - timing_per_token_ms/update_actor:0.0526600920773567 - timing_per_token_ms/adv:0.0006444501254628313 - timing_per_token_ms/gen:0.14051483340619084 - perf/total_num_tokens:188000 - perf/time_per_step:25.11627039499581 - perf/throughput:935.6484713065585 (TaskRunner pid=2146199) Training Progress: 12%|█▏ | 12/100 [07:30<59:53, 40.84s/it] (TaskRunner pid=2146199) step:13 - global_seqlen/min:21964 - global_seqlen/max:24582 - global_seqlen/minmax_diff:2618 - global_seqlen/balanced_min:23388 - global_seqlen/balanced_max:23391 - global_seqlen/mean:23390.125 - actor/entropy:0.03789813816547394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8378890752792358 - training/rollout_probs_diff_mean:0.0023348808754235506 - training/rollout_probs_diff_std:0.014388558454811573 - training/rollout_actor_probs_pearson_corr:0.9909489154815674 - rollout_corr/rollout_is_mean:0.9999727010726929 - rollout_corr/rollout_is_ratio_fraction_high:0.00013636644871439785 - rollout_corr/rollout_is_ratio_fraction_low:0.00042149631190113723 - rollout_corr/rollout_is_max:3.377307176589966 - rollout_corr/rollout_is_min:0.01650751568377018 - rollout_corr/rollout_is_std:0.03544498234987259 - rollout_corr/rollout_is_eff_sample_size:0.9987452297239231 - rollout_corr/rollout_is_seq_mean:1.0000264644622803 - rollout_corr/rollout_is_seq_std:0.0023594500962644815 - rollout_corr/rollout_is_seq_max:1.0164906978607178 - rollout_corr/rollout_is_seq_min:0.9947007894515991 - rollout_corr/rollout_is_seq_max_deviation:0.016490697860717773 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0391791239380836) - rollout_corr/training_log_ppl:np.float64(0.038156213904585456) - rollout_corr/kl:np.float64(0.0009916542165775422) - rollout_corr/k3_kl:np.float64(0.0009449568796142671) - rollout_corr/rollout_ppl:np.float64(1.0381425474770367) - rollout_corr/rollout_log_ppl:np.float64(0.037164559164011735) - rollout_corr/log_ppl_diff:np.float64(0.000991654740573722) - rollout_corr/log_ppl_abs_diff:np.float64(0.002114939181410591) - rollout_corr/log_ppl_diff_max:np.float64(0.01825466752052307) - rollout_corr/log_ppl_diff_min:np.float64(-0.007348746061325073) - rollout_corr/ppl_ratio:np.float64(1.0009963624179363) - rollout_corr/chi2_token:np.float64(0.001698100008070469) - rollout_corr/chi2_seq:np.float64(0.5576191481668502) - actor/pg_loss:np.float64(0.0001590256579220295) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007174677507464366) - actor/ppo_kl:np.float64(0.0001934444717460737) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.36268725991249084) - perf/mfu/actor:np.float64(0.06708696866493306) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.73674011230469) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017045806162059307 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017045806162059307 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:315.09765625 - response_length/max:648.0 - response_length/min:167.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.09765625 - response_length_non_aborted/max:648.0 - response_length_non_aborted/min:167.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:415.84375 - prompt_length/max:493.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.167566895484924e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9277914315462112) - timing_s/agent_loop/generate_sequences/max:np.float64(5.542748164385557) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4976240821924875) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.542748164385557) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:648 - timing_s/gen:11.227821374312043 - timing_s/reward:0.056434473022818565 - timing_s/old_log_prob:2.3685401435941458 - timing_s/adv:0.1418368723243475 - timing_s/update_actor:9.901469407603145 - timing_s/step:23.77626583725214 - timing_s/stop_profile:3.2296404242515564e-05 - timing_per_token_ms/update_actor:0.052914795280076234 - timing_per_token_ms/adv:0.0007579954805946286 - timing_per_token_ms/gen:0.1391907441184162 - perf/total_num_tokens:187121 - perf/time_per_step:23.77626583725214 - perf/throughput:983.7593993987423 (TaskRunner pid=2146199) Training Progress: 13%|█▎ | 13/100 [07:54<51:44, 35.69s/it] (TaskRunner pid=2146199) step:14 - global_seqlen/min:18494 - global_seqlen/max:25835 - global_seqlen/minmax_diff:7341 - global_seqlen/balanced_min:22344 - global_seqlen/balanced_max:22347 - global_seqlen/mean:22346.125 - actor/entropy:0.04147081822156906 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7724411487579346 - training/rollout_probs_diff_mean:0.002374181291088462 - training/rollout_probs_diff_std:0.014511930756270885 - training/rollout_actor_probs_pearson_corr:0.9915551543235779 - rollout_corr/rollout_is_mean:0.9999378323554993 - rollout_corr/rollout_is_ratio_fraction_high:9.91657652775757e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0003594759036786854 - rollout_corr/rollout_is_max:3.036044120788574 - rollout_corr/rollout_is_min:4.5424094423651695e-05 - rollout_corr/rollout_is_std:0.03521046042442322 - rollout_corr/rollout_is_eff_sample_size:0.9987616396172453 - rollout_corr/rollout_is_seq_mean:0.9999905228614807 - rollout_corr/rollout_is_seq_std:0.0020246116910129786 - rollout_corr/rollout_is_seq_max:1.0074162483215332 - rollout_corr/rollout_is_seq_min:0.994396984577179 - rollout_corr/rollout_is_seq_max_deviation:0.007416248321533203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0412489683367312) - rollout_corr/training_log_ppl:np.float64(0.040173851273721084) - rollout_corr/kl:np.float64(0.0007469751301272254) - rollout_corr/k3_kl:np.float64(0.0007798877124827186) - rollout_corr/rollout_ppl:np.float64(1.04046535724774) - rollout_corr/rollout_log_ppl:np.float64(0.03942687580638449) - rollout_corr/log_ppl_diff:np.float64(0.0007469754673365969) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019862048902723473) - rollout_corr/log_ppl_diff_max:np.float64(0.013173799961805344) - rollout_corr/log_ppl_diff_min:np.float64(-0.005907610058784485) - rollout_corr/ppl_ratio:np.float64(1.0007505374960601) - rollout_corr/chi2_token:np.float64(0.001629652688279748) - rollout_corr/chi2_seq:np.float64(0.5884975269436836) - actor/pg_loss:np.float64(0.00022451754193753004) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039736707094562007) - actor/ppo_kl:np.float64(6.446791335434909e-06) - actor/pg_clipfrac_lower:np.float64(7.75049647927517e-06) - actor/grad_norm:np.float64(0.2711631767451763) - perf/mfu/actor:np.float64(0.06485865763409836) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.46635818481445) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003732661483809352 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.003732661483809352 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:315.12890625 - response_length/max:789.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.12890625 - response_length_non_aborted/max:789.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.1875 - prompt_length/max:519.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.81890344619751e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8910789545625448) - timing_s/agent_loop/generate_sequences/max:np.float64(6.065597591921687) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.435446196657722) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.065597591921687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:756 - timing_s/gen:11.893560836091638 - timing_s/reward:0.05877726338803768 - timing_s/old_log_prob:2.365544741973281 - timing_s/adv:0.14879260212183 - timing_s/update_actor:9.8289910312742 - timing_s/step:24.381678000092506 - timing_s/stop_profile:0.00012483634054660797 - timing_per_token_ms/update_actor:0.054981518223373174 - timing_per_token_ms/adv:0.0008323176955838539 - timing_per_token_ms/gen:0.14742926178636764 - perf/total_num_tokens:178769 - perf/time_per_step:24.381678000092506 - perf/throughput:916.5130061973264 (TaskRunner pid=2146199) Training Progress: 14%|█▍ | 14/100 [08:18<46:16, 32.29s/it] (TaskRunner pid=2146199) step:15 - global_seqlen/min:20326 - global_seqlen/max:25244 - global_seqlen/minmax_diff:4918 - global_seqlen/balanced_min:23425 - global_seqlen/balanced_max:23428 - global_seqlen/mean:23426.375 - actor/entropy:0.04354999214410782 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8108813166618347 - training/rollout_probs_diff_mean:0.0025214324705302715 - training/rollout_probs_diff_std:0.015227340161800385 - training/rollout_actor_probs_pearson_corr:0.9907147884368896 - rollout_corr/rollout_is_mean:1.0000585317611694 - rollout_corr/rollout_is_ratio_fraction_high:0.0001255206298083067 - rollout_corr/rollout_is_ratio_fraction_low:0.0003537399461492896 - rollout_corr/rollout_is_max:3.7204794883728027 - rollout_corr/rollout_is_min:0.13603752851486206 - rollout_corr/rollout_is_std:0.03567628934979439 - rollout_corr/rollout_is_eff_sample_size:0.9987288203698291 - rollout_corr/rollout_is_seq_mean:1.0000773668289185 - rollout_corr/rollout_is_seq_std:0.0023061956744641066 - rollout_corr/rollout_is_seq_max:1.0096561908721924 - rollout_corr/rollout_is_seq_min:0.994753360748291 - rollout_corr/rollout_is_seq_max_deviation:0.009656190872192383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0427189744077623) - rollout_corr/training_log_ppl:np.float64(0.04136689609003952) - rollout_corr/kl:np.float64(0.0006346158153789361) - rollout_corr/k3_kl:np.float64(0.0008410603528083982) - rollout_corr/rollout_ppl:np.float64(1.0420630285516381) - rollout_corr/rollout_log_ppl:np.float64(0.040732280065640225) - rollout_corr/log_ppl_diff:np.float64(0.0006346160243992927) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017800193145376397) - rollout_corr/log_ppl_diff_max:np.float64(0.011124536395072937) - rollout_corr/log_ppl_diff_min:np.float64(-0.009195975959300995) - rollout_corr/ppl_ratio:np.float64(1.0006375317461789) - rollout_corr/chi2_token:np.float64(0.0022331061773002148) - rollout_corr/chi2_seq:np.float64(1.1457256809808314) - actor/pg_loss:np.float64(-7.0676906034350395e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004157328976361896) - actor/ppo_kl:np.float64(-8.964717824255786e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2902930714190006) - perf/mfu/actor:np.float64(0.06846971547876807) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.287353515625) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010683516738936305 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010683516738936305 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:342.32421875 - response_length/max:617.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:342.32421875 - response_length_non_aborted/max:617.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:389.75 - prompt_length/max:512.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000151732936501503 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7776077147573233) - timing_s/agent_loop/generate_sequences/max:np.float64(5.827654916793108) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6892597647674847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.827654916793108) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:617 - timing_s/gen:11.441723268479109 - timing_s/reward:0.05430179089307785 - timing_s/old_log_prob:2.391557900235057 - timing_s/adv:0.15398533828556538 - timing_s/update_actor:9.751950528472662 - timing_s/step:23.874049928039312 - timing_s/stop_profile:0.00012215040624141693 - timing_per_token_ms/update_actor:0.05203510214700664 - timing_per_token_ms/adv:0.0008216451450852158 - timing_per_token_ms/gen:0.13056111449168836 - perf/total_num_tokens:187411 - perf/time_per_step:23.874049928039312 - perf/throughput:981.2484714831088 (TaskRunner pid=2146199) Training Progress: 15%|█▌ | 15/100 [08:42<42:09, 29.76s/it] (TaskRunner pid=2146199) step:16 - global_seqlen/min:20847 - global_seqlen/max:26303 - global_seqlen/minmax_diff:5456 - global_seqlen/balanced_min:23500 - global_seqlen/balanced_max:23632 - global_seqlen/mean:23518.5 - actor/entropy:0.043477680534124374 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6190921068191528 - training/rollout_probs_diff_mean:0.0025269410107284784 - training/rollout_probs_diff_std:0.014399349689483643 - training/rollout_actor_probs_pearson_corr:0.9920213222503662 - rollout_corr/rollout_is_mean:0.9998637437820435 - rollout_corr/rollout_is_ratio_fraction_high:0.00010676156671252102 - rollout_corr/rollout_is_ratio_fraction_low:0.0003558718890417367 - rollout_corr/rollout_is_max:3.37223482131958 - rollout_corr/rollout_is_min:0.16767345368862152 - rollout_corr/rollout_is_std:0.0357547253370285 - rollout_corr/rollout_is_eff_sample_size:0.9987229939897616 - rollout_corr/rollout_is_seq_mean:0.9999080896377563 - rollout_corr/rollout_is_seq_std:0.0023896635975688696 - rollout_corr/rollout_is_seq_max:1.0138732194900513 - rollout_corr/rollout_is_seq_min:0.9909143447875977 - rollout_corr/rollout_is_seq_max_deviation:0.01387321949005127 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0447961436584592) - rollout_corr/training_log_ppl:np.float64(0.043338377654436044) - rollout_corr/kl:np.float64(0.0011088096911056056) - rollout_corr/k3_kl:np.float64(0.0009628797115830423) - rollout_corr/rollout_ppl:np.float64(1.043627536855638) - rollout_corr/rollout_log_ppl:np.float64(0.04222956743615214) - rollout_corr/log_ppl_diff:np.float64(0.0011088102182839066) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021916571058682166) - rollout_corr/log_ppl_diff_max:np.float64(0.01215120404958725) - rollout_corr/log_ppl_diff_min:np.float64(-0.007823154330253601) - rollout_corr/ppl_ratio:np.float64(1.0011134468950331) - rollout_corr/chi2_token:np.float64(0.0018072763923555613) - rollout_corr/chi2_seq:np.float64(0.7385764953214675) - actor/pg_loss:np.float64(-9.064516052603722e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027790915828518337) - actor/ppo_kl:np.float64(0.00025082025372924477) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2098446674644947) - perf/mfu/actor:np.float64(0.06811855123371083) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.33757066726685) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:329.296875 - response_length/max:986.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:329.296875 - response_length_non_aborted/max:986.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:405.65625 - prompt_length/max:495.0 - prompt_length/min:268.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011428818106651306 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7657798640429974) - timing_s/agent_loop/generate_sequences/max:np.float64(7.073206698521972) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.589238177832158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.073206698521972) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:424 - timing_s/agent_loop/slowest/response_length:986 - timing_s/gen:12.93631342612207 - timing_s/reward:0.05608239956200123 - timing_s/old_log_prob:2.3548991084098816 - timing_s/adv:0.15225598774850368 - timing_s/update_actor:9.83015213534236 - timing_s/step:25.420028114691377 - timing_s/stop_profile:0.00011796876788139343 - timing_per_token_ms/update_actor:0.0522469127247824 - timing_per_token_ms/adv:0.0008092352177461556 - timing_per_token_ms/gen:0.15345567528021434 - perf/total_num_tokens:188148 - perf/time_per_step:25.420028114691377 - perf/throughput:925.1956722426912 (TaskRunner pid=2146199) Training Progress: 16%|█▌ | 16/100 [09:08<39:51, 28.47s/it] (TaskRunner pid=2146199) step:17 - global_seqlen/min:19814 - global_seqlen/max:24781 - global_seqlen/minmax_diff:4967 - global_seqlen/balanced_min:22399 - global_seqlen/balanced_max:22405 - global_seqlen/mean:22402.75 - actor/entropy:0.0463063009083271 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37268006801605225 - training/rollout_probs_diff_mean:0.00243695848621428 - training/rollout_probs_diff_std:0.01342450175434351 - training/rollout_actor_probs_pearson_corr:0.9932261109352112 - rollout_corr/rollout_is_mean:1.0002028942108154 - rollout_corr/rollout_is_ratio_fraction_high:0.00011625804472714663 - rollout_corr/rollout_is_ratio_fraction_low:0.0002712687710300088 - rollout_corr/rollout_is_max:4.591778755187988 - rollout_corr/rollout_is_min:0.15520279109477997 - rollout_corr/rollout_is_std:0.03565957769751549 - rollout_corr/rollout_is_eff_sample_size:0.9987304850623345 - rollout_corr/rollout_is_seq_mean:1.0001238584518433 - rollout_corr/rollout_is_seq_std:0.0025866213254630566 - rollout_corr/rollout_is_seq_max:1.0107158422470093 - rollout_corr/rollout_is_seq_min:0.9888857007026672 - rollout_corr/rollout_is_seq_max_deviation:0.011114299297332764 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0448525743559003) - rollout_corr/training_log_ppl:np.float64(0.04305281082724832) - rollout_corr/kl:np.float64(0.0008356443202188757) - rollout_corr/k3_kl:np.float64(0.0008811692904391322) - rollout_corr/rollout_ppl:np.float64(1.0439724740572274) - rollout_corr/rollout_log_ppl:np.float64(0.042217166343561985) - rollout_corr/log_ppl_diff:np.float64(0.0008356444836863375) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021454203738358046) - rollout_corr/log_ppl_diff_max:np.float64(0.02130807936191559) - rollout_corr/log_ppl_diff_min:np.float64(-0.008684247732162476) - rollout_corr/ppl_ratio:np.float64(1.0008406278211623) - rollout_corr/chi2_token:np.float64(0.001763105159625411) - rollout_corr/chi2_seq:np.float64(1.3347710797097534) - actor/pg_loss:np.float64(-0.0001132437027990818) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003963027152167342) - actor/ppo_kl:np.float64(0.00024466345477236473) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.229076923802495) - perf/mfu/actor:np.float64(0.06566308456223045) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.31360244750977) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013062882935628295 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013062882935628295 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:302.3984375 - response_length/max:683.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:302.3984375 - response_length_non_aborted/max:683.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.6875 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015123561024665833 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1555626168847084) - timing_s/agent_loop/generate_sequences/max:np.float64(5.389894874766469) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.027427659078967) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.389894874766469) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:448 - timing_s/agent_loop/slowest/response_length:683 - timing_s/gen:11.307871432974935 - timing_s/reward:0.055363453924655914 - timing_s/old_log_prob:2.488108344376087 - timing_s/adv:0.14510678686201572 - timing_s/update_actor:9.701658526435494 - timing_s/step:23.792897498235106 - timing_s/stop_profile:0.00013346411287784576 - timing_per_token_ms/update_actor:0.05413207377685493 - timing_per_token_ms/adv:0.0008096482957561891 - timing_per_token_ms/gen:0.14607010919181201 - perf/total_num_tokens:179222 - perf/time_per_step:23.792897498235106 - perf/throughput:941.5730052071958 (TaskRunner pid=2146199) Training Progress: 17%|█▋ | 17/100 [09:31<37:27, 27.08s/it] (TaskRunner pid=2146199) step:18 - global_seqlen/min:20000 - global_seqlen/max:23598 - global_seqlen/minmax_diff:3598 - global_seqlen/balanced_min:21837 - global_seqlen/balanced_max:21838 - global_seqlen/mean:21837.75 - actor/entropy:0.03896229714155197 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7615939378738403 - training/rollout_probs_diff_mean:0.0023382746148854494 - training/rollout_probs_diff_std:0.014567550271749496 - training/rollout_actor_probs_pearson_corr:0.9904218316078186 - rollout_corr/rollout_is_mean:1.0000776052474976 - rollout_corr/rollout_is_ratio_fraction_high:0.00010664675937732682 - rollout_corr/rollout_is_ratio_fraction_low:0.00042658703750930727 - rollout_corr/rollout_is_max:4.670137405395508 - rollout_corr/rollout_is_min:0.1353355050086975 - rollout_corr/rollout_is_std:0.034609463065862656 - rollout_corr/rollout_is_eff_sample_size:0.9988037370269406 - rollout_corr/rollout_is_seq_mean:1.0001204013824463 - rollout_corr/rollout_is_seq_std:0.0025972514413297176 - rollout_corr/rollout_is_seq_max:1.0225214958190918 - rollout_corr/rollout_is_seq_min:0.9927294254302979 - rollout_corr/rollout_is_seq_max_deviation:0.022521495819091797 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.041425188537687) - rollout_corr/training_log_ppl:np.float64(0.03946447566659117) - rollout_corr/kl:np.float64(0.0011154848630654524) - rollout_corr/k3_kl:np.float64(0.0011972512637412613) - rollout_corr/rollout_ppl:np.float64(1.0402454063296318) - rollout_corr/rollout_log_ppl:np.float64(0.03834899080084142) - rollout_corr/log_ppl_diff:np.float64(0.0011154848657497496) - rollout_corr/log_ppl_abs_diff:np.float64(0.002522762328680983) - rollout_corr/log_ppl_diff_max:np.float64(0.016387417912483215) - rollout_corr/log_ppl_diff_min:np.float64(-0.008698612451553345) - rollout_corr/ppl_ratio:np.float64(1.001121771056205) - rollout_corr/chi2_token:np.float64(0.0035759725142270327) - rollout_corr/chi2_seq:np.float64(1.0916502529289573) - actor/pg_loss:np.float64(0.0002637462457641959) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006489941197287408) - actor/ppo_kl:np.float64(0.0005292700741037493) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4650009907782078) - perf/mfu/actor:np.float64(0.06327140411758841) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.34241104125977) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009464899776503444 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009464899776503444 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.0234375 - response_length/max:647.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.0234375 - response_length_non_aborted/max:647.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:389.40625 - prompt_length/max:492.0 - prompt_length/min:206.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00020122528076171875 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8001448679715395) - timing_s/agent_loop/generate_sequences/max:np.float64(5.56584762968123) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.281876176064543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.56584762968123) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:647 - timing_s/gen:11.237427227199078 - timing_s/reward:0.056326283141970634 - timing_s/old_log_prob:2.436118945479393 - timing_s/adv:0.1545155793428421 - timing_s/update_actor:9.799792379140854 - timing_s/step:23.77557411044836 - timing_s/stop_profile:0.00011693127453327179 - timing_per_token_ms/update_actor:0.05609433423281276 - timing_per_token_ms/adv:0.000884452263527848 - timing_per_token_ms/gen:0.14980439954140665 - perf/total_num_tokens:174702 - perf/time_per_step:23.77557411044836 - perf/throughput:918.4951706551318 (TaskRunner pid=2146199) Training Progress: 18%|█▊ | 18/100 [09:55<35:41, 26.11s/it] (TaskRunner pid=2146199) step:19 - global_seqlen/min:19905 - global_seqlen/max:26460 - global_seqlen/minmax_diff:6555 - global_seqlen/balanced_min:22852 - global_seqlen/balanced_max:22854 - global_seqlen/mean:22853.0 - actor/entropy:0.03838445246219635 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41412943601608276 - training/rollout_probs_diff_mean:0.0021190159022808075 - training/rollout_probs_diff_std:0.012656391598284245 - training/rollout_actor_probs_pearson_corr:0.9927424788475037 - rollout_corr/rollout_is_mean:1.0001617670059204 - rollout_corr/rollout_is_ratio_fraction_high:0.00010653409117367119 - rollout_corr/rollout_is_ratio_fraction_low:0.00010653409117367119 - rollout_corr/rollout_is_max:5.301244258880615 - rollout_corr/rollout_is_min:0.061663877218961716 - rollout_corr/rollout_is_std:0.03153102472424507 - rollout_corr/rollout_is_eff_sample_size:0.9990069009605514 - rollout_corr/rollout_is_seq_mean:1.0001816749572754 - rollout_corr/rollout_is_seq_std:0.002223767340183258 - rollout_corr/rollout_is_seq_max:1.0155545473098755 - rollout_corr/rollout_is_seq_min:0.993762731552124 - rollout_corr/rollout_is_seq_max_deviation:0.015554547309875488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.035927452147007) - rollout_corr/training_log_ppl:np.float64(0.0349866963479144) - rollout_corr/kl:np.float64(0.000972015449452357) - rollout_corr/k3_kl:np.float64(0.0007442191562265776) - rollout_corr/rollout_ppl:np.float64(1.0349205769598484) - rollout_corr/rollout_log_ppl:np.float64(0.03401468031370314) - rollout_corr/log_ppl_diff:np.float64(0.0009720160342112649) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018738987855613232) - rollout_corr/log_ppl_diff_max:np.float64(0.02324722521007061) - rollout_corr/log_ppl_diff_min:np.float64(-0.006566643714904785) - rollout_corr/ppl_ratio:np.float64(1.0009760349057615) - rollout_corr/chi2_token:np.float64(0.0009242140222340822) - rollout_corr/chi2_seq:np.float64(0.34708500327542424) - actor/pg_loss:np.float64(0.00011837633792310953) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003336126005706319) - actor/ppo_kl:np.float64(0.000562068689792472) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22513309493660927) - perf/mfu/actor:np.float64(0.06660225748014949) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.41831588745117) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002775804838165641 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002775804838165641 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:330.0 - response_length/max:737.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:330.0 - response_length_non_aborted/max:737.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.15625 - prompt_length/max:494.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.08550214767456e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6514490824192762) - timing_s/agent_loop/generate_sequences/max:np.float64(5.689075980335474) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.537914513493888) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.689075980335474) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:737 - timing_s/gen:11.444722078740597 - timing_s/reward:0.050500670447945595 - timing_s/old_log_prob:2.3623054400086403 - timing_s/adv:0.15039121732115746 - timing_s/update_actor:9.73127668723464 - timing_s/step:23.84060306660831 - timing_s/stop_profile:0.00013444572687149048 - timing_per_token_ms/update_actor:0.0532275668798114 - timing_per_token_ms/adv:0.0008226010661683228 - timing_per_token_ms/gen:0.1354725624850923 - perf/total_num_tokens:182824 - perf/time_per_step:23.84060306660831 - perf/throughput:958.5747447810342 (TaskRunner pid=2146199) Training Progress: 19%|█▉ | 19/100 [10:19<34:20, 25.44s/it] (TaskRunner pid=2146199) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2146199) validation generation end (TaskRunner pid=2146199) [prompt] system (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) The unit cell volume for a crystal with orthogonal lattice parameters (α = β = γ = 90.00º) is calculated using the formula: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = a \times b \times c $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given: (TaskRunner pid=2146199) - $ a = 7.82 $ Å (TaskRunner pid=2146199) - $ b = 7.82 $ Å (TaskRunner pid=2146199) - $ c = 17.88 $ Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Substituting the values: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Then multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 61.1524 \times 17.88 \approx 1094.31 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Thus, the unit cell volume is approximately 1094.31 ų. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2146577) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2146199) step:20 - global_seqlen/min:20761 - global_seqlen/max:26293 - global_seqlen/minmax_diff:5532 - global_seqlen/balanced_min:22881 - global_seqlen/balanced_max:22886 - global_seqlen/mean:22883.875 - actor/entropy:0.040829796344041824 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6767197847366333 - training/rollout_probs_diff_mean:0.0022959536872804165 - training/rollout_probs_diff_std:0.015229374170303345 - training/rollout_actor_probs_pearson_corr:0.990498960018158 - rollout_corr/rollout_is_mean:1.0000412464141846 - rollout_corr/rollout_is_ratio_fraction_high:0.00012437965779099613 - rollout_corr/rollout_is_ratio_fraction_low:0.00041045286343432963 - rollout_corr/rollout_is_max:3.4335153102874756 - rollout_corr/rollout_is_min:0.019543590024113655 - rollout_corr/rollout_is_std:0.03540291637182236 - rollout_corr/rollout_is_eff_sample_size:0.9987482024906889 - rollout_corr/rollout_is_seq_mean:1.0000364780426025 - rollout_corr/rollout_is_seq_std:0.0022689122706651688 - rollout_corr/rollout_is_seq_max:1.0094542503356934 - rollout_corr/rollout_is_seq_min:0.9915512204170227 - rollout_corr/rollout_is_seq_max_deviation:0.00945425033569336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0418692524544895) - rollout_corr/training_log_ppl:np.float64(0.04061956122768606) - rollout_corr/kl:np.float64(0.0018356995592583303) - rollout_corr/k3_kl:np.float64(0.0017537820963298145) - rollout_corr/rollout_ppl:np.float64(1.0399280181154609) - rollout_corr/rollout_log_ppl:np.float64(0.0387838612887208) - rollout_corr/log_ppl_diff:np.float64(0.0018356999462412205) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029169848930905573) - rollout_corr/log_ppl_diff_max:np.float64(0.0967784896492958) - rollout_corr/log_ppl_diff_min:np.float64(-0.007019244134426117) - rollout_corr/ppl_ratio:np.float64(1.0018716061022133) - rollout_corr/chi2_token:np.float64(0.0020720630418509245) - rollout_corr/chi2_seq:np.float64(0.8615165909286588) - actor/pg_loss:np.float64(0.00026437954511493444) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000842500560793269) - actor/ppo_kl:np.float64(0.0009866946499386131) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3665076829493046) - perf/mfu/actor:np.float64(0.06644497641178498) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.37300872802734) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6954787234042553) - val-aux/sciknoweval/reward/std@16:np.float64(0.05673634872061028) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7178085106382978) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.03705586901793213) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.675) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.05720844428095515) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6972234042553191) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.05582239840843878) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7309255319148935) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.022313915758041675) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6530638297872341) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.054445750732956544) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7015319148936169) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.04185952319961764) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7392872340425531) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.012458186908246488) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6295851063829787) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.04789796807680137) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7015319148936171) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.026179899924259224) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7431063829787234) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.00565033207953325) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6077021276595744) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03702821049345705) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7005531914893618) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.016586801963155637) - val-aux/sciknoweval/score/mean@16:np.float64(0.6954787234042553) - val-aux/sciknoweval/score/std@16:np.float64(0.05673634872061028) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7178085106382978) - val-aux/sciknoweval/score/best@2/std:np.float64(0.03705586901793213) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.675) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.05720844428095515) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6972234042553191) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.05582239840843878) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7309255319148935) - val-aux/sciknoweval/score/best@4/std:np.float64(0.022313915758041675) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6530638297872341) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.054445750732956544) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7015319148936169) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.04185952319961764) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7392872340425531) - val-aux/sciknoweval/score/best@8/std:np.float64(0.012458186908246488) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6295851063829787) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.04789796807680137) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7015319148936171) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.026179899924259224) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7431063829787234) - val-aux/sciknoweval/score/best@16/std:np.float64(0.00565033207953325) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6077021276595744) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03702821049345705) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7005531914893618) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.016586801963155637) - val-core/sciknoweval/acc/mean@16:np.float64(0.6954787234042553) - val-aux/sciknoweval/acc/std@16:np.float64(0.05673634872061028) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7178085106382978) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.03705586901793213) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.675) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.05720844428095515) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6972234042553191) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.05582239840843878) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7309255319148935) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.022313915758041675) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6530638297872341) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.054445750732956544) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7015319148936169) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.04185952319961764) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7392872340425531) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.012458186908246488) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6295851063829787) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.04789796807680137) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7015319148936171) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.026179899924259224) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7431063829787234) - val-core/sciknoweval/acc/best@16/std:np.float64(0.00565033207953325) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6077021276595744) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03702821049345705) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7005531914893618) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.016586801963155637) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001842373632825911 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001842373632825911 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:314.05859375 - response_length/max:722.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.05859375 - response_length_non_aborted/max:722.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.0625 - prompt_length/max:492.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010284781455993652 - timing_s/agent_loop/generate_sequences/min:np.float64(1.192504346370697) - timing_s/agent_loop/generate_sequences/max:np.float64(5.488217432051897) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1740152926868177) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.488217432051897) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:287 - timing_s/agent_loop/slowest/response_length:669 - timing_s/gen:11.343108167871833 - timing_s/reward:0.05593980848789215 - timing_s/old_log_prob:2.4695548079907894 - timing_s/adv:0.14687232300639153 - timing_s/update_actor:9.7808505166322 - timing_s/step:23.89146515727043 - timing_s/testing:37.24984707869589 - timing_s/save_checkpoint:6.69863430224359 - timing_s/stop_profile:0.00014093518257141113 - timing_per_token_ms/update_actor:0.05342654225208908 - timing_per_token_ms/adv:0.0008022697369129547 - timing_per_token_ms/gen:0.14108518971469586 - perf/total_num_tokens:183071 - perf/time_per_step:23.89146515727043 - perf/throughput:957.8263555358466 (TaskRunner pid=2146199) Training Progress: 20%|██ | 20/100 [11:27<50:56, 38.20s/it] (TaskRunner pid=2146199) step:21 - global_seqlen/min:20225 - global_seqlen/max:25681 - global_seqlen/minmax_diff:5456 - global_seqlen/balanced_min:23009 - global_seqlen/balanced_max:23017 - global_seqlen/mean:23014.375 - actor/entropy:0.04493596777319908 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9706429243087769 - training/rollout_probs_diff_mean:0.0024657186586409807 - training/rollout_probs_diff_std:0.01483361516147852 - training/rollout_actor_probs_pearson_corr:0.9916916489601135 - rollout_corr/rollout_is_mean:0.9998268485069275 - rollout_corr/rollout_is_ratio_fraction_high:6.008820855640806e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00037254689959809184 - rollout_corr/rollout_is_max:2.550130605697632 - rollout_corr/rollout_is_min:2.836771818692796e-05 - rollout_corr/rollout_is_std:0.03629247099161148 - rollout_corr/rollout_is_eff_sample_size:0.9986843513528347 - rollout_corr/rollout_is_seq_mean:0.9997856616973877 - rollout_corr/rollout_is_seq_std:0.0021155045833438635 - rollout_corr/rollout_is_seq_max:1.0108908414840698 - rollout_corr/rollout_is_seq_min:0.9926724433898926 - rollout_corr/rollout_is_seq_max_deviation:0.010890841484069824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0451039499603212) - rollout_corr/training_log_ppl:np.float64(0.04355830265376426) - rollout_corr/kl:np.float64(0.0011976572949095043) - rollout_corr/k3_kl:np.float64(0.0010930493323459967) - rollout_corr/rollout_ppl:np.float64(1.0438524163328111) - rollout_corr/rollout_log_ppl:np.float64(0.0423606450913212) - rollout_corr/log_ppl_diff:np.float64(0.0011976575697190128) - rollout_corr/log_ppl_abs_diff:np.float64(0.00206501908314749) - rollout_corr/log_ppl_diff_max:np.float64(0.03474971652030945) - rollout_corr/log_ppl_diff_min:np.float64(-0.006067708134651184) - rollout_corr/ppl_ratio:np.float64(1.0012036834377795) - rollout_corr/chi2_token:np.float64(0.0016887260135263205) - rollout_corr/chi2_seq:np.float64(0.3236536830663681) - actor/pg_loss:np.float64(2.0634150132536888e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009130278394877678) - actor/ppo_kl:np.float64(0.00012112113509132882) - actor/pg_clipfrac_lower:np.float64(1.4796401956118643e-05) - actor/grad_norm:np.float64(0.42226750403642654) - perf/mfu/actor:np.float64(0.06720410238218269) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.24273300170898) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0004506615805439651 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0004506615805439651 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.04296875 - response_length/max:793.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:325.04296875 - response_length_non_aborted/max:793.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.15625 - prompt_length/max:524.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.0755759477615356e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4703508745878935) - timing_s/agent_loop/generate_sequences/max:np.float64(6.180332293733954) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4129051694835653) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.180332293733954) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:446 - timing_s/agent_loop/slowest/response_length:780 - timing_s/gen:11.762525411322713 - timing_s/reward:0.05409410037100315 - timing_s/old_log_prob:2.5207739155739546 - timing_s/adv:0.11892217770218849 - timing_s/update_actor:9.887562368065119 - timing_s/step:24.437047641724348 - timing_s/stop_profile:0.00015973858535289764 - timing_per_token_ms/update_actor:0.053703187508161304 - timing_per_token_ms/adv:0.000645912487859156 - timing_per_token_ms/gen:0.14135781821300925 - perf/total_num_tokens:184115 - perf/time_per_step:24.437047641724348 - perf/throughput:941.7821390463206 (TaskRunner pid=2146199) Training Progress: 21%|██ | 21/100 [11:52<44:52, 34.09s/it] (TaskRunner pid=2146199) step:22 - global_seqlen/min:18410 - global_seqlen/max:26029 - global_seqlen/minmax_diff:7619 - global_seqlen/balanced_min:22729 - global_seqlen/balanced_max:22733 - global_seqlen/mean:22730.625 - actor/entropy:0.04354904219508171 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5994649529457092 - training/rollout_probs_diff_mean:0.002329489216208458 - training/rollout_probs_diff_std:0.013881352730095387 - training/rollout_actor_probs_pearson_corr:0.9925063848495483 - rollout_corr/rollout_is_mean:1.0000649690628052 - rollout_corr/rollout_is_ratio_fraction_high:0.0001294406974921003 - rollout_corr/rollout_is_ratio_fraction_low:0.00024711404694244266 - rollout_corr/rollout_is_max:3.9057557582855225 - rollout_corr/rollout_is_min:0.03691454231739044 - rollout_corr/rollout_is_std:0.03593597561120987 - rollout_corr/rollout_is_eff_sample_size:0.9987105091185257 - rollout_corr/rollout_is_seq_mean:1.0000625848770142 - rollout_corr/rollout_is_seq_std:0.0021109809167683125 - rollout_corr/rollout_is_seq_max:1.0080187320709229 - rollout_corr/rollout_is_seq_min:0.9923405647277832 - rollout_corr/rollout_is_seq_max_deviation:0.008018732070922852 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0428022961132228) - rollout_corr/training_log_ppl:np.float64(0.041575555958843324) - rollout_corr/kl:np.float64(0.0007461832131205881) - rollout_corr/k3_kl:np.float64(0.0008189501936222143) - rollout_corr/rollout_ppl:np.float64(1.0420160847716033) - rollout_corr/rollout_log_ppl:np.float64(0.04082937218663574) - rollout_corr/log_ppl_diff:np.float64(0.000746183772207587) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018291979340574471) - rollout_corr/log_ppl_diff_max:np.float64(0.01106233149766922) - rollout_corr/log_ppl_diff_min:np.float64(-0.005633614957332611) - rollout_corr/ppl_ratio:np.float64(1.0007491316646338) - rollout_corr/chi2_token:np.float64(0.0018020751886069775) - rollout_corr/chi2_seq:np.float64(0.6032293292228132) - actor/pg_loss:np.float64(0.00015866453759372234) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025709564761200454) - actor/ppo_kl:np.float64(6.950214837875279e-05) - actor/pg_clipfrac_lower:np.float64(1.076101943908725e-05) - actor/grad_norm:np.float64(0.2663578763604164) - perf/mfu/actor:np.float64(0.06676206427732029) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.41703414916992) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001550346496514976 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001550346496514976 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:331.95703125 - response_length/max:720.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.95703125 - response_length_non_aborted/max:720.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:378.375 - prompt_length/max:509.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.372163236141205e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.124783132225275) - timing_s/agent_loop/generate_sequences/max:np.float64(5.772342914715409) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.204808233123913) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.772342914715409) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:684 - timing_s/gen:12.005846351385117 - timing_s/reward:0.09300452470779419 - timing_s/old_log_prob:2.3728371523320675 - timing_s/adv:0.11896676570177078 - timing_s/update_actor:9.822040136903524 - timing_s/step:24.501356223598123 - timing_s/stop_profile:0.00010884180665016174 - timing_per_token_ms/update_actor:0.05401325379803418 - timing_per_token_ms/adv:0.0006542207138044531 - timing_per_token_ms/gen:0.14127683071963282 - perf/total_num_tokens:181845 - perf/time_per_step:24.501356223598123 - perf/throughput:927.729256803643 (TaskRunner pid=2146199) Training Progress: 22%|██▏ | 22/100 [12:16<40:35, 31.22s/it] (TaskRunner pid=2146199) step:23 - global_seqlen/min:16183 - global_seqlen/max:25704 - global_seqlen/minmax_diff:9521 - global_seqlen/balanced_min:22001 - global_seqlen/balanced_max:22003 - global_seqlen/mean:22002.25 - actor/entropy:0.045259740203619 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8767268061637878 - training/rollout_probs_diff_mean:0.002389344619587064 - training/rollout_probs_diff_std:0.013534430414438248 - training/rollout_actor_probs_pearson_corr:0.9929080009460449 - rollout_corr/rollout_is_mean:0.9998787045478821 - rollout_corr/rollout_is_ratio_fraction_high:8.998122939374298e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0003599249175749719 - rollout_corr/rollout_is_max:3.176034688949585 - rollout_corr/rollout_is_min:0.004620974883437157 - rollout_corr/rollout_is_std:0.03346775472164154 - rollout_corr/rollout_is_eff_sample_size:0.9988809248290607 - rollout_corr/rollout_is_seq_mean:0.9999144077301025 - rollout_corr/rollout_is_seq_std:0.001997526967898011 - rollout_corr/rollout_is_seq_max:1.0090209245681763 - rollout_corr/rollout_is_seq_min:0.9943142533302307 - rollout_corr/rollout_is_seq_max_deviation:0.00902092456817627 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.043508417904377) - rollout_corr/training_log_ppl:np.float64(0.042274245090993645) - rollout_corr/kl:np.float64(0.0009261817039174502) - rollout_corr/k3_kl:np.float64(0.00077575315688172) - rollout_corr/rollout_ppl:np.float64(1.0425332817249) - rollout_corr/rollout_log_ppl:np.float64(0.04134806297315663) - rollout_corr/log_ppl_diff:np.float64(0.0009261821178370155) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019429696185397916) - rollout_corr/log_ppl_diff_max:np.float64(0.010432511568069458) - rollout_corr/log_ppl_diff_min:np.float64(-0.006018925458192825) - rollout_corr/ppl_ratio:np.float64(1.0009296950884163) - rollout_corr/chi2_token:np.float64(0.0012991195544600487) - rollout_corr/chi2_seq:np.float64(0.5500934515148401) - actor/pg_loss:np.float64(-2.4922890588641167e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.291479828883894e-05) - actor/ppo_kl:np.float64(0.00016697796750975158) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.05438255472108722) - perf/mfu/actor:np.float64(0.06368270389166085) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.47252655029297) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011488675372675061 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011488675372675061 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:303.8828125 - response_length/max:650.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:303.8828125 - response_length_non_aborted/max:650.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.6875 - prompt_length/max:493.0 - prompt_length/min:257.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.594244718551636e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8659123498946428) - timing_s/agent_loop/generate_sequences/max:np.float64(5.346146440133452) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.33719733444741) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.346146440133452) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:617 - timing_s/gen:11.069917641580105 - timing_s/reward:0.059640225023031235 - timing_s/old_log_prob:2.4271519351750612 - timing_s/adv:0.11804725229740143 - timing_s/update_actor:9.784748075529933 - timing_s/step:23.545776020735502 - timing_s/stop_profile:0.00011649914085865021 - timing_per_token_ms/update_actor:0.05558947423291898 - timing_per_token_ms/adv:0.0006706544347589533 - timing_per_token_ms/gen:0.14229783327223314 - perf/total_num_tokens:176018 - perf/time_per_step:23.545776020735502 - perf/throughput:934.4457358561382 (TaskRunner pid=2146199) Training Progress: 23%|██▎ | 23/100 [12:40<37:07, 28.93s/it] (TaskRunner pid=2146199) step:24 - global_seqlen/min:20314 - global_seqlen/max:26171 - global_seqlen/minmax_diff:5857 - global_seqlen/balanced_min:23498 - global_seqlen/balanced_max:23504 - global_seqlen/mean:23502.25 - actor/entropy:0.041591692715883255 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9998687505722046 - training/rollout_probs_diff_mean:0.002245469717308879 - training/rollout_probs_diff_std:0.01447212416678667 - training/rollout_actor_probs_pearson_corr:0.9912927746772766 - rollout_corr/rollout_is_mean:0.9998348355293274 - rollout_corr/rollout_is_ratio_fraction_high:7.806053326930851e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00031224213307723403 - rollout_corr/rollout_is_max:2.817819356918335 - rollout_corr/rollout_is_min:9.455720828555059e-08 - rollout_corr/rollout_is_std:0.03345706686377525 - rollout_corr/rollout_is_eff_sample_size:0.9988817574281683 - rollout_corr/rollout_is_seq_mean:0.9998316764831543 - rollout_corr/rollout_is_seq_std:0.001975232968106866 - rollout_corr/rollout_is_seq_max:1.0053426027297974 - rollout_corr/rollout_is_seq_min:0.9929408431053162 - rollout_corr/rollout_is_seq_max_deviation:0.007059156894683838 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.041077284142375) - rollout_corr/training_log_ppl:np.float64(0.03989186929902644) - rollout_corr/kl:np.float64(0.001015942492461619) - rollout_corr/k3_kl:np.float64(0.0010839477918005436) - rollout_corr/rollout_ppl:np.float64(1.0400033150799572) - rollout_corr/rollout_log_ppl:np.float64(0.03887592650971783) - rollout_corr/log_ppl_diff:np.float64(0.0010159427893086104) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020764827550010523) - rollout_corr/log_ppl_diff_max:np.float64(0.05382328853011131) - rollout_corr/log_ppl_diff_min:np.float64(-0.005679916590452194) - rollout_corr/ppl_ratio:np.float64(1.0010282266885042) - rollout_corr/chi2_token:np.float64(0.0016270780470222235) - rollout_corr/chi2_seq:np.float64(0.6201781923882663) - actor/pg_loss:np.float64(0.00023143517319113016) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023588482963532442) - actor/ppo_kl:np.float64(-0.000204702424177583) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20537637174129486) - perf/mfu/actor:np.float64(0.06762185663414236) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.49060821533203) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.89453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.89453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008238173904828727 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008238173904828727 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:350.2890625 - response_length/max:738.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:350.2890625 - response_length_non_aborted/max:738.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.15625 - prompt_length/max:477.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014334917068481445 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0240103006362915) - timing_s/agent_loop/generate_sequences/max:np.float64(6.07601099088788) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.818593486685131) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.07601099088788) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:720 - timing_s/gen:11.714534655213356 - timing_s/reward:0.05965903215110302 - timing_s/old_log_prob:2.3026589397341013 - timing_s/adv:0.11843935586512089 - timing_s/update_actor:9.813919864594936 - timing_s/step:24.098712066188455 - timing_s/stop_profile:0.00011205300688743591 - timing_per_token_ms/update_actor:0.052196703850668213 - timing_per_token_ms/adv:0.0006299362607044053 - timing_per_token_ms/gen:0.13063468402450382 - perf/total_num_tokens:188018 - perf/time_per_step:24.098712066188455 - perf/throughput:975.2492139600557 (TaskRunner pid=2146199) Training Progress: 24%|██▍ | 24/100 [13:04<34:49, 27.50s/it] (TaskRunner pid=2146199) step:25 - global_seqlen/min:17633 - global_seqlen/max:25737 - global_seqlen/minmax_diff:8104 - global_seqlen/balanced_min:22612 - global_seqlen/balanced_max:22618 - global_seqlen/mean:22615.75 - actor/entropy:0.04362443462014198 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5922955274581909 - training/rollout_probs_diff_mean:0.0024653850123286247 - training/rollout_probs_diff_std:0.014283888041973114 - training/rollout_actor_probs_pearson_corr:0.9920059442520142 - rollout_corr/rollout_is_mean:0.9999774098396301 - rollout_corr/rollout_is_ratio_fraction_high:0.0001481957151554525 - rollout_corr/rollout_is_ratio_fraction_low:0.00028404180193319917 - rollout_corr/rollout_is_max:3.445549726486206 - rollout_corr/rollout_is_min:0.03414463624358177 - rollout_corr/rollout_is_std:0.035899464040994644 - rollout_corr/rollout_is_eff_sample_size:0.9987125304574529 - rollout_corr/rollout_is_seq_mean:0.9998875856399536 - rollout_corr/rollout_is_seq_std:0.0023354438599199057 - rollout_corr/rollout_is_seq_max:1.008617639541626 - rollout_corr/rollout_is_seq_min:0.9919745326042175 - rollout_corr/rollout_is_seq_max_deviation:0.008617639541625977 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0493602002970874) - rollout_corr/training_log_ppl:np.float64(0.047062202538654674) - rollout_corr/kl:np.float64(0.0011820345746613725) - rollout_corr/k3_kl:np.float64(0.0010784849316571865) - rollout_corr/rollout_ppl:np.float64(1.0481097651645541) - rollout_corr/rollout_log_ppl:np.float64(0.045880167574068764) - rollout_corr/log_ppl_diff:np.float64(0.0011820349645859096) - rollout_corr/log_ppl_abs_diff:np.float64(0.00221388423233293) - rollout_corr/log_ppl_diff_max:np.float64(0.014117575250566006) - rollout_corr/log_ppl_diff_min:np.float64(-0.009414911270141602) - rollout_corr/ppl_ratio:np.float64(1.0011872979812324) - rollout_corr/chi2_token:np.float64(0.002581575885415077) - rollout_corr/chi2_seq:np.float64(0.5620029144920409) - actor/pg_loss:np.float64(7.945706602185965e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002110175996676844) - actor/ppo_kl:np.float64(0.00026081126885202366) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21556227281689644) - perf/mfu/actor:np.float64(0.06516885409122161) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.4535026550293) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0030426434241235256 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0030426434241235256 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:316.3046875 - response_length/max:696.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:316.3046875 - response_length_non_aborted/max:696.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.4375 - prompt_length/max:481.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.837187826633453e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5630173198878765) - timing_s/agent_loop/generate_sequences/max:np.float64(5.738300919532776) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.379125988969463) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.738300919532776) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:696 - timing_s/gen:11.339737389236689 - timing_s/reward:0.059109415858983994 - timing_s/old_log_prob:2.29127031005919 - timing_s/adv:0.11943884380161762 - timing_s/update_actor:9.831803061068058 - timing_s/step:23.728821083903313 - timing_s/stop_profile:0.00010754726827144623 - timing_per_token_ms/update_actor:0.054341570924400354 - timing_per_token_ms/adv:0.0006601530117374928 - timing_per_token_ms/gen:0.14004170955166706 - perf/total_num_tokens:180926 - perf/time_per_step:23.728821083903313 - perf/throughput:953.0920191960832 (TaskRunner pid=2146199) Training Progress: 25%|██▌ | 25/100 [13:28<32:58, 26.38s/it] (TaskRunner pid=2146199) step:26 - global_seqlen/min:21890 - global_seqlen/max:25976 - global_seqlen/minmax_diff:4086 - global_seqlen/balanced_min:23476 - global_seqlen/balanced_max:23483 - global_seqlen/mean:23481.125 - actor/entropy:0.045579973608255386 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9706757068634033 - training/rollout_probs_diff_mean:0.0026706012431532145 - training/rollout_probs_diff_std:0.01532005611807108 - training/rollout_actor_probs_pearson_corr:0.9912408590316772 - rollout_corr/rollout_is_mean:0.9999898672103882 - rollout_corr/rollout_is_ratio_fraction_high:8.118295227177441e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0003827196196652949 - rollout_corr/rollout_is_max:3.0358901023864746 - rollout_corr/rollout_is_min:1.1825401998066809e-05 - rollout_corr/rollout_is_std:0.03693227097392082 - rollout_corr/rollout_is_eff_sample_size:0.9986379841355687 - rollout_corr/rollout_is_seq_mean:0.9999938011169434 - rollout_corr/rollout_is_seq_std:0.002214487874880433 - rollout_corr/rollout_is_seq_max:1.0080057382583618 - rollout_corr/rollout_is_seq_min:0.9936314821243286 - rollout_corr/rollout_is_seq_max_deviation:0.008005738258361816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0473792543634772) - rollout_corr/training_log_ppl:np.float64(0.04601353934776853) - rollout_corr/kl:np.float64(0.0009864631402276558) - rollout_corr/k3_kl:np.float64(0.0009527379513656342) - rollout_corr/rollout_ppl:np.float64(1.0463388008065522) - rollout_corr/rollout_log_ppl:np.float64(0.045027075730104116) - rollout_corr/log_ppl_diff:np.float64(0.0009864636176644126) - rollout_corr/log_ppl_abs_diff:np.float64(0.002059470103631611) - rollout_corr/log_ppl_diff_max:np.float64(0.01571808010339737) - rollout_corr/log_ppl_diff_min:np.float64(-0.006790239363908768) - rollout_corr/ppl_ratio:np.float64(1.0009906524792314) - rollout_corr/chi2_token:np.float64(0.0019492092542350292) - rollout_corr/chi2_seq:np.float64(1.0629137614741921) - actor/pg_loss:np.float64(3.956095315515995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000325970333051373) - actor/ppo_kl:np.float64(0.00016295430630419716) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23984023183584213) - perf/mfu/actor:np.float64(0.06821446108850142) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(101.53064727783203) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002621049527078867 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002621049527078867 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:336.81640625 - response_length/max:701.0 - response_length/min:165.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:336.81640625 - response_length_non_aborted/max:701.0 - response_length_non_aborted/min:165.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.96875 - prompt_length/max:491.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001916065812110901 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8374372590333223) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8689656760543585) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6888440722759697) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8689656760543585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:661 - timing_s/gen:11.604685235768557 - timing_s/reward:0.05911627784371376 - timing_s/old_log_prob:2.316767279058695 - timing_s/adv:0.11668118275702 - timing_s/update_actor:9.850833578035235 - timing_s/step:24.03637538291514 - timing_s/stop_profile:0.00011649169027805328 - timing_per_token_ms/update_actor:0.05244017044559852 - timing_per_token_ms/adv:0.0006211434862949496 - timing_per_token_ms/gen:0.1345860856569273 - perf/total_num_tokens:187849 - perf/time_per_step:24.03637538291514 - perf/throughput:976.8995793221882 (TaskRunner pid=2146199) Training Progress: 26%|██▌ | 26/100 [13:52<31:41, 25.70s/it] (TaskRunner pid=2146199) step:27 - global_seqlen/min:20113 - global_seqlen/max:26304 - global_seqlen/minmax_diff:6191 - global_seqlen/balanced_min:23304 - global_seqlen/balanced_max:23307 - global_seqlen/mean:23305.75 - actor/entropy:0.03360840678215027 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7172134518623352 - training/rollout_probs_diff_mean:0.0018713483586907387 - training/rollout_probs_diff_std:0.012340115383267403 - training/rollout_actor_probs_pearson_corr:0.9924281239509583 - rollout_corr/rollout_is_mean:1.0001565217971802 - rollout_corr/rollout_is_ratio_fraction_high:0.00014184732572175562 - rollout_corr/rollout_is_ratio_fraction_low:0.00024823282728902996 - rollout_corr/rollout_is_max:4.4510602951049805 - rollout_corr/rollout_is_min:0.18729040026664734 - rollout_corr/rollout_is_std:0.03218955174088478 - rollout_corr/rollout_is_eff_sample_size:0.9989651433115747 - rollout_corr/rollout_is_seq_mean:1.0001486539840698 - rollout_corr/rollout_is_seq_std:0.00219075009226799 - rollout_corr/rollout_is_seq_max:1.0124483108520508 - rollout_corr/rollout_is_seq_min:0.9954181909561157 - rollout_corr/rollout_is_seq_max_deviation:0.012448310852050781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0357399084605277) - rollout_corr/training_log_ppl:np.float64(0.0347986096785462) - rollout_corr/kl:np.float64(0.0008901205218698749) - rollout_corr/k3_kl:np.float64(0.0009873300563949527) - rollout_corr/rollout_ppl:np.float64(1.0348039334639907) - rollout_corr/rollout_log_ppl:np.float64(0.0339084887127683) - rollout_corr/log_ppl_diff:np.float64(0.0008901209657778963) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020289634303480852) - rollout_corr/log_ppl_diff_max:np.float64(0.032865457236766815) - rollout_corr/log_ppl_diff_min:np.float64(-0.007571380585432053) - rollout_corr/ppl_ratio:np.float64(1.000895927194506) - rollout_corr/chi2_token:np.float64(0.0020306783262640238) - rollout_corr/chi2_seq:np.float64(0.9142170131672174) - actor/pg_loss:np.float64(3.3000833354890347e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024783711933196173) - actor/ppo_kl:np.float64(0.00044361639040602086) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2805748023092747) - perf/mfu/actor:np.float64(0.06768846515249424) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.25595092773438) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004546502139419317 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.004546502139419317 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:330.4609375 - response_length/max:808.0 - response_length/min:166.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:330.4609375 - response_length_non_aborted/max:808.0 - response_length_non_aborted/min:166.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.84375 - prompt_length/max:494.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00021702423691749573 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9053292758762836) - timing_s/agent_loop/generate_sequences/max:np.float64(6.251024179160595) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.584437644276477) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.251024179160595) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:808 - timing_s/gen:12.235861191526055 - timing_s/reward:0.05250731110572815 - timing_s/old_log_prob:2.481621041893959 - timing_s/adv:0.14334496296942234 - timing_s/update_actor:9.925678854808211 - timing_s/step:24.9610702842474 - timing_s/stop_profile:0.00013760477304458618 - timing_per_token_ms/update_actor:0.05323621238754498 - timing_per_token_ms/adv:0.0007688283093733432 - timing_per_token_ms/gen:0.14463534825322177 - perf/total_num_tokens:186446 - perf/time_per_step:24.9610702842474 - perf/throughput:933.6839219874298 (TaskRunner pid=2146199) Training Progress: 27%|██▋ | 27/100 [14:17<31:08, 25.60s/it] (TaskRunner pid=2146199) step:28 - global_seqlen/min:19880 - global_seqlen/max:27721 - global_seqlen/minmax_diff:7841 - global_seqlen/balanced_min:23130 - global_seqlen/balanced_max:23140 - global_seqlen/mean:23137.0 - actor/entropy:0.05440784618258476 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4863184690475464 - training/rollout_probs_diff_mean:0.0024080276489257812 - training/rollout_probs_diff_std:0.013055422343313694 - training/rollout_actor_probs_pearson_corr:0.9944478273391724 - rollout_corr/rollout_is_mean:1.000238299369812 - rollout_corr/rollout_is_ratio_fraction_high:9.016319381771609e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00025921917404048145 - rollout_corr/rollout_is_max:7.410891056060791 - rollout_corr/rollout_is_min:0.13879206776618958 - rollout_corr/rollout_is_std:0.034511156380176544 - rollout_corr/rollout_is_eff_sample_size:0.9988107536069124 - rollout_corr/rollout_is_seq_mean:1.0002248287200928 - rollout_corr/rollout_is_seq_std:0.0022856718860566616 - rollout_corr/rollout_is_seq_max:1.0136717557907104 - rollout_corr/rollout_is_seq_min:0.993253767490387 - rollout_corr/rollout_is_seq_max_deviation:0.01367175579071045 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0518874395638704) - rollout_corr/training_log_ppl:np.float64(0.049300971942557226) - rollout_corr/kl:np.float64(0.0007346817514148896) - rollout_corr/k3_kl:np.float64(0.000867667970940289) - rollout_corr/rollout_ppl:np.float64(1.0511001562699676) - rollout_corr/rollout_log_ppl:np.float64(0.04856628991092293) - rollout_corr/log_ppl_diff:np.float64(0.0007346820316342928) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017266277386625006) - rollout_corr/log_ppl_diff_max:np.float64(0.00947115570306778) - rollout_corr/log_ppl_diff_min:np.float64(-0.006286837160587311) - rollout_corr/ppl_ratio:np.float64(1.0007377294823527) - rollout_corr/chi2_token:np.float64(0.0020542640704661608) - rollout_corr/chi2_seq:np.float64(0.4279042992275208) - actor/pg_loss:np.float64(-4.434306174516678e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023718912325421115) - actor/ppo_kl:np.float64(0.00025536056455521816) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23834340274333954) - perf/mfu/actor:np.float64(0.06741080035227487) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.25299072265625) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004911076743155718 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004911076743155718 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:346.59375 - response_length/max:828.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.59375 - response_length_non_aborted/max:828.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:376.4375 - prompt_length/max:478.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014181248843669891 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1437689363956451) - timing_s/agent_loop/generate_sequences/max:np.float64(6.196736728772521) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.438780921183934) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.196736728772521) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:755 - timing_s/gen:11.902349235489964 - timing_s/reward:0.0586091224104166 - timing_s/old_log_prob:2.3908287342637777 - timing_s/adv:0.12477444484829903 - timing_s/update_actor:9.773709451779723 - timing_s/step:24.33500487729907 - timing_s/stop_profile:0.000125860795378685 - timing_per_token_ms/update_actor:0.05280346118651793 - timing_per_token_ms/adv:0.0006741066519443912 - timing_per_token_ms/gen:0.13414422995548153 - perf/total_num_tokens:185096 - perf/time_per_step:24.33500487729907 - perf/throughput:950.7703046151172 (TaskRunner pid=2146199) Training Progress: 28%|██▊ | 28/100 [14:42<30:17, 25.24s/it] (TaskRunner pid=2146199) step:29 - global_seqlen/min:19790 - global_seqlen/max:28657 - global_seqlen/minmax_diff:8867 - global_seqlen/balanced_min:23982 - global_seqlen/balanced_max:23987 - global_seqlen/mean:23984.625 - actor/entropy:0.030021118000149727 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6650738716125488 - training/rollout_probs_diff_mean:0.0018870807252824306 - training/rollout_probs_diff_std:0.012741130776703358 - training/rollout_actor_probs_pearson_corr:0.9910599589347839 - rollout_corr/rollout_is_mean:1.0000355243682861 - rollout_corr/rollout_is_ratio_fraction_high:0.0001472237054258585 - rollout_corr/rollout_is_ratio_fraction_low:0.0003284220874775201 - rollout_corr/rollout_is_max:5.969290256500244 - rollout_corr/rollout_is_min:0.02673604153096676 - rollout_corr/rollout_is_std:0.033001456409692764 - rollout_corr/rollout_is_eff_sample_size:0.9989119698204697 - rollout_corr/rollout_is_seq_mean:1.0000556707382202 - rollout_corr/rollout_is_seq_std:0.002674776129424572 - rollout_corr/rollout_is_seq_max:1.017096996307373 - rollout_corr/rollout_is_seq_min:0.9926179647445679 - rollout_corr/rollout_is_seq_max_deviation:0.017096996307373047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0329697006382048) - rollout_corr/training_log_ppl:np.float64(0.032201128145061375) - rollout_corr/kl:np.float64(0.0009554920058079119) - rollout_corr/k3_kl:np.float64(0.0009288907390754275) - rollout_corr/rollout_ppl:np.float64(1.0319800111465156) - rollout_corr/rollout_log_ppl:np.float64(0.031245635787854553) - rollout_corr/log_ppl_diff:np.float64(0.0009554923572068219) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018781634162223781) - rollout_corr/log_ppl_diff_max:np.float64(0.015385210514068604) - rollout_corr/log_ppl_diff_min:np.float64(-0.007063932716846466) - rollout_corr/ppl_ratio:np.float64(1.0009591272100806) - rollout_corr/chi2_token:np.float64(0.0030820860993117094) - rollout_corr/chi2_seq:np.float64(1.002304163761437) - actor/pg_loss:np.float64(0.00011053378693759441) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00043514460821825196) - actor/ppo_kl:np.float64(0.00021312317062438524) - actor/pg_clipfrac_lower:np.float64(1.0306728654541075e-05) - actor/grad_norm:np.float64(0.2175781037658453) - perf/mfu/actor:np.float64(0.06936433677395533) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.33856582641602) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026202986482530832 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026202986482530832 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.92578125 - response_length/max:739.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.92578125 - response_length_non_aborted/max:739.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:404.59375 - prompt_length/max:512.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013334676623344421 - timing_s/agent_loop/generate_sequences/min:np.float64(1.743859464302659) - timing_s/agent_loop/generate_sequences/max:np.float64(6.019832329824567) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6303252360521583) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.019832329824567) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:512 - timing_s/agent_loop/slowest/response_length:739 - timing_s/gen:11.951574511826038 - timing_s/reward:0.0616659726947546 - timing_s/old_log_prob:2.3734438735991716 - timing_s/adv:0.1184399276971817 - timing_s/update_actor:9.913735531270504 - timing_s/step:24.506156511604786 - timing_s/stop_profile:0.00011850334703922272 - timing_per_token_ms/update_actor:0.051667138485959774 - timing_per_token_ms/adv:0.0006172700620563262 - timing_per_token_ms/gen:0.13535038687926568 - perf/total_num_tokens:191877 - perf/time_per_step:24.506156511604786 - perf/throughput:978.7183473117126 (TaskRunner pid=2146199) Training Progress: 29%|██▉ | 29/100 [15:06<29:37, 25.03s/it] (TaskRunner pid=2146199) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2146199) validation generation end (TaskRunner pid=2146199) [prompt] system (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) The unit cell volume for a crystal with orthogonal lattice parameters (α = β = γ = 90.00º) can be calculated using the formula: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = a \times b \times c $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given: (TaskRunner pid=2146199) - $ a = 7.82 $ Å (TaskRunner pid=2146199) - $ b = 7.82 $ Å (TaskRunner pid=2146199) - $ c = 17.88 $ Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Substituting the values: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Then multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 61.1524 \times 17.88 \approx 1094.31 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2146577) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2146199) step:30 - global_seqlen/min:20834 - global_seqlen/max:31260 - global_seqlen/minmax_diff:10426 - global_seqlen/balanced_min:26154 - global_seqlen/balanced_max:26159 - global_seqlen/mean:26157.625 - actor/entropy:0.04098328948020935 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7479928135871887 - training/rollout_probs_diff_mean:0.002162351505830884 - training/rollout_probs_diff_std:0.013324693776667118 - training/rollout_actor_probs_pearson_corr:0.9926473498344421 - rollout_corr/rollout_is_mean:1.0001035928726196 - rollout_corr/rollout_is_ratio_fraction_high:0.00011599470599321648 - rollout_corr/rollout_is_ratio_fraction_low:0.00019332450756337494 - rollout_corr/rollout_is_max:4.358939170837402 - rollout_corr/rollout_is_min:0.017727527767419815 - rollout_corr/rollout_is_std:0.033197738230228424 - rollout_corr/rollout_is_eff_sample_size:0.9988993612770696 - rollout_corr/rollout_is_seq_mean:1.0001381635665894 - rollout_corr/rollout_is_seq_std:0.002059181686490774 - rollout_corr/rollout_is_seq_max:1.0120402574539185 - rollout_corr/rollout_is_seq_min:0.991795003414154 - rollout_corr/rollout_is_seq_max_deviation:0.012040257453918457 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.047046871855855) - rollout_corr/training_log_ppl:np.float64(0.04482210733476677) - rollout_corr/kl:np.float64(0.000570720605579389) - rollout_corr/k3_kl:np.float64(0.0009003356013366215) - rollout_corr/rollout_ppl:np.float64(1.0464187851175666) - rollout_corr/rollout_log_ppl:np.float64(0.044251386911128066) - rollout_corr/log_ppl_diff:np.float64(0.0005707204236387042) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015987365750333993) - rollout_corr/log_ppl_diff_max:np.float64(0.01038980484008789) - rollout_corr/log_ppl_diff_min:np.float64(-0.0075468141585588455) - rollout_corr/ppl_ratio:np.float64(1.0005733871366829) - rollout_corr/chi2_token:np.float64(0.0030484930612146854) - rollout_corr/chi2_seq:np.float64(0.9922418568748981) - actor/pg_loss:np.float64(-5.303020589053631e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002557900356805476) - actor/ppo_kl:np.float64(8.289198712319035e-06) - actor/pg_clipfrac_lower:np.float64(1.169535971712321e-05) - actor/grad_norm:np.float64(0.25283773243427277) - perf/mfu/actor:np.float64(0.07488209255568148) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.44367218017578) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7167553191489362) - val-aux/sciknoweval/reward/std@16:np.float64(0.04836419760222217) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.734031914893617) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.02920582335590251) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6987659574468085) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.05227484092241596) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7164574468085108) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.048321703643777844) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7433404255319148) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.015697991544214862) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6767978723404255) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.05188436658008571) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7230425531914895) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.034521592859457155) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7482446808510639) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.008716267240126854) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.654031914893617) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.0445327976940271) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7267765957446808) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.02009514118888133) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7512978723404256) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.005961771227041093) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6345212765957446) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03192228855632085) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7268085106382978) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.011081133612329246) - val-aux/sciknoweval/score/mean@16:np.float64(0.7167553191489362) - val-aux/sciknoweval/score/std@16:np.float64(0.04836419760222217) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.734031914893617) - val-aux/sciknoweval/score/best@2/std:np.float64(0.02920582335590251) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6987659574468085) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.05227484092241596) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7164574468085108) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.048321703643777844) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7433404255319148) - val-aux/sciknoweval/score/best@4/std:np.float64(0.015697991544214862) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6767978723404255) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.05188436658008571) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7230425531914895) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.034521592859457155) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7482446808510639) - val-aux/sciknoweval/score/best@8/std:np.float64(0.008716267240126854) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.654031914893617) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.0445327976940271) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7267765957446808) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.02009514118888133) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7512978723404256) - val-aux/sciknoweval/score/best@16/std:np.float64(0.005961771227041093) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6345212765957446) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03192228855632085) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7268085106382978) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.011081133612329246) - val-core/sciknoweval/acc/mean@16:np.float64(0.7167553191489362) - val-aux/sciknoweval/acc/std@16:np.float64(0.04836419760222217) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.734031914893617) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.02920582335590251) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6987659574468085) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.05227484092241596) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7164574468085108) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.048321703643777844) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7433404255319148) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.015697991544214862) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6767978723404255) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.05188436658008571) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7230425531914895) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.034521592859457155) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7482446808510639) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.008716267240126854) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.654031914893617) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.0445327976940271) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7267765957446808) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.02009514118888133) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7512978723404256) - val-core/sciknoweval/acc/best@16/std:np.float64(0.005961771227041093) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6345212765957446) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03192228855632085) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7268085106382978) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.011081133612329246) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005803360138088465 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.005803360138088465 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:404.11328125 - response_length/max:831.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.11328125 - response_length_non_aborted/max:831.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:413.3125 - prompt_length/max:482.0 - prompt_length/min:206.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011833757162094116 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5526590440422297) - timing_s/agent_loop/generate_sequences/max:np.float64(6.901175409555435) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.24034725362435) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.901175409555435) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:831 - timing_s/gen:12.894607612863183 - timing_s/reward:0.06270061247050762 - timing_s/old_log_prob:2.333733009174466 - timing_s/adv:0.12523001059889793 - timing_s/update_actor:9.932297594845295 - timing_s/step:25.434717739000916 - timing_s/testing:39.29435325041413 - timing_s/save_checkpoint:6.898772217333317 - timing_s/stop_profile:0.00016747228801250458 - timing_per_token_ms/update_actor:0.04746368217128512 - timing_per_token_ms/adv:0.0005984393202694145 - timing_per_token_ms/gen:0.12464218159805113 - perf/total_num_tokens:209261 - perf/time_per_step:25.434717739000916 - perf/throughput:1028.4220673654497 (TaskRunner pid=2146199) Training Progress: 30%|███ | 30/100 [16:18<45:32, 39.04s/it] (TaskRunner pid=2146199) step:31 - global_seqlen/min:22362 - global_seqlen/max:26133 - global_seqlen/minmax_diff:3771 - global_seqlen/balanced_min:24193 - global_seqlen/balanced_max:24197 - global_seqlen/mean:24195.625 - actor/entropy:0.041051458567380905 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5331467390060425 - training/rollout_probs_diff_mean:0.0020818959455937147 - training/rollout_probs_diff_std:0.012100167572498322 - training/rollout_actor_probs_pearson_corr:0.9940868020057678 - rollout_corr/rollout_is_mean:1.000025749206543 - rollout_corr/rollout_is_ratio_fraction_high:6.494842091342434e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015154630818869919 - rollout_corr/rollout_is_max:2.6787970066070557 - rollout_corr/rollout_is_min:0.06881428509950638 - rollout_corr/rollout_is_std:0.03130907192826271 - rollout_corr/rollout_is_eff_sample_size:0.9990207019767339 - rollout_corr/rollout_is_seq_mean:1.0000669956207275 - rollout_corr/rollout_is_seq_std:0.0018148089293390512 - rollout_corr/rollout_is_seq_max:1.0074193477630615 - rollout_corr/rollout_is_seq_min:0.993800938129425 - rollout_corr/rollout_is_seq_max_deviation:0.0074193477630615234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0434305830858648) - rollout_corr/training_log_ppl:np.float64(0.04203238038462587) - rollout_corr/kl:np.float64(0.0007811994401852596) - rollout_corr/k3_kl:np.float64(0.0010347507429706582) - rollout_corr/rollout_ppl:np.float64(1.0426035225391388) - rollout_corr/rollout_log_ppl:np.float64(0.041251180472045235) - rollout_corr/log_ppl_diff:np.float64(0.0007811999125806324) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020230914319654403) - rollout_corr/log_ppl_diff_max:np.float64(0.012856822460889816) - rollout_corr/log_ppl_diff_min:np.float64(-0.010156881995499134) - rollout_corr/ppl_ratio:np.float64(1.0007853372953832) - rollout_corr/chi2_token:np.float64(0.002933218376711011) - rollout_corr/chi2_seq:np.float64(1.7169771187473089) - actor/pg_loss:np.float64(0.00024299102369695902) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033148087550216587) - actor/ppo_kl:np.float64(0.0003257236089493354) - actor/pg_clipfrac_lower:np.float64(2.867059356503887e-05) - actor/grad_norm:np.float64(0.1899427566677332) - perf/mfu/actor:np.float64(0.06967584496041151) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.16240692138672) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014708110829815269 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014708110829815269 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.86328125 - response_length/max:872.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.86328125 - response_length_non_aborted/max:872.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:395.25 - prompt_length/max:492.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.458667874336243e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.757045580074191) - timing_s/agent_loop/generate_sequences/max:np.float64(6.3989690616726875) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7471343511351733) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.3989690616726875) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:271 - timing_s/agent_loop/slowest/response_length:872 - timing_s/gen:12.270365277305245 - timing_s/reward:0.05584356002509594 - timing_s/old_log_prob:2.4889576714485884 - timing_s/adv:0.135251484811306 - timing_s/update_actor:9.869210064411163 - timing_s/step:24.91895973496139 - timing_s/stop_profile:2.6123598217964172e-05 - timing_per_token_ms/update_actor:0.05098654232124177 - timing_per_token_ms/adv:0.0006987393630630848 - timing_per_token_ms/gen:0.13282347319584378 - perf/total_num_tokens:193565 - perf/time_per_step:24.91895973496139 - perf/throughput:970.9725147977766 (TaskRunner pid=2146199) Training Progress: 31%|███ | 31/100 [16:43<40:01, 34.81s/it] (TaskRunner pid=2146199) step:32 - global_seqlen/min:18953 - global_seqlen/max:28527 - global_seqlen/minmax_diff:9574 - global_seqlen/balanced_min:24327 - global_seqlen/balanced_max:24331 - global_seqlen/mean:24329.875 - actor/entropy:0.037253238260746 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5851324200630188 - training/rollout_probs_diff_mean:0.0019729994237422943 - training/rollout_probs_diff_std:0.01258387602865696 - training/rollout_actor_probs_pearson_corr:0.9927456974983215 - rollout_corr/rollout_is_mean:1.0000933408737183 - rollout_corr/rollout_is_ratio_fraction_high:9.65841391007416e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001931682782014832 - rollout_corr/rollout_is_max:3.0360348224639893 - rollout_corr/rollout_is_min:0.059966862201690674 - rollout_corr/rollout_is_std:0.03227276727557182 - rollout_corr/rollout_is_eff_sample_size:0.9989597900192954 - rollout_corr/rollout_is_seq_mean:1.000119924545288 - rollout_corr/rollout_is_seq_std:0.002079813042655587 - rollout_corr/rollout_is_seq_max:1.0103451013565063 - rollout_corr/rollout_is_seq_min:0.9926933646202087 - rollout_corr/rollout_is_seq_max_deviation:0.010345101356506348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0397841832600534) - rollout_corr/training_log_ppl:np.float64(0.03838184628239105) - rollout_corr/kl:np.float64(0.0006357688799791816) - rollout_corr/k3_kl:np.float64(0.0006630392528705897) - rollout_corr/rollout_ppl:np.float64(1.039116082713008) - rollout_corr/rollout_log_ppl:np.float64(0.037746077452538884) - rollout_corr/log_ppl_diff:np.float64(0.000635768829852168) - rollout_corr/log_ppl_abs_diff:np.float64(0.001663975653173111) - rollout_corr/log_ppl_diff_max:np.float64(0.011524975299835205) - rollout_corr/log_ppl_diff_min:np.float64(-0.005549527704715729) - rollout_corr/ppl_ratio:np.float64(1.0006383440922946) - rollout_corr/chi2_token:np.float64(0.0013726239558309317) - rollout_corr/chi2_seq:np.float64(0.4417000797111541) - actor/pg_loss:np.float64(-4.8585934564471245e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001409936012350954) - actor/ppo_kl:np.float64(0.00016448168571425725) - actor/pg_clipfrac_lower:np.float64(7.2472171268600505e-06) - actor/grad_norm:np.float64(0.19629834406077862) - perf/mfu/actor:np.float64(0.0704994224106969) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.36459732055664) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003498492296785116 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003498492296785116 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:363.99609375 - response_length/max:829.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:363.99609375 - response_length_non_aborted/max:829.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6382505893707275e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7081327475607395) - timing_s/agent_loop/generate_sequences/max:np.float64(6.998795060440898) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.891505555024196) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.998795060440898) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:419 - timing_s/agent_loop/slowest/response_length:819 - timing_s/gen:12.932953741401434 - timing_s/reward:0.06094272807240486 - timing_s/old_log_prob:2.4036419335752726 - timing_s/adv:0.1503080315887928 - timing_s/update_actor:9.872573409229517 - timing_s/step:25.507261792197824 - timing_s/stop_profile:5.361437797546387e-05 - timing_per_token_ms/update_actor:0.05072248320855284 - timing_per_token_ms/adv:0.0007722400525526375 - timing_per_token_ms/gen:0.13879091402295948 - perf/total_num_tokens:194639 - perf/time_per_step:25.507261792197824 - perf/throughput:953.8411138839699 (TaskRunner pid=2146199) Training Progress: 32%|███▏ | 32/100 [17:08<36:17, 32.03s/it] (TaskRunner pid=2146199) step:33 - global_seqlen/min:19039 - global_seqlen/max:24201 - global_seqlen/minmax_diff:5162 - global_seqlen/balanced_min:22979 - global_seqlen/balanced_max:22984 - global_seqlen/mean:22982.25 - actor/entropy:0.041336026042699814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7802475094795227 - training/rollout_probs_diff_mean:0.0020745492074638605 - training/rollout_probs_diff_std:0.013532165437936783 - training/rollout_actor_probs_pearson_corr:0.9925845861434937 - rollout_corr/rollout_is_mean:1.0001330375671387 - rollout_corr/rollout_is_ratio_fraction_high:9.385045268572867e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016423828492406756 - rollout_corr/rollout_is_max:12.166744232177734 - rollout_corr/rollout_is_min:0.00915502943098545 - rollout_corr/rollout_is_std:0.03246978670358658 - rollout_corr/rollout_is_eff_sample_size:0.9989469423519113 - rollout_corr/rollout_is_seq_mean:1.0001661777496338 - rollout_corr/rollout_is_seq_std:0.002779231406748295 - rollout_corr/rollout_is_seq_max:1.0319596529006958 - rollout_corr/rollout_is_seq_min:0.9946264624595642 - rollout_corr/rollout_is_seq_max_deviation:0.0319596529006958 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.041108446661383) - rollout_corr/training_log_ppl:np.float64(0.03940494276866957) - rollout_corr/kl:np.float64(0.0007135979777146417) - rollout_corr/k3_kl:np.float64(0.0009249509639310816) - rollout_corr/rollout_ppl:np.float64(1.0403625490143895) - rollout_corr/rollout_log_ppl:np.float64(0.03869134492742887) - rollout_corr/log_ppl_diff:np.float64(0.0007135978485166561) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017976856161112664) - rollout_corr/log_ppl_diff_max:np.float64(0.03488359600305557) - rollout_corr/log_ppl_diff_min:np.float64(-0.006541695445775986) - rollout_corr/ppl_ratio:np.float64(1.000718776602298) - rollout_corr/chi2_token:np.float64(0.0032578017562627792) - rollout_corr/chi2_seq:np.float64(0.6134596411138773) - actor/pg_loss:np.float64(6.561563350260258e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0001494009704003929) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08391597121953964) - perf/mfu/actor:np.float64(0.06693175766848874) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.45356750488281) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002111635112669319 - critic/advantages/max:0.75 - critic/advantages/min:-0.25 - critic/returns/mean:0.0002111635112669319 - critic/returns/max:0.75 - critic/returns/min:-0.25 - response_length/mean:332.9765625 - response_length/max:670.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.9765625 - response_length_non_aborted/max:670.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.21875 - prompt_length/max:480.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.881620407104492e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2467848639935255) - timing_s/agent_loop/generate_sequences/max:np.float64(5.626865619793534) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3089464717486408) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.626865619793534) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:651 - timing_s/gen:11.545319786295295 - timing_s/reward:0.05907109193503857 - timing_s/old_log_prob:2.339627142995596 - timing_s/adv:0.1317504532635212 - timing_s/update_actor:9.780792701989412 - timing_s/step:23.94333771057427 - timing_s/stop_profile:0.0001141224056482315 - timing_per_token_ms/update_actor:0.05319753669674103 - timing_per_token_ms/adv:0.0007165880911547019 - timing_per_token_ms/gen:0.13544168117002528 - perf/total_num_tokens:183858 - perf/time_per_step:23.94333771057427 - perf/throughput:959.8599108364989 (TaskRunner pid=2146199) Training Progress: 33%|███▎ | 33/100 [17:32<33:04, 29.61s/it] (TaskRunner pid=2146199) step:34 - global_seqlen/min:21562 - global_seqlen/max:27886 - global_seqlen/minmax_diff:6324 - global_seqlen/balanced_min:24394 - global_seqlen/balanced_max:24397 - global_seqlen/mean:24396.0 - actor/entropy:0.031612105667591095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975830912590027 - training/rollout_probs_diff_mean:0.0018762743566185236 - training/rollout_probs_diff_std:0.011752952821552753 - training/rollout_actor_probs_pearson_corr:0.9924901127815247 - rollout_corr/rollout_is_mean:0.9999131560325623 - rollout_corr/rollout_is_ratio_fraction_high:0.00012340693501755595 - rollout_corr/rollout_is_ratio_fraction_low:0.00011218811414437369 - rollout_corr/rollout_is_max:3.103707790374756 - rollout_corr/rollout_is_min:0.13723966479301453 - rollout_corr/rollout_is_std:0.030719062313437462 - rollout_corr/rollout_is_eff_sample_size:0.9990567529817086 - rollout_corr/rollout_is_seq_mean:0.9999560117721558 - rollout_corr/rollout_is_seq_std:0.0018219159683212638 - rollout_corr/rollout_is_seq_max:1.0097365379333496 - rollout_corr/rollout_is_seq_min:0.9951523542404175 - rollout_corr/rollout_is_seq_max_deviation:0.00973653793334961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0318559589795768) - rollout_corr/training_log_ppl:np.float64(0.031117280894250143) - rollout_corr/kl:np.float64(0.0006807175022558454) - rollout_corr/k3_kl:np.float64(0.0006105148828829954) - rollout_corr/rollout_ppl:np.float64(1.0311444289982319) - rollout_corr/rollout_log_ppl:np.float64(0.03043656325644406) - rollout_corr/log_ppl_diff:np.float64(0.0006807176378060831) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015738616366434144) - rollout_corr/log_ppl_diff_max:np.float64(0.009404921904206276) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052501969039440155) - rollout_corr/ppl_ratio:np.float64(1.0006830974016339) - rollout_corr/chi2_token:np.float64(0.0011129339691251516) - rollout_corr/chi2_seq:np.float64(0.42043763399124146) - actor/pg_loss:np.float64(5.0016213208436966e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002445464988340973) - actor/ppo_kl:np.float64(0.0001362860669615884) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.203148128464818) - perf/mfu/actor:np.float64(0.07051054867183593) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.53108596801758) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003320768242701888 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:-0.003320768242701888 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:348.1875 - response_length/max:792.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.1875 - response_length_non_aborted/max:792.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:414.1875 - prompt_length/max:482.0 - prompt_length/min:271.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015438348054885864 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0449177529662848) - timing_s/agent_loop/generate_sequences/max:np.float64(6.294473625719547) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.808839218596404) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.294473625719547) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:792 - timing_s/gen:12.141927586868405 - timing_s/reward:0.06110231205821037 - timing_s/old_log_prob:2.4126295670866966 - timing_s/adv:0.13646303676068783 - timing_s/update_actor:10.055538360029459 - timing_s/step:24.893884820863605 - timing_s/stop_profile:0.00011407583951950073 - timing_per_token_ms/update_actor:0.05152247479110028 - timing_per_token_ms/adv:0.0006992080502986547 - timing_per_token_ms/gen:0.13621799931417616 - perf/total_num_tokens:195168 - perf/time_per_step:24.893884820863605 - perf/throughput:979.9997138073714 (TaskRunner pid=2146199) Training Progress: 34%|███▍ | 34/100 [17:57<31:02, 28.22s/it] (TaskRunner pid=2146199) step:35 - global_seqlen/min:20215 - global_seqlen/max:25344 - global_seqlen/minmax_diff:5129 - global_seqlen/balanced_min:21961 - global_seqlen/balanced_max:21963 - global_seqlen/mean:21961.875 - actor/entropy:0.034592803567647934 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8776243329048157 - training/rollout_probs_diff_mean:0.0021816876251250505 - training/rollout_probs_diff_std:0.01385975070297718 - training/rollout_actor_probs_pearson_corr:0.9902499318122864 - rollout_corr/rollout_is_mean:0.9999921917915344 - rollout_corr/rollout_is_ratio_fraction_high:7.768096111249179e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002718833857215941 - rollout_corr/rollout_is_max:3.192864418029785 - rollout_corr/rollout_is_min:0.003602058393880725 - rollout_corr/rollout_is_std:0.03281489759683609 - rollout_corr/rollout_is_eff_sample_size:0.9989242218232214 - rollout_corr/rollout_is_seq_mean:0.9999827146530151 - rollout_corr/rollout_is_seq_std:0.0020841837394982576 - rollout_corr/rollout_is_seq_max:1.008485198020935 - rollout_corr/rollout_is_seq_min:0.9935790300369263 - rollout_corr/rollout_is_seq_max_deviation:0.008485198020935059 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0337875122204423) - rollout_corr/training_log_ppl:np.float64(0.032936989367954084) - rollout_corr/kl:np.float64(0.0019988857879171817) - rollout_corr/k3_kl:np.float64(0.002178219949172444) - rollout_corr/rollout_ppl:np.float64(1.0316676865331829) - rollout_corr/rollout_log_ppl:np.float64(0.03093810328573454) - rollout_corr/log_ppl_diff:np.float64(0.0019988860494777327) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031889932415651856) - rollout_corr/log_ppl_diff_max:np.float64(0.09800203144550323) - rollout_corr/log_ppl_diff_min:np.float64(-0.015378333628177643) - rollout_corr/ppl_ratio:np.float64(1.002046098234132) - rollout_corr/chi2_token:np.float64(0.011006752727553248) - rollout_corr/chi2_seq:np.float64(2.3485067249275744) - actor/pg_loss:np.float64(0.00018588989041745663) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028034445858793333) - actor/ppo_kl:np.float64(0.0013254604241779155) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21878059161826968) - perf/mfu/actor:np.float64(0.06389838683409352) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.46062850952148) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0019630626775324345 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0019630626775324345 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.71484375 - response_length/max:639.0 - response_length/min:162.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.71484375 - response_length_non_aborted/max:639.0 - response_length_non_aborted/min:162.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.59375 - prompt_length/max:459.0 - prompt_length/min:274.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015405751764774323 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9133115001022816) - timing_s/agent_loop/generate_sequences/max:np.float64(5.257205193862319) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.298467695094587) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.257205193862319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:639 - timing_s/gen:11.134622627869248 - timing_s/reward:0.07279274426400661 - timing_s/old_log_prob:2.3010979741811752 - timing_s/adv:0.13786019012331963 - timing_s/update_actor:9.841628031805158 - timing_s/step:23.573449570685625 - timing_s/stop_profile:2.7097761631011963e-05 - timing_per_token_ms/update_actor:0.05601541325481748 - timing_per_token_ms/adv:0.0007846563085080374 - timing_per_token_ms/gen:0.14415803710391445 - perf/total_num_tokens:175695 - perf/time_per_step:23.573449570685625 - perf/throughput:931.6360312115851 (TaskRunner pid=2146199) Training Progress: 35%|███▌ | 35/100 [18:21<29:03, 26.83s/it] (TaskRunner pid=2146199) step:36 - global_seqlen/min:21226 - global_seqlen/max:27449 - global_seqlen/minmax_diff:6223 - global_seqlen/balanced_min:23867 - global_seqlen/balanced_max:23870 - global_seqlen/mean:23868.875 - actor/entropy:0.04816257208585739 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.698370635509491 - training/rollout_probs_diff_mean:0.002242090180516243 - training/rollout_probs_diff_std:0.012639750726521015 - training/rollout_actor_probs_pearson_corr:0.9943584203720093 - rollout_corr/rollout_is_mean:0.9998730421066284 - rollout_corr/rollout_is_ratio_fraction_high:6.732570909662172e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00023563999275211245 - rollout_corr/rollout_is_max:2.667916774749756 - rollout_corr/rollout_is_min:0.1927582025527954 - rollout_corr/rollout_is_std:0.03287297114729881 - rollout_corr/rollout_is_eff_sample_size:0.998920415343582 - rollout_corr/rollout_is_seq_mean:0.9998993873596191 - rollout_corr/rollout_is_seq_std:0.0018510455265641212 - rollout_corr/rollout_is_seq_max:1.0064433813095093 - rollout_corr/rollout_is_seq_min:0.9930350184440613 - rollout_corr/rollout_is_seq_max_deviation:0.006964981555938721 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.044641064479947) - rollout_corr/training_log_ppl:np.float64(0.04320200844267674) - rollout_corr/kl:np.float64(0.0008237751112716296) - rollout_corr/k3_kl:np.float64(0.0008292371547327093) - rollout_corr/rollout_ppl:np.float64(1.043781690299511) - rollout_corr/rollout_log_ppl:np.float64(0.0423782327770823) - rollout_corr/log_ppl_diff:np.float64(0.0008237756655944395) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018528917289586389) - rollout_corr/log_ppl_diff_max:np.float64(0.017406612634658813) - rollout_corr/log_ppl_diff_min:np.float64(-0.004776503890752792) - rollout_corr/ppl_ratio:np.float64(1.0008270821999758) - rollout_corr/chi2_token:np.float64(0.001644051168113947) - rollout_corr/chi2_seq:np.float64(0.6040738048031926) - actor/pg_loss:np.float64(3.29664908349514e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004676401558754151) - actor/ppo_kl:np.float64(0.0001375782363908229) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18394410610198975) - perf/mfu/actor:np.float64(0.06901324852389146) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.48899745941162) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002496661909390241 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002496661909390241 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:348.12109375 - response_length/max:825.0 - response_length/min:169.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:348.12109375 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:169.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.78125 - prompt_length/max:509.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.0588358640670776e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.005093587562442) - timing_s/agent_loop/generate_sequences/max:np.float64(6.43594379350543) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.791974599516834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.43594379350543) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:12.295685900375247 - timing_s/reward:0.05928549915552139 - timing_s/old_log_prob:2.3820836525410414 - timing_s/adv:0.1463883128017187 - timing_s/update_actor:9.901526821777225 - timing_s/step:24.866580214351416 - timing_s/stop_profile:0.00011244602501392365 - timing_per_token_ms/update_actor:0.05185375736066962 - timing_per_token_ms/adv:0.0007666276311813959 - timing_per_token_ms/gen:0.13796929835809701 - perf/total_num_tokens:190951 - perf/time_per_step:24.866580214351416 - perf/throughput:959.8776669026807 (TaskRunner pid=2146199) Training Progress: 36%|███▌ | 36/100 [18:46<28:00, 26.25s/it] (TaskRunner pid=2146199) step:37 - global_seqlen/min:18950 - global_seqlen/max:26830 - global_seqlen/minmax_diff:7880 - global_seqlen/balanced_min:22827 - global_seqlen/balanced_max:22833 - global_seqlen/mean:22830.125 - actor/entropy:0.05485541746020317 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400483965873718 - training/rollout_probs_diff_mean:0.002277080435305834 - training/rollout_probs_diff_std:0.012139431200921535 - training/rollout_actor_probs_pearson_corr:0.9953346252441406 - rollout_corr/rollout_is_mean:1.0000300407409668 - rollout_corr/rollout_is_ratio_fraction_high:4.2929048504447564e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.000246842042542994 - rollout_corr/rollout_is_max:3.121652126312256 - rollout_corr/rollout_is_min:0.29609420895576477 - rollout_corr/rollout_is_std:0.032294921576976776 - rollout_corr/rollout_is_eff_sample_size:0.9989580056012856 - rollout_corr/rollout_is_seq_mean:0.9999511241912842 - rollout_corr/rollout_is_seq_std:0.0018588656093925238 - rollout_corr/rollout_is_seq_max:1.0064083337783813 - rollout_corr/rollout_is_seq_min:0.9936684966087341 - rollout_corr/rollout_is_seq_max_deviation:0.006408333778381348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0570684894919395) - rollout_corr/training_log_ppl:np.float64(0.05434764964229544) - rollout_corr/kl:np.float64(0.0006963051944213206) - rollout_corr/k3_kl:np.float64(0.0008267889464050882) - rollout_corr/rollout_ppl:np.float64(1.0563281043432653) - rollout_corr/rollout_log_ppl:np.float64(0.053651343601814006) - rollout_corr/log_ppl_diff:np.float64(0.0006963060404814314) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018168634305766318) - rollout_corr/log_ppl_diff_max:np.float64(0.014626367017626762) - rollout_corr/log_ppl_diff_min:np.float64(-0.008725512772798538) - rollout_corr/ppl_ratio:np.float64(1.000699708936736) - rollout_corr/chi2_token:np.float64(0.0020924240816384554) - rollout_corr/chi2_seq:np.float64(0.9453298361040652) - actor/pg_loss:np.float64(9.609817061573267e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002787431171782373) - actor/ppo_kl:np.float64(6.046893940592213e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1914802067913115) - perf/mfu/actor:np.float64(0.06615304878928799) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.48441696166992) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.86328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.86328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017694816924631596 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017694816924631596 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:363.97265625 - response_length/max:869.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:363.97265625 - response_length_non_aborted/max:869.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.46875 - prompt_length/max:491.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001492723822593689 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1054687462747097) - timing_s/agent_loop/generate_sequences/max:np.float64(6.8245424050837755) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.485032575088553) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.8245424050837755) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:869 - timing_s/gen:12.756928218528628 - timing_s/reward:0.05746540240943432 - timing_s/old_log_prob:2.372117780148983 - timing_s/adv:0.12212868966162205 - timing_s/update_actor:9.85143380612135 - timing_s/step:25.24659343995154 - timing_s/stop_profile:0.00010420382022857666 - timing_per_token_ms/update_actor:0.05393878595781533 - timing_per_token_ms/adv:0.0006686816742222286 - timing_per_token_ms/gen:0.13691069919109466 - perf/total_num_tokens:182641 - perf/time_per_step:25.24659343995154 - perf/throughput:904.2853664317503 (TaskRunner pid=2146199) Training Progress: 37%|███▋ | 37/100 [19:11<27:15, 25.97s/it] (TaskRunner pid=2146199) step:38 - global_seqlen/min:18926 - global_seqlen/max:27654 - global_seqlen/minmax_diff:8728 - global_seqlen/balanced_min:23901 - global_seqlen/balanced_max:23912 - global_seqlen/mean:23907.75 - actor/entropy:0.047446999698877335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.531129002571106 - training/rollout_probs_diff_mean:0.002227243734523654 - training/rollout_probs_diff_std:0.0123433917760849 - training/rollout_actor_probs_pearson_corr:0.9945385456085205 - rollout_corr/rollout_is_mean:1.0001999139785767 - rollout_corr/rollout_is_ratio_fraction_high:4.389911919133738e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012072257959516719 - rollout_corr/rollout_is_max:12.971456527709961 - rollout_corr/rollout_is_min:0.2179751992225647 - rollout_corr/rollout_is_std:0.033010486513376236 - rollout_corr/rollout_is_eff_sample_size:0.9989117319204782 - rollout_corr/rollout_is_seq_mean:1.0002120733261108 - rollout_corr/rollout_is_seq_std:0.0025841258466243744 - rollout_corr/rollout_is_seq_max:1.0283762216567993 - rollout_corr/rollout_is_seq_min:0.9948088526725769 - rollout_corr/rollout_is_seq_max_deviation:0.028376221656799316 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0457029887475073) - rollout_corr/training_log_ppl:np.float64(0.04414891906162666) - rollout_corr/kl:np.float64(0.0005466930246846857) - rollout_corr/k3_kl:np.float64(0.0006913253129532393) - rollout_corr/rollout_ppl:np.float64(1.0451257401145995) - rollout_corr/rollout_log_ppl:np.float64(0.04360222542345582) - rollout_corr/log_ppl_diff:np.float64(0.0005466936381708365) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016182184390345355) - rollout_corr/log_ppl_diff_max:np.float64(0.0073298439383506775) - rollout_corr/log_ppl_diff_min:np.float64(-0.007042583078145981) - rollout_corr/ppl_ratio:np.float64(1.0005490474868566) - rollout_corr/chi2_token:np.float64(0.0023478365037590265) - rollout_corr/chi2_seq:np.float64(0.6291012752335519) - actor/pg_loss:np.float64(3.431923687458038e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019431741384323686) - actor/ppo_kl:np.float64(0.00013115449044676097) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12189817987382412) - perf/mfu/actor:np.float64(0.06951355703710535) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.47376251220703) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002044052816927433 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0002044052816927433 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:355.9296875 - response_length/max:1084.0 - response_length/min:174.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.9296875 - response_length_non_aborted/max:1084.0 - response_length_non_aborted/min:174.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.1875 - prompt_length/max:541.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013918057084083557 - timing_s/agent_loop/generate_sequences/min:np.float64(2.071706922724843) - timing_s/agent_loop/generate_sequences/max:np.float64(7.753574607893825) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.878488728085358) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.753574607893825) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:1084 - timing_s/gen:13.528071438893676 - timing_s/reward:0.061092011630535126 - timing_s/old_log_prob:2.3009579200297594 - timing_s/adv:0.18098087795078754 - timing_s/update_actor:9.778840530663729 - timing_s/step:25.936472302302718 - timing_s/stop_profile:0.00012098811566829681 - timing_per_token_ms/update_actor:0.05112798428680934 - timing_per_token_ms/adv:0.000946245871897123 - timing_per_token_ms/gen:0.1484676072663324 - perf/total_num_tokens:191262 - perf/time_per_step:25.936472302302718 - perf/throughput:921.7811011976905 (TaskRunner pid=2146199) Training Progress: 38%|███▊ | 38/100 [19:37<26:50, 25.97s/it] (TaskRunner pid=2146199) step:39 - global_seqlen/min:20949 - global_seqlen/max:31407 - global_seqlen/minmax_diff:10458 - global_seqlen/balanced_min:24202 - global_seqlen/balanced_max:24204 - global_seqlen/mean:24203.625 - actor/entropy:0.04480616748332977 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5851317644119263 - training/rollout_probs_diff_mean:0.002163441851735115 - training/rollout_probs_diff_std:0.012231297791004181 - training/rollout_actor_probs_pearson_corr:0.994264543056488 - rollout_corr/rollout_is_mean:0.9999681115150452 - rollout_corr/rollout_is_ratio_fraction_high:4.230968625051901e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00022212584735825658 - rollout_corr/rollout_is_max:2.759589433670044 - rollout_corr/rollout_is_min:0.06433095782995224 - rollout_corr/rollout_is_std:0.03189564496278763 - rollout_corr/rollout_is_eff_sample_size:0.9989835828686019 - rollout_corr/rollout_is_seq_mean:0.999954879283905 - rollout_corr/rollout_is_seq_std:0.0018635618034750223 - rollout_corr/rollout_is_seq_max:1.0056302547454834 - rollout_corr/rollout_is_seq_min:0.99262535572052 - rollout_corr/rollout_is_seq_max_deviation:0.0073746442794799805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0451326332986355) - rollout_corr/training_log_ppl:np.float64(0.04381238060705073) - rollout_corr/kl:np.float64(0.000682713947391278) - rollout_corr/k3_kl:np.float64(0.000664126420105049) - rollout_corr/rollout_ppl:np.float64(1.0444145100191236) - rollout_corr/rollout_log_ppl:np.float64(0.04312966660654638) - rollout_corr/log_ppl_diff:np.float64(0.0006827140005043475) - rollout_corr/log_ppl_abs_diff:np.float64(0.001680082759776269) - rollout_corr/log_ppl_diff_max:np.float64(0.010615622624754906) - rollout_corr/log_ppl_diff_min:np.float64(-0.004345402121543884) - rollout_corr/ppl_ratio:np.float64(1.0006851316429675) - rollout_corr/chi2_token:np.float64(0.00123889883980155) - rollout_corr/chi2_seq:np.float64(0.48666499531827867) - actor/pg_loss:np.float64(3.7066638469696045e-07) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.5882329282467254e-05) - actor/ppo_kl:np.float64(6.605616595578567e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.03712567733600736) - perf/mfu/actor:np.float64(0.07095881272685295) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.39851760864258) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00034244402195326984 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.00034244402195326984 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:369.30078125 - response_length/max:829.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:369.30078125 - response_length_non_aborted/max:829.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:387.0625 - prompt_length/max:505.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014620833098888397 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6433184519410133) - timing_s/agent_loop/generate_sequences/max:np.float64(6.751023404300213) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8786594851044356) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.751023404300213) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:829 - timing_s/gen:12.41662877984345 - timing_s/reward:0.061029620468616486 - timing_s/old_log_prob:2.378844490274787 - timing_s/adv:0.12006829492747784 - timing_s/update_actor:9.741109393537045 - timing_s/step:24.80386934801936 - timing_s/stop_profile:0.00012201443314552307 - timing_per_token_ms/update_actor:0.05030811187134698 - timing_per_token_ms/adv:0.0006200945877295128 - timing_per_token_ms/gen:0.13133591542128228 - perf/total_num_tokens:193629 - perf/time_per_step:24.80386934801936 - perf/throughput:975.8003745465103 (TaskRunner pid=2146199) Training Progress: 39%|███▉ | 39/100 [20:02<26:04, 25.65s/it] (TaskRunner pid=2146199) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2146199) validation generation end (TaskRunner pid=2146199) [prompt] system (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) The given crystal structure has lattice parameters a = 7.82 Å, b = 7.82 Å, and c = 17.88 Å, with all angles α, β, and γ equal to 90.00º. This indicates that the crystal structure is cubic, specifically a rectangular prism (orthorhombic structure), where the unit cell is a rectangular box with all angles being 90 degrees. (TaskRunner pid=2146199) (TaskRunner pid=2146199) The volume $ V $ of a unit cell in an orthorhombic system is calculated using the formula: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ (TaskRunner pid=2146199) V = a \times b \times c (TaskRunner pid=2146199) $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Substituting the given values: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ (TaskRunner pid=2146199) V = 7.82 \times 7.82 \times 17.88 (TaskRunner pid=2146199) $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ (TaskRunner pid=2146199) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2146199) $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Then multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ (TaskRunner pid=2146199) 61.1524 \times 17.88 = 1094.31 (TaskRunner pid=2146199) $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2146199) step:40 - global_seqlen/min:18409 - global_seqlen/max:25737 - global_seqlen/minmax_diff:7328 - global_seqlen/balanced_min:23063 - global_seqlen/balanced_max:23064 - global_seqlen/mean:23063.625 - actor/entropy:0.04833457991480827 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9706875681877136 - training/rollout_probs_diff_mean:0.002290506614372134 - training/rollout_probs_diff_std:0.0133439926430583 - training/rollout_actor_probs_pearson_corr:0.9935715198516846 - rollout_corr/rollout_is_mean:0.9999451637268066 - rollout_corr/rollout_is_ratio_fraction_high:9.174627484753728e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010321455920347944 - rollout_corr/rollout_is_max:4.229223728179932 - rollout_corr/rollout_is_min:0.029312225058674812 - rollout_corr/rollout_is_std:0.03301951289176941 - rollout_corr/rollout_is_eff_sample_size:0.9989107803216455 - rollout_corr/rollout_is_seq_mean:0.9998956918716431 - rollout_corr/rollout_is_seq_std:0.002063117455691099 - rollout_corr/rollout_is_seq_max:1.0090301036834717 - rollout_corr/rollout_is_seq_min:0.9925200939178467 - rollout_corr/rollout_is_seq_max_deviation:0.00903010368347168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0452492511831224) - rollout_corr/training_log_ppl:np.float64(0.04387878982561233) - rollout_corr/kl:np.float64(0.0007843014797828118) - rollout_corr/k3_kl:np.float64(0.0007133167228481341) - rollout_corr/rollout_ppl:np.float64(1.0444238455966115) - rollout_corr/rollout_log_ppl:np.float64(0.04309448781077663) - rollout_corr/log_ppl_diff:np.float64(0.0007843020148357027) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017357429833282367) - rollout_corr/log_ppl_diff_max:np.float64(0.016103308647871017) - rollout_corr/log_ppl_diff_min:np.float64(-0.006824567914009094) - rollout_corr/ppl_ratio:np.float64(1.0007876271847636) - rollout_corr/chi2_token:np.float64(0.001205054810270667) - rollout_corr/chi2_seq:np.float64(1.3779361643828452) - actor/pg_loss:np.float64(-1.5004887245595455e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023513154019383364) - actor/ppo_kl:np.float64(8.206514168662693e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11769850086420774) - perf/mfu/actor:np.float64(0.06763552157335836) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.41744232177734) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7393617021276596) - val-aux/sciknoweval/reward/std@16:np.float64(0.06065668194547041) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7650851063829788) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.03809023285144097) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7141489361702127) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.06397505893909075) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7393297872340425) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.06060239571882074) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7770106382978722) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.015086301007490598) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6836702127659574) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.05698176432442756) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7499148936170213) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.04868065192464131) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7806489361702128) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.006870755974743139) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.659031914893617) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.03901530074037762) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7583297872340425) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.034970553431379225) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7832340425531914) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0054864880801541) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6452127659574468) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.018253869118641186) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7641276595744682) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.024516999425667062) - val-aux/sciknoweval/score/mean@16:np.float64(0.7393617021276596) - val-aux/sciknoweval/score/std@16:np.float64(0.06065668194547041) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7650851063829788) - val-aux/sciknoweval/score/best@2/std:np.float64(0.03809023285144097) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7141489361702127) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.06397505893909075) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7393297872340425) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.06060239571882074) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7770106382978722) - val-aux/sciknoweval/score/best@4/std:np.float64(0.015086301007490598) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6836702127659574) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.05698176432442756) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7499148936170213) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.04868065192464131) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7806489361702128) - val-aux/sciknoweval/score/best@8/std:np.float64(0.006870755974743139) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.659031914893617) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.03901530074037762) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7583297872340425) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.034970553431379225) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7832340425531914) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0054864880801541) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6452127659574468) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.018253869118641186) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7641276595744682) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.024516999425667062) - val-core/sciknoweval/acc/mean@16:np.float64(0.7393617021276596) - val-aux/sciknoweval/acc/std@16:np.float64(0.06065668194547041) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7650851063829788) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.03809023285144097) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7141489361702127) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.06397505893909075) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7393297872340425) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.06060239571882074) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7770106382978722) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.015086301007490598) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6836702127659574) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.05698176432442756) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7499148936170213) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.04868065192464131) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7806489361702128) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.006870755974743139) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.659031914893617) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.03901530074037762) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7583297872340425) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.034970553431379225) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7832340425531914) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0054864880801541) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6452127659574468) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.018253869118641186) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7641276595744682) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.024516999425667062) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031451769173145294 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031451769173145294 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:340.61328125 - response_length/max:841.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:340.61328125 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:380.125 - prompt_length/max:468.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.79006290435791e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7846473064273596) - timing_s/agent_loop/generate_sequences/max:np.float64(6.414747044444084) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5249824257334694) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.414747044444084) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:841 - timing_s/gen:12.175397414714098 - timing_s/reward:0.06089763529598713 - timing_s/old_log_prob:2.262747224420309 - timing_s/adv:0.11960082687437534 - timing_s/update_actor:9.820753671228886 - timing_s/step:24.527741599828005 - timing_s/testing:40.52107731625438 - timing_s/save_checkpoint:6.7504358272999525 - timing_s/stop_profile:0.00010003708302974701 - timing_per_token_ms/update_actor:0.05322642077746281 - timing_per_token_ms/adv:0.0006482113440231932 - timing_per_token_ms/gen:0.1396309209573047 - perf/total_num_tokens:184509 - perf/time_per_step:24.527741599828005 - perf/throughput:940.3077289497264 (TaskRunner pid=2146199) Training Progress: 40%|████ | 40/100 [21:14<39:30, 39.51s/it] (WorkerDict pid=2146577) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2146199) step:41 - global_seqlen/min:22364 - global_seqlen/max:28937 - global_seqlen/minmax_diff:6573 - global_seqlen/balanced_min:25721 - global_seqlen/balanced_max:25724 - global_seqlen/mean:25723.125 - actor/entropy:0.050238970667123795 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9155643582344055 - training/rollout_probs_diff_mean:0.002400044584646821 - training/rollout_probs_diff_std:0.01305351685732603 - training/rollout_actor_probs_pearson_corr:0.9940615296363831 - rollout_corr/rollout_is_mean:0.9999544620513916 - rollout_corr/rollout_is_ratio_fraction_high:6.780777766834944e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00022279696713667363 - rollout_corr/rollout_is_max:4.58758020401001 - rollout_corr/rollout_is_min:0.009281577542424202 - rollout_corr/rollout_is_std:0.03381149470806122 - rollout_corr/rollout_is_eff_sample_size:0.998857850491772 - rollout_corr/rollout_is_seq_mean:0.9999521970748901 - rollout_corr/rollout_is_seq_std:0.0019201664254069328 - rollout_corr/rollout_is_seq_max:1.009830117225647 - rollout_corr/rollout_is_seq_min:0.9954932332038879 - rollout_corr/rollout_is_seq_max_deviation:0.009830117225646973 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0524404924362898) - rollout_corr/training_log_ppl:np.float64(0.05051660006483871) - rollout_corr/kl:np.float64(0.0009451009327534621) - rollout_corr/k3_kl:np.float64(0.0008613142946491958) - rollout_corr/rollout_ppl:np.float64(1.0514407688751817) - rollout_corr/rollout_log_ppl:np.float64(0.04957149836172903) - rollout_corr/log_ppl_diff:np.float64(0.0009451017031096853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017478646204835968) - rollout_corr/log_ppl_diff_max:np.float64(0.01430695503950119) - rollout_corr/log_ppl_diff_min:np.float64(-0.0073303282260894775) - rollout_corr/ppl_ratio:np.float64(1.000948389293626) - rollout_corr/chi2_token:np.float64(0.001788080669939518) - rollout_corr/chi2_seq:np.float64(1.042974109761417) - actor/pg_loss:np.float64(0.00019658554811030626) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000522654822361801) - actor/ppo_kl:np.float64(0.00020227026021046868) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.27989770099520683) - perf/mfu/actor:np.float64(0.07562782383756639) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(104.20849227905273) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0030852537602186203 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0030852537602186203 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:403.25390625 - response_length/max:844.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:403.25390625 - response_length_non_aborted/max:844.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:400.59375 - prompt_length/max:499.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.542432725429535e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.104304973036051) - timing_s/agent_loop/generate_sequences/max:np.float64(7.163554267957807) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.3063042574358406) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.163554267957807) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:844 - timing_s/gen:12.912682687863708 - timing_s/reward:0.059044428169727325 - timing_s/old_log_prob:2.4625996947288513 - timing_s/adv:0.12919568829238415 - timing_s/update_actor:9.838808562606573 - timing_s/step:25.507524743676186 - timing_s/stop_profile:0.00011672452092170715 - timing_per_token_ms/update_actor:0.047811106555903364 - timing_per_token_ms/adv:0.0006278187831590454 - timing_per_token_ms/gen:0.12508289682430723 - perf/total_num_tokens:205785 - perf/time_per_step:25.507524743676186 - perf/throughput:1008.4524178057404 (TaskRunner pid=2146199) Training Progress: 41%|████ | 41/100 [21:39<34:44, 35.33s/it] (TaskRunner pid=2146199) step:42 - global_seqlen/min:23580 - global_seqlen/max:30366 - global_seqlen/minmax_diff:6786 - global_seqlen/balanced_min:26212 - global_seqlen/balanced_max:26223 - global_seqlen/mean:26218.0 - actor/entropy:0.04833400622010231 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6611925363540649 - training/rollout_probs_diff_mean:0.002317308448255062 - training/rollout_probs_diff_std:0.013368863612413406 - training/rollout_actor_probs_pearson_corr:0.9936009645462036 - rollout_corr/rollout_is_mean:1.0000509023666382 - rollout_corr/rollout_is_ratio_fraction_high:7.637085946043953e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001909271377371624 - rollout_corr/rollout_is_max:2.7454490661621094 - rollout_corr/rollout_is_min:0.02648228593170643 - rollout_corr/rollout_is_std:0.03313124179840088 - rollout_corr/rollout_is_eff_sample_size:0.9989037623362461 - rollout_corr/rollout_is_seq_mean:1.000060796737671 - rollout_corr/rollout_is_seq_std:0.002085470361635089 - rollout_corr/rollout_is_seq_max:1.0072025060653687 - rollout_corr/rollout_is_seq_min:0.9903523325920105 - rollout_corr/rollout_is_seq_max_deviation:0.009647667407989502 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.051990379113704) - rollout_corr/training_log_ppl:np.float64(0.04970106136170216) - rollout_corr/kl:np.float64(0.0009528191514469597) - rollout_corr/k3_kl:np.float64(0.0010969023608851103) - rollout_corr/rollout_ppl:np.float64(1.0509791201911867) - rollout_corr/rollout_log_ppl:np.float64(0.04874824195121619) - rollout_corr/log_ppl_diff:np.float64(0.0009528194104859722) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021394124696598738) - rollout_corr/log_ppl_diff_max:np.float64(0.0255894735455513) - rollout_corr/log_ppl_diff_min:np.float64(-0.006002049893140793) - rollout_corr/ppl_ratio:np.float64(1.0009584189392626) - rollout_corr/chi2_token:np.float64(0.0025035273283720016) - rollout_corr/chi2_seq:np.float64(1.0566490055061877) - actor/pg_loss:np.float64(8.254428394138813e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003225053301321168) - actor/ppo_kl:np.float64(0.00030728954818926013) - actor/pg_clipfrac_lower:np.float64(1.014610370475566e-05) - actor/grad_norm:np.float64(0.2630367539823055) - perf/mfu/actor:np.float64(0.0767180936699128) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(104.11519622802734) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012183538638055325 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012183538638055325 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:409.1875 - response_length/max:865.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:409.1875 - response_length_non_aborted/max:865.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:410.125 - prompt_length/max:510.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010949186980724335 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7010023910552263) - timing_s/agent_loop/generate_sequences/max:np.float64(7.34863194078207) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.438730318630405) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.34863194078207) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:445 - timing_s/agent_loop/slowest/response_length:846 - timing_s/gen:13.201608955860138 - timing_s/reward:0.0633738711476326 - timing_s/old_log_prob:2.3262849245220423 - timing_s/adv:0.12737544439733028 - timing_s/update_actor:9.84446905553341 - timing_s/step:25.649843774735928 - timing_s/stop_profile:0.00011464953422546387 - timing_per_token_ms/update_actor:0.046935640855201625 - timing_per_token_ms/adv:0.0006072900507157787 - timing_per_token_ms/gen:0.12602727352088874 - perf/total_num_tokens:209744 - perf/time_per_step:25.649843774735928 - perf/throughput:1022.1504750771107 (TaskRunner pid=2146199) Training Progress: 42%|████▏ | 42/100 [22:05<31:21, 32.44s/it] (TaskRunner pid=2146199) step:43 - global_seqlen/min:20616 - global_seqlen/max:30340 - global_seqlen/minmax_diff:9724 - global_seqlen/balanced_min:24893 - global_seqlen/balanced_max:24897 - global_seqlen/mean:24896.0 - actor/entropy:0.04827848821878433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4558523893356323 - training/rollout_probs_diff_mean:0.0024351486936211586 - training/rollout_probs_diff_std:0.013620599173009396 - training/rollout_actor_probs_pearson_corr:0.9933694005012512 - rollout_corr/rollout_is_mean:1.0000344514846802 - rollout_corr/rollout_is_ratio_fraction_high:4.215140870655887e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00021075703261885792 - rollout_corr/rollout_is_max:3.047253370285034 - rollout_corr/rollout_is_min:0.15522444248199463 - rollout_corr/rollout_is_std:0.03526797145605087 - rollout_corr/rollout_is_eff_sample_size:0.9987578343765675 - rollout_corr/rollout_is_seq_mean:1.000058650970459 - rollout_corr/rollout_is_seq_std:0.0018858517287299037 - rollout_corr/rollout_is_seq_max:1.0111699104309082 - rollout_corr/rollout_is_seq_min:0.994408905506134 - rollout_corr/rollout_is_seq_max_deviation:0.011169910430908203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0514707742258906) - rollout_corr/training_log_ppl:np.float64(0.04937093240914692) - rollout_corr/kl:np.float64(0.0009204373019819911) - rollout_corr/k3_kl:np.float64(0.0008898747961474385) - rollout_corr/rollout_ppl:np.float64(1.0504933116026223) - rollout_corr/rollout_log_ppl:np.float64(0.04845049455843764) - rollout_corr/log_ppl_diff:np.float64(0.000920437850709277) - rollout_corr/log_ppl_abs_diff:np.float64(0.001846184547048324) - rollout_corr/log_ppl_diff_max:np.float64(0.010085105895996094) - rollout_corr/log_ppl_diff_min:np.float64(-0.00828292965888977) - rollout_corr/ppl_ratio:np.float64(1.0009237919002771) - rollout_corr/chi2_token:np.float64(0.0016981374938040972) - rollout_corr/chi2_seq:np.float64(0.4614659077487886) - actor/pg_loss:np.float64(0.00017189455684274435) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039900138654047623) - actor/ppo_kl:np.float64(0.00027663618505613385) - actor/pg_clipfrac_lower:np.float64(1.415307997376658e-05) - actor/grad_norm:np.float64(0.2620270363986492) - perf/mfu/actor:np.float64(0.07147921491217804) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.54963302612305) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015912156086415052 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0015912156086415052 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:370.6875 - response_length/max:925.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:370.6875 - response_length_non_aborted/max:925.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.3125 - prompt_length/max:494.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018392130732536316 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9507076852023602) - timing_s/agent_loop/generate_sequences/max:np.float64(7.141313912346959) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.058645684221119) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.141313912346959) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:925 - timing_s/gen:13.160315915942192 - timing_s/reward:0.06195548176765442 - timing_s/old_log_prob:2.2966495901346207 - timing_s/adv:0.12157766707241535 - timing_s/update_actor:9.996987046673894 - timing_s/step:25.72245409153402 - timing_s/stop_profile:0.00010517984628677368 - timing_per_token_ms/update_actor:0.050193741196747944 - timing_per_token_ms/adv:0.0006104277146550417 - timing_per_token_ms/gen:0.13868146092503575 - perf/total_num_tokens:199168 - perf/time_per_step:25.72245409153402 - perf/throughput:967.8703249467154 (TaskRunner pid=2146199) Training Progress: 43%|████▎ | 43/100 [22:31<28:55, 30.45s/it] (TaskRunner pid=2146199) step:44 - global_seqlen/min:22664 - global_seqlen/max:25495 - global_seqlen/minmax_diff:2831 - global_seqlen/balanced_min:24164 - global_seqlen/balanced_max:24168 - global_seqlen/mean:24166.0 - actor/entropy:0.04174882173538208 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9045923948287964 - training/rollout_probs_diff_mean:0.0021803160198032856 - training/rollout_probs_diff_std:0.013533946126699448 - training/rollout_actor_probs_pearson_corr:0.9925721287727356 - rollout_corr/rollout_is_mean:1.0000650882720947 - rollout_corr/rollout_is_ratio_fraction_high:0.00010393281991127878 - rollout_corr/rollout_is_ratio_fraction_low:0.00023904547560960054 - rollout_corr/rollout_is_max:4.69563102722168 - rollout_corr/rollout_is_min:0.0953548476099968 - rollout_corr/rollout_is_std:0.033987320959568024 - rollout_corr/rollout_is_eff_sample_size:0.9988461947885154 - rollout_corr/rollout_is_seq_mean:1.0000879764556885 - rollout_corr/rollout_is_seq_std:0.0023249476216733456 - rollout_corr/rollout_is_seq_max:1.0101128816604614 - rollout_corr/rollout_is_seq_min:0.9928390979766846 - rollout_corr/rollout_is_seq_max_deviation:0.010112881660461426 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0428532068617642) - rollout_corr/training_log_ppl:np.float64(0.041628760651292396) - rollout_corr/kl:np.float64(0.001001472390226965) - rollout_corr/k3_kl:np.float64(0.0008349532170441876) - rollout_corr/rollout_ppl:np.float64(1.041793908458203) - rollout_corr/rollout_log_ppl:np.float64(0.040627288033647346) - rollout_corr/log_ppl_diff:np.float64(0.0010014726176450495) - rollout_corr/log_ppl_abs_diff:np.float64(0.001965643570656539) - rollout_corr/log_ppl_diff_max:np.float64(0.012516435235738754) - rollout_corr/log_ppl_diff_min:np.float64(-0.005302894860506058) - rollout_corr/ppl_ratio:np.float64(1.001005484489724) - rollout_corr/chi2_token:np.float64(0.0013426775112748146) - rollout_corr/chi2_seq:np.float64(0.4853538442403078) - actor/pg_loss:np.float64(4.9682799726724625e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024175247563107405) - actor/ppo_kl:np.float64(0.0004345733158785947) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19688117131590843) - perf/mfu/actor:np.float64(0.07059824090627488) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.5389518737793) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002252743812277913 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002252743812277913 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:375.84375 - response_length/max:812.0 - response_length/min:178.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:375.84375 - response_length_non_aborted/max:812.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.34375 - prompt_length/max:524.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.038485586643219e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.11026918515563) - timing_s/agent_loop/generate_sequences/max:np.float64(6.88125741109252) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.06706091833621) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.88125741109252) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:411 - timing_s/agent_loop/slowest/response_length:795 - timing_s/gen:12.74365590326488 - timing_s/reward:0.06154830567538738 - timing_s/old_log_prob:2.2962511219084263 - timing_s/adv:0.12040187232196331 - timing_s/update_actor:9.759649621322751 - timing_s/step:25.068323547020555 - timing_s/stop_profile:0.00011921487748622894 - timing_per_token_ms/update_actor:0.05048233893343308 - timing_per_token_ms/adv:0.0006227854854028559 - timing_per_token_ms/gen:0.13244840674383554 - perf/total_num_tokens:193328 - perf/time_per_step:25.068323547020555 - perf/throughput:964.0054291892288 (TaskRunner pid=2146199) Training Progress: 44%|████▍ | 44/100 [22:56<26:55, 28.85s/it] (TaskRunner pid=2146199) step:45 - global_seqlen/min:18529 - global_seqlen/max:26606 - global_seqlen/minmax_diff:8077 - global_seqlen/balanced_min:23226 - global_seqlen/balanced_max:23233 - global_seqlen/mean:23231.25 - actor/entropy:0.05041806772351265 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.939910888671875 - training/rollout_probs_diff_mean:0.0024595295544713736 - training/rollout_probs_diff_std:0.014184881933033466 - training/rollout_actor_probs_pearson_corr:0.9932952523231506 - rollout_corr/rollout_is_mean:0.9998582005500793 - rollout_corr/rollout_is_ratio_fraction_high:0.00010988769645337015 - rollout_corr/rollout_is_ratio_fraction_low:0.0003516406286507845 - rollout_corr/rollout_is_max:3.250331401824951 - rollout_corr/rollout_is_min:0.05105861276388168 - rollout_corr/rollout_is_std:0.03565790504217148 - rollout_corr/rollout_is_eff_sample_size:0.9987298905286599 - rollout_corr/rollout_is_seq_mean:0.9998773336410522 - rollout_corr/rollout_is_seq_std:0.0022542960941791534 - rollout_corr/rollout_is_seq_max:1.0078169107437134 - rollout_corr/rollout_is_seq_min:0.9919712543487549 - rollout_corr/rollout_is_seq_max_deviation:0.008028745651245117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0486881639808416) - rollout_corr/training_log_ppl:np.float64(0.04712455965545814) - rollout_corr/kl:np.float64(0.000986228527544597) - rollout_corr/k3_kl:np.float64(0.0009043137568607307) - rollout_corr/rollout_ppl:np.float64(1.047645221464336) - rollout_corr/rollout_log_ppl:np.float64(0.046138330698340724) - rollout_corr/log_ppl_diff:np.float64(0.0009862289571174188) - rollout_corr/log_ppl_abs_diff:np.float64(0.001943429937455221) - rollout_corr/log_ppl_diff_max:np.float64(0.014261238276958466) - rollout_corr/log_ppl_diff_min:np.float64(-0.00514577329158783) - rollout_corr/ppl_ratio:np.float64(1.0009902347810566) - rollout_corr/chi2_token:np.float64(0.001611231593415141) - rollout_corr/chi2_seq:np.float64(0.7453246037475765) - actor/pg_loss:np.float64(0.00018049287609755993) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000512831802097935) - actor/ppo_kl:np.float64(9.556847214930997e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28934789076447487) - perf/mfu/actor:np.float64(0.06865060759745487) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.41498231887817) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:1 - critic/score/mean:0.78515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001116733648814261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001116733648814261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.4765625 - response_length/max:961.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.4765625 - response_length_non_aborted/max:961.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:370.5 - prompt_length/max:483.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001455899327993393 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8288240395486355) - timing_s/agent_loop/generate_sequences/max:np.float64(7.314498461782932) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.803905305598164) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.314498461782932) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:961 - timing_s/gen:13.205390265211463 - timing_s/reward:0.06098140962421894 - timing_s/old_log_prob:2.2992793917655945 - timing_s/adv:0.12455618008971214 - timing_s/update_actor:9.692075682803988 - timing_s/step:25.46947219967842 - timing_s/stop_profile:0.00011390075087547302 - timing_per_token_ms/update_actor:0.05214999022224368 - timing_per_token_ms/adv:0.0006701973639478727 - timing_per_token_ms/gen:0.14511098948607132 - perf/total_num_tokens:185850 - perf/time_per_step:25.46947219967842 - perf/throughput:912.1213748706311 (TaskRunner pid=2146199) Training Progress: 45%|████▌ | 45/100 [23:21<25:31, 27.85s/it] (TaskRunner pid=2146199) step:46 - global_seqlen/min:21427 - global_seqlen/max:26965 - global_seqlen/minmax_diff:5538 - global_seqlen/balanced_min:24185 - global_seqlen/balanced_max:24193 - global_seqlen/mean:24189.625 - actor/entropy:0.05843335762619972 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8514844179153442 - training/rollout_probs_diff_mean:0.002853688783943653 - training/rollout_probs_diff_std:0.015187487937510014 - training/rollout_actor_probs_pearson_corr:0.9931524395942688 - rollout_corr/rollout_is_mean:1.000073790550232 - rollout_corr/rollout_is_ratio_fraction_high:0.00010418728925287724 - rollout_corr/rollout_is_ratio_fraction_low:0.0002604682231321931 - rollout_corr/rollout_is_max:7.96063232421875 - rollout_corr/rollout_is_min:0.09759208559989929 - rollout_corr/rollout_is_std:0.038173478096723557 - rollout_corr/rollout_is_eff_sample_size:0.9985447871657489 - rollout_corr/rollout_is_seq_mean:1.0000745058059692 - rollout_corr/rollout_is_seq_std:0.0024244121741503477 - rollout_corr/rollout_is_seq_max:1.0171749591827393 - rollout_corr/rollout_is_seq_min:0.9928064346313477 - rollout_corr/rollout_is_seq_max_deviation:0.017174959182739258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0607345919124782) - rollout_corr/training_log_ppl:np.float64(0.0581217721082794) - rollout_corr/kl:np.float64(0.0014772235255975374) - rollout_corr/k3_kl:np.float64(0.00150407016360532) - rollout_corr/rollout_ppl:np.float64(1.0591790787875652) - rollout_corr/rollout_log_ppl:np.float64(0.056644548536496586) - rollout_corr/log_ppl_diff:np.float64(0.0014772235717828153) - rollout_corr/log_ppl_abs_diff:np.float64(0.002647029568834114) - rollout_corr/log_ppl_diff_max:np.float64(0.0180678628385067) - rollout_corr/log_ppl_diff_min:np.float64(-0.013853691518306732) - rollout_corr/ppl_ratio:np.float64(1.0014852569438517) - rollout_corr/chi2_token:np.float64(0.008371114498004317) - rollout_corr/chi2_seq:np.float64(6.4867037024814636) - actor/pg_loss:np.float64(7.591512985527515e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006503565891762264) - actor/ppo_kl:np.float64(0.0006854172221775201) - actor/pg_clipfrac_lower:np.float64(2.030308314715512e-05) - actor/grad_norm:np.float64(0.3519108295440674) - perf/mfu/actor:np.float64(0.07113595200912443) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.54940795898438) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:1 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016057866159826517 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016057866159826517 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:374.92578125 - response_length/max:801.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:374.92578125 - response_length_non_aborted/max:801.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:381.0 - prompt_length/max:481.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.761936962604523e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6900712680071592) - timing_s/agent_loop/generate_sequences/max:np.float64(6.451910957694054) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.033618862835283) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.451910957694054) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:271 - timing_s/agent_loop/slowest/response_length:801 - timing_s/gen:12.604724714532495 - timing_s/reward:0.06205034628510475 - timing_s/old_log_prob:2.278717679902911 - timing_s/adv:0.1322988048195839 - timing_s/update_actor:9.617559352889657 - timing_s/step:24.78110233694315 - timing_s/stop_profile:0.000115908682346344 - timing_per_token_ms/update_actor:0.04969878280920879 - timing_per_token_ms/adv:0.0006836546909035583 - timing_per_token_ms/gen:0.13132520722364316 - perf/total_num_tokens:193517 - perf/time_per_step:24.78110233694315 - perf/throughput:976.1319198435581 (TaskRunner pid=2146199) Training Progress: 46%|████▌ | 46/100 [23:46<24:14, 26.94s/it] (TaskRunner pid=2146199) step:47 - global_seqlen/min:21007 - global_seqlen/max:28791 - global_seqlen/minmax_diff:7784 - global_seqlen/balanced_min:25475 - global_seqlen/balanced_max:25481 - global_seqlen/mean:25477.0 - actor/entropy:0.05275755748152733 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083588361740112 - training/rollout_probs_diff_mean:0.0025903242640197277 - training/rollout_probs_diff_std:0.013454040512442589 - training/rollout_actor_probs_pearson_corr:0.9939318895339966 - rollout_corr/rollout_is_mean:0.9998558163642883 - rollout_corr/rollout_is_ratio_fraction_high:0.00011591066868277267 - rollout_corr/rollout_is_ratio_fraction_low:0.0002607990172691643 - rollout_corr/rollout_is_max:2.9357402324676514 - rollout_corr/rollout_is_min:0.1746438890695572 - rollout_corr/rollout_is_std:0.035481955856084824 - rollout_corr/rollout_is_eff_sample_size:0.9987423758844771 - rollout_corr/rollout_is_seq_mean:0.9998483061790466 - rollout_corr/rollout_is_seq_std:0.0020597924012690783 - rollout_corr/rollout_is_seq_max:1.0084785223007202 - rollout_corr/rollout_is_seq_min:0.9923441410064697 - rollout_corr/rollout_is_seq_max_deviation:0.008478522300720215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.05636456143111) - rollout_corr/training_log_ppl:np.float64(0.05421838367328746) - rollout_corr/kl:np.float64(0.0010813371544315942) - rollout_corr/k3_kl:np.float64(0.0010256878935948066) - rollout_corr/rollout_ppl:np.float64(1.0552101419307292) - rollout_corr/rollout_log_ppl:np.float64(0.05313704631043947) - rollout_corr/log_ppl_diff:np.float64(0.0010813373628479894) - rollout_corr/log_ppl_abs_diff:np.float64(0.001953723574843025) - rollout_corr/log_ppl_diff_max:np.float64(0.01056503877043724) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066755786538124084) - rollout_corr/ppl_ratio:np.float64(1.001085146330297) - rollout_corr/chi2_token:np.float64(0.0027294796891510487) - rollout_corr/chi2_seq:np.float64(0.8529609069228172) - actor/pg_loss:np.float64(9.709945879876614e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022560217075806577) - actor/ppo_kl:np.float64(0.0002011541448965204) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18023865018039942) - perf/mfu/actor:np.float64(0.07436321123352709) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.5498046875) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002857922576367855 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002857922576367855 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:404.40625 - response_length/max:814.0 - response_length/min:198.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.40625 - response_length_non_aborted/max:814.0 - response_length_non_aborted/min:198.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.75 - prompt_length/max:490.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014630146324634552 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3290767408907413) - timing_s/agent_loop/generate_sequences/max:np.float64(7.020034801214933) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.509792248674785) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.020034801214933) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:414 - timing_s/agent_loop/slowest/response_length:814 - timing_s/gen:12.896446391940117 - timing_s/reward:0.0626764353364706 - timing_s/old_log_prob:2.3000318836420774 - timing_s/adv:0.16442351788282394 - timing_s/update_actor:9.82856030575931 - timing_s/step:25.338740235194564 - timing_s/stop_profile:0.00012014247477054596 - timing_per_token_ms/update_actor:0.048222712180394624 - timing_per_token_ms/adv:0.0008067252712388818 - timing_per_token_ms/gen:0.1245696467809686 - perf/total_num_tokens:203816 - perf/time_per_step:25.338740235194564 - perf/throughput:1005.4564577213432 (TaskRunner pid=2146199) Training Progress: 47%|████▋ | 47/100 [24:12<23:23, 26.48s/it] (TaskRunner pid=2146199) step:48 - global_seqlen/min:22032 - global_seqlen/max:29064 - global_seqlen/minmax_diff:7032 - global_seqlen/balanced_min:25498 - global_seqlen/balanced_max:25500 - global_seqlen/mean:25499.0 - actor/entropy:0.04434153437614441 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.992967963218689 - training/rollout_probs_diff_mean:0.0023447973653674126 - training/rollout_probs_diff_std:0.014210212975740433 - training/rollout_actor_probs_pearson_corr:0.9921810030937195 - rollout_corr/rollout_is_mean:1.0000815391540527 - rollout_corr/rollout_is_ratio_fraction_high:8.659174636704847e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00028863915940746665 - rollout_corr/rollout_is_max:3.37223482131958 - rollout_corr/rollout_is_min:0.0066951303742825985 - rollout_corr/rollout_is_std:0.035338882356882095 - rollout_corr/rollout_is_eff_sample_size:0.9987528400421808 - rollout_corr/rollout_is_seq_mean:1.0001084804534912 - rollout_corr/rollout_is_seq_std:0.0019933267030864954 - rollout_corr/rollout_is_seq_max:1.0075260400772095 - rollout_corr/rollout_is_seq_min:0.9929347038269043 - rollout_corr/rollout_is_seq_max_deviation:0.007526040077209473 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0459272447042167) - rollout_corr/training_log_ppl:np.float64(0.04459278326794447) - rollout_corr/kl:np.float64(0.0008026134670942042) - rollout_corr/k3_kl:np.float64(0.0010650326459575865) - rollout_corr/rollout_ppl:np.float64(1.0450774431228638) - rollout_corr/rollout_log_ppl:np.float64(0.04379016965140181) - rollout_corr/log_ppl_diff:np.float64(0.0008026136165426578) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019220001304347534) - rollout_corr/log_ppl_diff_max:np.float64(0.03549540787935257) - rollout_corr/log_ppl_diff_min:np.float64(-0.006428159773349762) - rollout_corr/ppl_ratio:np.float64(1.000808133278042) - rollout_corr/chi2_token:np.float64(0.002460258547216654) - rollout_corr/chi2_seq:np.float64(0.9476184649392962) - actor/pg_loss:np.float64(0.0001292722299695015) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028929511245223694) - actor/ppo_kl:np.float64(0.00011834800343724083) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29330509901046753) - perf/mfu/actor:np.float64(0.07469785903149992) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.59997940063477) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:1 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029934286139905453 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029934286139905453 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.0 - response_length/max:772.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.0 - response_length_non_aborted/max:772.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.84375 - prompt_length/max:488.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.766485214233398e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.962918195873499) - timing_s/agent_loop/generate_sequences/max:np.float64(6.8410259541124105) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.380887989165785) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.8410259541124105) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:772 - timing_s/gen:12.71031053736806 - timing_s/reward:0.06214878149330616 - timing_s/old_log_prob:2.3244321551173925 - timing_s/adv:0.12186715565621853 - timing_s/update_actor:9.725567547604442 - timing_s/step:25.031608840450644 - timing_s/stop_profile:0.00011808797717094421 - timing_per_token_ms/update_actor:0.04767622037925233 - timing_per_token_ms/adv:0.000597411445822476 - timing_per_token_ms/gen:0.12228977964678321 - perf/total_num_tokens:203992 - perf/time_per_step:25.031608840450644 - perf/throughput:1018.6720383227649 (TaskRunner pid=2146199) Training Progress: 48%|████▊ | 48/100 [24:37<22:35, 26.06s/it] (TaskRunner pid=2146199) step:49 - global_seqlen/min:22267 - global_seqlen/max:29506 - global_seqlen/minmax_diff:7239 - global_seqlen/balanced_min:25325 - global_seqlen/balanced_max:25336 - global_seqlen/mean:25333.0 - actor/entropy:0.04938375577330589 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4130066931247711 - training/rollout_probs_diff_mean:0.0025434454437345266 - training/rollout_probs_diff_std:0.01367312390357256 - training/rollout_actor_probs_pearson_corr:0.9934003949165344 - rollout_corr/rollout_is_mean:1.0000077486038208 - rollout_corr/rollout_is_ratio_fraction_high:6.867053889436647e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002452519256621599 - rollout_corr/rollout_is_max:7.9477386474609375 - rollout_corr/rollout_is_min:0.18729040026664734 - rollout_corr/rollout_is_std:0.03583298996090889 - rollout_corr/rollout_is_eff_sample_size:0.9987174054849994 - rollout_corr/rollout_is_seq_mean:1.000023603439331 - rollout_corr/rollout_is_seq_std:0.002729950938373804 - rollout_corr/rollout_is_seq_max:1.0252361297607422 - rollout_corr/rollout_is_seq_min:0.9913330674171448 - rollout_corr/rollout_is_seq_max_deviation:0.025236129760742188 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0507496995851398) - rollout_corr/training_log_ppl:np.float64(0.04913640160611976) - rollout_corr/kl:np.float64(0.0014581667759543393) - rollout_corr/k3_kl:np.float64(0.0011762470113154677) - rollout_corr/rollout_ppl:np.float64(1.0492076580412686) - rollout_corr/rollout_log_ppl:np.float64(0.047678234113845974) - rollout_corr/log_ppl_diff:np.float64(0.0014581674922737875) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023839171653889935) - rollout_corr/log_ppl_diff_max:np.float64(0.019139260053634644) - rollout_corr/log_ppl_diff_min:np.float64(-0.007026508450508118) - rollout_corr/ppl_ratio:np.float64(1.0014637822750956) - rollout_corr/chi2_token:np.float64(0.00223412923514843) - rollout_corr/chi2_seq:np.float64(1.0949563854373991) - actor/pg_loss:np.float64(0.00034242914989590645) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006906314793013735) - actor/ppo_kl:np.float64(0.0007121005893691024) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2707485258579254) - perf/mfu/actor:np.float64(0.07335820600659577) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.67672348022461) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007933899760246277 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0007933899760246277 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:398.1875 - response_length/max:1164.0 - response_length/min:196.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:398.1875 - response_length_non_aborted/max:1164.0 - response_length_non_aborted/min:196.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:393.46875 - prompt_length/max:486.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.410215377807617e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.303459979593754) - timing_s/agent_loop/generate_sequences/max:np.float64(8.49587570503354) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.2898935974008054) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.49587570503354) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1164 - timing_s/gen:14.148086585104465 - timing_s/reward:0.06304617412388325 - timing_s/old_log_prob:2.311548363417387 - timing_s/adv:0.1533728539943695 - timing_s/update_actor:9.837313449010253 - timing_s/step:26.602564496919513 - timing_s/stop_profile:0.00019768811762332916 - timing_per_token_ms/update_actor:0.04854001425517237 - timing_per_token_ms/adv:0.0007567839083131168 - timing_per_token_ms/gen:0.13879381754340434 - perf/total_num_tokens:202664 - perf/time_per_step:26.602564496919513 - perf/throughput:952.2766123895113 (TaskRunner pid=2146199) Training Progress: 49%|████▉ | 49/100 [25:03<22:18, 26.24s/it] (TaskRunner pid=2146199) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2146199) validation generation end (TaskRunner pid=2146199) [prompt] system (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) The given crystal structure has a rectangular (orthorhombic) unit cell because all the angles α, β, and γ are 90.00º. In an orthorhombic system, the volume of the unit cell is calculated as: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = a \times b \times c $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given: (TaskRunner pid=2146199) - $ a = 7.82 $ Å (TaskRunner pid=2146199) - $ b = 7.82 $ Å (TaskRunner pid=2146199) - $ c = 17.88 $ Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Substituting the values: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Then multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Thus, the volume of the unit cell is 1094.31 ų. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (TaskRunner pid=2146199) step:50 - global_seqlen/min:19545 - global_seqlen/max:29595 - global_seqlen/minmax_diff:10050 - global_seqlen/balanced_min:25379 - global_seqlen/balanced_max:25384 - global_seqlen/mean:25381.25 - actor/entropy:0.04783216118812561 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7039056420326233 - training/rollout_probs_diff_mean:0.00252446997910738 - training/rollout_probs_diff_std:0.014383954927325249 - training/rollout_actor_probs_pearson_corr:0.9927150011062622 - rollout_corr/rollout_is_mean:0.999859094619751 - rollout_corr/rollout_is_ratio_fraction_high:4.985144187230617e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002592274977359921 - rollout_corr/rollout_is_max:5.754598617553711 - rollout_corr/rollout_is_min:0.07292912155389786 - rollout_corr/rollout_is_std:0.03485316038131714 - rollout_corr/rollout_is_eff_sample_size:0.9987866122287246 - rollout_corr/rollout_is_seq_mean:0.9999264478683472 - rollout_corr/rollout_is_seq_std:0.0026216376572847366 - rollout_corr/rollout_is_seq_max:1.0235873460769653 - rollout_corr/rollout_is_seq_min:0.9895705580711365 - rollout_corr/rollout_is_seq_max_deviation:0.023587346076965332 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0526855518110096) - rollout_corr/training_log_ppl:np.float64(0.05071893223430379) - rollout_corr/kl:np.float64(0.001377381433474767) - rollout_corr/k3_kl:np.float64(0.001280714352795087) - rollout_corr/rollout_ppl:np.float64(1.0512280194088817) - rollout_corr/rollout_log_ppl:np.float64(0.049341550715325866) - rollout_corr/log_ppl_diff:np.float64(0.001377381518977927) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023279554916371126) - rollout_corr/log_ppl_diff_max:np.float64(0.022277753800153732) - rollout_corr/log_ppl_diff_min:np.float64(-0.006641877815127373) - rollout_corr/ppl_ratio:np.float64(1.001384488772601) - rollout_corr/chi2_token:np.float64(0.0023646841291338205) - rollout_corr/chi2_seq:np.float64(0.9455087245441973) - actor/pg_loss:np.float64(0.00024931400548666716) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032460508600706817) - actor/ppo_kl:np.float64(0.0004278824517598423) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2263021469116211) - perf/mfu/actor:np.float64(0.07391291988771881) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.61050033569336) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.754654255319149) - val-aux/sciknoweval/reward/std@16:np.float64(0.043705150970647255) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7703723404255318) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.036030076779019676) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7382021276595745) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.03575971681867158) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7542765957446809) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.04423605326107866) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7842978723404255) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.03044081024797837) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7249468085106383) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.03036873282472957) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.755031914893617) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.03073653652091519) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7963829787234042) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.024887646288860848) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7126595744680853) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.02476570728187557) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7547340425531914) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.017075358189754013) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8071063829787234) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.018555689505034636) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.7013085106382979) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.017951256269916015) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7539893617021277) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.011314030888742872) - val-aux/sciknoweval/score/mean@16:np.float64(0.754654255319149) - val-aux/sciknoweval/score/std@16:np.float64(0.043705150970647255) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7703723404255318) - val-aux/sciknoweval/score/best@2/std:np.float64(0.036030076779019676) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7382021276595745) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.03575971681867158) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7542765957446809) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.04423605326107866) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7842978723404255) - val-aux/sciknoweval/score/best@4/std:np.float64(0.03044081024797837) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7249468085106383) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.03036873282472957) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.755031914893617) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.03073653652091519) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7963829787234042) - val-aux/sciknoweval/score/best@8/std:np.float64(0.024887646288860848) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7126595744680853) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.02476570728187557) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7547340425531914) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.017075358189754013) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8071063829787234) - val-aux/sciknoweval/score/best@16/std:np.float64(0.018555689505034636) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.7013085106382979) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.017951256269916015) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7539893617021277) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.011314030888742872) - val-core/sciknoweval/acc/mean@16:np.float64(0.754654255319149) - val-aux/sciknoweval/acc/std@16:np.float64(0.043705150970647255) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7703723404255318) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.036030076779019676) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7382021276595745) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.03575971681867158) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7542765957446809) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.04423605326107866) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7842978723404255) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.03044081024797837) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7249468085106383) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.03036873282472957) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.755031914893617) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.03073653652091519) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7963829787234042) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.024887646288860848) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7126595744680853) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.02476570728187557) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7547340425531914) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.017075358189754013) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8071063829787234) - val-core/sciknoweval/acc/best@16/std:np.float64(0.018555689505034636) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.7013085106382979) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.017951256269916015) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7539893617021277) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.011314030888742872) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020650960505008698 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020650960505008698 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:391.7890625 - response_length/max:811.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:391.7890625 - response_length_non_aborted/max:811.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.375 - prompt_length/max:492.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.69855284690857e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.132479701191187) - timing_s/agent_loop/generate_sequences/max:np.float64(6.845823027193546) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.2515298968646675) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.845823027193546) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:811 - timing_s/gen:12.828976206481457 - timing_s/reward:0.0607075821608305 - timing_s/old_log_prob:2.345235312357545 - timing_s/adv:0.11771114729344845 - timing_s/update_actor:9.792069472372532 - timing_s/step:25.233451556414366 - timing_s/testing:42.79238680936396 - timing_s/save_checkpoint:6.64641591720283 - timing_s/stop_profile:0.00011313706636428833 - timing_per_token_ms/update_actor:0.04822491737194057 - timing_per_token_ms/adv:0.0005797150814747523 - timing_per_token_ms/gen:0.1279085944533436 - perf/total_num_tokens:203050 - perf/time_per_step:25.233451556414366 - perf/throughput:1005.8572424487868 (TaskRunner pid=2146199) Training Progress: 50%|█████ | 50/100 [26:18<33:59, 40.79s/it] (WorkerDict pid=2146577) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2146199) step:51 - global_seqlen/min:22948 - global_seqlen/max:30587 - global_seqlen/minmax_diff:7639 - global_seqlen/balanced_min:25824 - global_seqlen/balanced_max:25826 - global_seqlen/mean:25825.0 - actor/entropy:0.047966454178094864 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6983711123466492 - training/rollout_probs_diff_mean:0.0022839412558823824 - training/rollout_probs_diff_std:0.012598144821822643 - training/rollout_actor_probs_pearson_corr:0.9943193793296814 - rollout_corr/rollout_is_mean:1.0000054836273193 - rollout_corr/rollout_is_ratio_fraction_high:6.572029087692499e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016899504407774657 - rollout_corr/rollout_is_max:3.8465442657470703 - rollout_corr/rollout_is_min:0.13807864487171173 - rollout_corr/rollout_is_std:0.03345528244972229 - rollout_corr/rollout_is_eff_sample_size:0.9988821142567822 - rollout_corr/rollout_is_seq_mean:1.0000351667404175 - rollout_corr/rollout_is_seq_std:0.001773972762748599 - rollout_corr/rollout_is_seq_max:1.007016658782959 - rollout_corr/rollout_is_seq_min:0.9964668154716492 - rollout_corr/rollout_is_seq_max_deviation:0.007016658782958984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.053481592796743) - rollout_corr/training_log_ppl:np.float64(0.05124455909572134) - rollout_corr/kl:np.float64(0.0009008934078220676) - rollout_corr/k3_kl:np.float64(0.0008094352686356387) - rollout_corr/rollout_ppl:np.float64(1.0525373336859047) - rollout_corr/rollout_log_ppl:np.float64(0.05034366530526313) - rollout_corr/log_ppl_diff:np.float64(0.0009008937904582126) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016325298729498172) - rollout_corr/log_ppl_diff_max:np.float64(0.00915449857711792) - rollout_corr/log_ppl_diff_min:np.float64(-0.00464421883225441) - rollout_corr/ppl_ratio:np.float64(1.0009034115355462) - rollout_corr/chi2_token:np.float64(0.0013905810192227364) - rollout_corr/chi2_seq:np.float64(0.24303906597197056) - actor/pg_loss:np.float64(-5.012843757867813e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003733771582119516) - actor/ppo_kl:np.float64(0.0002866417318674319) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21566109731793404) - perf/mfu/actor:np.float64(0.07476824194362565) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(104.36287689208984) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:1 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038728031795471907 - critic/advantages/max:0.5 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0038728031795471907 - critic/returns/max:0.5 - critic/returns/min:-0.75 - response_length/mean:416.0625 - response_length/max:723.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:416.0625 - response_length_non_aborted/max:723.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.96875 - prompt_length/max:519.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001530572772026062 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9842853043228388) - timing_s/agent_loop/generate_sequences/max:np.float64(6.769318075850606) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.696395583436242) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.769318075850606) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:420 - timing_s/agent_loop/slowest/response_length:672 - timing_s/gen:12.555826744064689 - timing_s/reward:0.05338887870311737 - timing_s/old_log_prob:2.4519566763192415 - timing_s/adv:0.11838049255311489 - timing_s/update_actor:9.93727552331984 - timing_s/step:25.152532739564776 - timing_s/stop_profile:5.325488746166229e-05 - timing_per_token_ms/update_actor:0.04809910708286467 - timing_per_token_ms/adv:0.0005729936716026858 - timing_per_token_ms/gen:0.11788180434190221 - perf/total_num_tokens:206600 - perf/time_per_step:25.152532739564776 - perf/throughput:1026.7355684375052 (TaskRunner pid=2146199) Training Progress: 51%|█████ | 51/100 [26:43<29:29, 36.11s/it] (TaskRunner pid=2146199) step:52 - global_seqlen/min:24063 - global_seqlen/max:30643 - global_seqlen/minmax_diff:6580 - global_seqlen/balanced_min:27381 - global_seqlen/balanced_max:27384 - global_seqlen/mean:27383.0 - actor/entropy:0.04148118942975998 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3794129490852356 - training/rollout_probs_diff_mean:0.0021210412960499525 - training/rollout_probs_diff_std:0.012127730995416641 - training/rollout_actor_probs_pearson_corr:0.9938873648643494 - rollout_corr/rollout_is_mean:1.000080943107605 - rollout_corr/rollout_is_ratio_fraction_high:0.00012082298053428531 - rollout_corr/rollout_is_ratio_fraction_low:0.00021575530990958214 - rollout_corr/rollout_is_max:4.886763572692871 - rollout_corr/rollout_is_min:0.2553877532482147 - rollout_corr/rollout_is_std:0.03285301849246025 - rollout_corr/rollout_is_eff_sample_size:0.9989219617224367 - rollout_corr/rollout_is_seq_mean:1.000226616859436 - rollout_corr/rollout_is_seq_std:0.0021179071627557278 - rollout_corr/rollout_is_seq_max:1.0087249279022217 - rollout_corr/rollout_is_seq_min:0.9942286610603333 - rollout_corr/rollout_is_seq_max_deviation:0.00872492790222168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0469471900723875) - rollout_corr/training_log_ppl:np.float64(0.04545338210209593) - rollout_corr/kl:np.float64(0.0006419764257740157) - rollout_corr/k3_kl:np.float64(0.0007060423155191842) - rollout_corr/rollout_ppl:np.float64(1.0462591429241002) - rollout_corr/rollout_log_ppl:np.float64(0.04481140532607242) - rollout_corr/log_ppl_diff:np.float64(0.0006419767760235118) - rollout_corr/log_ppl_abs_diff:np.float64(0.001562883453516406) - rollout_corr/log_ppl_diff_max:np.float64(0.011684022843837738) - rollout_corr/log_ppl_diff_min:np.float64(-0.005429472774267197) - rollout_corr/ppl_ratio:np.float64(1.000644227489829) - rollout_corr/chi2_token:np.float64(0.0016550992149859667) - rollout_corr/chi2_seq:np.float64(0.5347908311523497) - actor/pg_loss:np.float64(-4.171160981059074e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023765303967593354) - actor/ppo_kl:np.float64(0.00018683375900252486) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12192103452980518) - perf/mfu/actor:np.float64(0.07844641483725558) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.9221305847168) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:1 - critic/score/mean:0.76953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0023128970060497522 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0023128970060497522 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:452.625 - response_length/max:822.0 - response_length/min:177.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:452.625 - response_length_non_aborted/max:822.0 - response_length_non_aborted/min:177.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:403.09375 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.5616721510887146e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1250411551445723) - timing_s/agent_loop/generate_sequences/max:np.float64(7.574766311794519) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.067384771245997) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.574766311794519) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:406 - timing_s/agent_loop/slowest/response_length:822 - timing_s/gen:13.288536546751857 - timing_s/reward:0.06399823538959026 - timing_s/old_log_prob:2.357649089768529 - timing_s/adv:0.11962497048079967 - timing_s/update_actor:10.02290122769773 - timing_s/step:25.938690785318613 - timing_s/stop_profile:2.9431656002998352e-05 - timing_per_token_ms/update_actor:0.045753301444772895 - timing_per_token_ms/adv:0.0005460731588978548 - timing_per_token_ms/gen:0.11468289618503053 - perf/total_num_tokens:219064 - perf/time_per_step:25.938690785318613 - perf/throughput:1055.6816543531515 (TaskRunner pid=2146199) Training Progress: 52%|█████▏ | 52/100 [27:09<26:26, 33.06s/it] (TaskRunner pid=2146199) step:53 - global_seqlen/min:22194 - global_seqlen/max:29378 - global_seqlen/minmax_diff:7184 - global_seqlen/balanced_min:25722 - global_seqlen/balanced_max:25723 - global_seqlen/mean:25722.125 - actor/entropy:0.045347992330789566 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9706128835678101 - training/rollout_probs_diff_mean:0.002372557995840907 - training/rollout_probs_diff_std:0.014113826677203178 - training/rollout_actor_probs_pearson_corr:0.9923505187034607 - rollout_corr/rollout_is_mean:0.9998732805252075 - rollout_corr/rollout_is_ratio_fraction_high:2.8757393010891974e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00024923073942773044 - rollout_corr/rollout_is_max:2.7349207401275635 - rollout_corr/rollout_is_min:7.710630598012358e-05 - rollout_corr/rollout_is_std:0.0333106592297554 - rollout_corr/rollout_is_eff_sample_size:0.9988915108353655 - rollout_corr/rollout_is_seq_mean:0.9999209046363831 - rollout_corr/rollout_is_seq_std:0.0020313896238803864 - rollout_corr/rollout_is_seq_max:1.0061156749725342 - rollout_corr/rollout_is_seq_min:0.9936782121658325 - rollout_corr/rollout_is_seq_max_deviation:0.0063217878341674805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.051966101396829) - rollout_corr/training_log_ppl:np.float64(0.05026089035709447) - rollout_corr/kl:np.float64(0.0011716701351689451) - rollout_corr/k3_kl:np.float64(0.0010751419230672354) - rollout_corr/rollout_ppl:np.float64(1.0507349884137511) - rollout_corr/rollout_log_ppl:np.float64(0.049089219741290435) - rollout_corr/log_ppl_diff:np.float64(0.0011716706176230218) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020845381368417293) - rollout_corr/log_ppl_diff_max:np.float64(0.03334299847483635) - rollout_corr/log_ppl_diff_min:np.float64(-0.004928275942802429) - rollout_corr/ppl_ratio:np.float64(1.0011795153841376) - rollout_corr/chi2_token:np.float64(0.0012073535472154617) - rollout_corr/chi2_seq:np.float64(0.5232194799464196) - actor/pg_loss:np.float64(-6.142677739262581e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002523157354517025) - actor/ppo_kl:np.float64(0.0002233287712201104) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16869864612817764) - perf/mfu/actor:np.float64(0.07444512581583518) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.71920013427734) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002926064655184746 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002926064655184746 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:407.50390625 - response_length/max:817.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:407.50390625 - response_length_non_aborted/max:817.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:502.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0005197636783123016 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0927931386977434) - timing_s/agent_loop/generate_sequences/max:np.float64(7.136662200093269) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.428619973019522) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.136662200093269) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:492 - timing_s/agent_loop/slowest/response_length:817 - timing_s/gen:13.178558483719826 - timing_s/reward:0.056101761758327484 - timing_s/old_log_prob:2.5733994133770466 - timing_s/adv:0.1274885218590498 - timing_s/update_actor:9.954374069347978 - timing_s/step:25.999704658985138 - timing_s/stop_profile:0.00011053681373596191 - timing_per_token_ms/update_actor:0.04837457086723967 - timing_per_token_ms/adv:0.0006195469943630717 - timing_per_token_ms/gen:0.12632699536737402 - perf/total_num_tokens:205777 - perf/time_per_step:25.999704658985138 - perf/throughput:989.3237379952618 (TaskRunner pid=2146199) Training Progress: 53%|█████▎ | 53/100 [27:36<24:21, 31.09s/it] (TaskRunner pid=2146199) step:54 - global_seqlen/min:22225 - global_seqlen/max:32555 - global_seqlen/minmax_diff:10330 - global_seqlen/balanced_min:28021 - global_seqlen/balanced_max:28435 - global_seqlen/mean:28074.25 - actor/entropy:0.050688885152339935 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948752164840698 - training/rollout_probs_diff_mean:0.0021564157214015722 - training/rollout_probs_diff_std:0.012509707361459732 - training/rollout_actor_probs_pearson_corr:0.9947131276130676 - rollout_corr/rollout_is_mean:1.0001189708709717 - rollout_corr/rollout_is_ratio_fraction_high:0.00010016527085099369 - rollout_corr/rollout_is_ratio_fraction_low:0.0001669421180849895 - rollout_corr/rollout_is_max:4.710118770599365 - rollout_corr/rollout_is_min:0.012120777741074562 - rollout_corr/rollout_is_std:0.03328559920191765 - rollout_corr/rollout_is_eff_sample_size:0.9988934139614173 - rollout_corr/rollout_is_seq_mean:1.0001349449157715 - rollout_corr/rollout_is_seq_std:0.002031756564974785 - rollout_corr/rollout_is_seq_max:1.0122159719467163 - rollout_corr/rollout_is_seq_min:0.9934861063957214 - rollout_corr/rollout_is_seq_max_deviation:0.012215971946716309 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.054738669656217) - rollout_corr/training_log_ppl:np.float64(0.05224958515645994) - rollout_corr/kl:np.float64(0.0006914916284692652) - rollout_corr/k3_kl:np.float64(0.0008615583958970774) - rollout_corr/rollout_ppl:np.float64(1.053984981495887) - rollout_corr/rollout_log_ppl:np.float64(0.051558093260609894) - rollout_corr/log_ppl_diff:np.float64(0.0006914918958500493) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016903485884540714) - rollout_corr/log_ppl_diff_max:np.float64(0.02006283402442932) - rollout_corr/log_ppl_diff_min:np.float64(-0.007497852668166161) - rollout_corr/ppl_ratio:np.float64(1.0006947827059776) - rollout_corr/chi2_token:np.float64(0.0022294067312031984) - rollout_corr/chi2_seq:np.float64(1.1644210352096707) - actor/pg_loss:np.float64(9.966641664505005e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029754303795925807) - actor/ppo_kl:np.float64(8.75349244502388e-05) - actor/pg_clipfrac_lower:np.float64(4.6837531044729985e-06) - actor/grad_norm:np.float64(0.22745876014232635) - perf/mfu/actor:np.float64(0.08115548760789604) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.72404098510742) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:6.155990558909252e-05 - critic/advantages/max:0.625 - critic/advantages/min:-0.5 - critic/returns/mean:6.155990558909252e-05 - critic/returns/max:0.625 - critic/returns/min:-0.5 - response_length/mean:467.9765625 - response_length/max:1653.0 - response_length/min:209.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:467.9765625 - response_length_non_aborted/max:1653.0 - response_length_non_aborted/min:209.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:409.34375 - prompt_length/max:524.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.813237607479095e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.5668025631457567) - timing_s/agent_loop/generate_sequences/max:np.float64(11.435457868501544) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.112390608905116) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.435457868501544) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:1653 - timing_s/gen:17.307704316452146 - timing_s/reward:0.06812970340251923 - timing_s/old_log_prob:2.3803684432059526 - timing_s/adv:0.11982751451432705 - timing_s/update_actor:9.849199259653687 - timing_s/step:29.81224417500198 - timing_s/stop_profile:0.00010670535266399384 - timing_per_token_ms/update_actor:0.04385334986532894 - timing_per_token_ms/adv:0.0005335294554365969 - timing_per_token_ms/gen:0.14446924355563467 - perf/total_num_tokens:224594 - perf/time_per_step:29.81224417500198 - perf/throughput:941.7020012046154 (TaskRunner pid=2146199) Training Progress: 54%|█████▍ | 54/100 [28:06<23:33, 30.72s/it] (TaskRunner pid=2146199) step:55 - global_seqlen/min:22824 - global_seqlen/max:31101 - global_seqlen/minmax_diff:8277 - global_seqlen/balanced_min:27210 - global_seqlen/balanced_max:27215 - global_seqlen/mean:27212.125 - actor/entropy:0.03937111794948578 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7327499389648438 - training/rollout_probs_diff_mean:0.001998358638957143 - training/rollout_probs_diff_std:0.012327085249125957 - training/rollout_actor_probs_pearson_corr:0.9933736324310303 - rollout_corr/rollout_is_mean:0.9999356269836426 - rollout_corr/rollout_is_ratio_fraction_high:2.677973679965362e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015175183943938464 - rollout_corr/rollout_is_max:2.404892683029175 - rollout_corr/rollout_is_min:0.08478856086730957 - rollout_corr/rollout_is_std:0.03068605810403824 - rollout_corr/rollout_is_eff_sample_size:0.9990592516626259 - rollout_corr/rollout_is_seq_mean:0.9998985528945923 - rollout_corr/rollout_is_seq_std:0.0018251852598041296 - rollout_corr/rollout_is_seq_max:1.005946397781372 - rollout_corr/rollout_is_seq_min:0.9921255707740784 - rollout_corr/rollout_is_seq_max_deviation:0.00787442922592163 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.045398538466543) - rollout_corr/training_log_ppl:np.float64(0.043960788469121326) - rollout_corr/kl:np.float64(0.0007093173290892985) - rollout_corr/k3_kl:np.float64(0.0007489784828464963) - rollout_corr/rollout_ppl:np.float64(1.0446436894126236) - rollout_corr/rollout_log_ppl:np.float64(0.04325147051258682) - rollout_corr/log_ppl_diff:np.float64(0.0007093179565345054) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016133448907567072) - rollout_corr/log_ppl_diff_max:np.float64(0.012227427214384079) - rollout_corr/log_ppl_diff_min:np.float64(-0.0077995359897613525) - rollout_corr/ppl_ratio:np.float64(1.0007119183428586) - rollout_corr/chi2_token:np.float64(0.0015946859493851662) - rollout_corr/chi2_seq:np.float64(0.6929341466166079) - actor/pg_loss:np.float64(-1.062639057636261e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003660259849311842) - actor/ppo_kl:np.float64(3.530177728094941e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22072983905673027) - perf/mfu/actor:np.float64(0.07971274541131826) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.79170608520508) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0010499887866899371 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0010499887866899371 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:437.59765625 - response_length/max:839.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.59765625 - response_length_non_aborted/max:839.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.78125 - prompt_length/max:519.0 - prompt_length/min:271.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014859624207019806 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1434442829340696) - timing_s/agent_loop/generate_sequences/max:np.float64(7.360479846596718) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.855869824379624) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.360479846596718) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:449 - timing_s/agent_loop/slowest/response_length:839 - timing_s/gen:13.065175823867321 - timing_s/reward:0.08288484252989292 - timing_s/old_log_prob:2.3260674998164177 - timing_s/adv:0.1202508956193924 - timing_s/update_actor:9.77362084761262 - timing_s/step:25.44756579771638 - timing_s/stop_profile:0.00011766701936721802 - timing_per_token_ms/update_actor:0.0448955238134316 - timing_per_token_ms/adv:0.0005523773667960165 - timing_per_token_ms/gen:0.11662732268571588 - perf/total_num_tokens:217697 - perf/time_per_step:25.44756579771638 - perf/throughput:1069.3409820141605 (TaskRunner pid=2146199) Training Progress: 55%|█████▌ | 55/100 [28:31<21:52, 29.16s/it] (TaskRunner pid=2146199) step:56 - global_seqlen/min:20051 - global_seqlen/max:30785 - global_seqlen/minmax_diff:10734 - global_seqlen/balanced_min:26229 - global_seqlen/balanced_max:26232 - global_seqlen/mean:26230.875 - actor/entropy:0.0529906265437603 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311316251754761 - training/rollout_probs_diff_mean:0.0022014498244971037 - training/rollout_probs_diff_std:0.01207329798489809 - training/rollout_actor_probs_pearson_corr:0.995204508304596 - rollout_corr/rollout_is_mean:1.0000399351119995 - rollout_corr/rollout_is_ratio_fraction_high:4.456685564946383e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002050075272563845 - rollout_corr/rollout_is_max:3.5453238487243652 - rollout_corr/rollout_is_min:0.25997379422187805 - rollout_corr/rollout_is_std:0.031217558309435844 - rollout_corr/rollout_is_eff_sample_size:0.9990262938763221 - rollout_corr/rollout_is_seq_mean:1.000037431716919 - rollout_corr/rollout_is_seq_std:0.0017291160766035318 - rollout_corr/rollout_is_seq_max:1.0075055360794067 - rollout_corr/rollout_is_seq_min:0.9947180151939392 - rollout_corr/rollout_is_seq_max_deviation:0.007505536079406738 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0539798219688237) - rollout_corr/training_log_ppl:np.float64(0.051764934929451556) - rollout_corr/kl:np.float64(0.0005245974664322972) - rollout_corr/k3_kl:np.float64(0.0006912269404182325) - rollout_corr/rollout_ppl:np.float64(1.053418517112732) - rollout_corr/rollout_log_ppl:np.float64(0.05124033657648397) - rollout_corr/log_ppl_diff:np.float64(0.0005245983529675868) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015802559655639925) - rollout_corr/log_ppl_diff_max:np.float64(0.009826317429542542) - rollout_corr/log_ppl_diff_min:np.float64(-0.006994552910327911) - rollout_corr/ppl_ratio:np.float64(1.0005270240362734) - rollout_corr/chi2_token:np.float64(0.0019079502671957016) - rollout_corr/chi2_seq:np.float64(2.325224193977192) - actor/pg_loss:np.float64(5.835480988025665e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020010624029964674) - actor/ppo_kl:np.float64(-4.901502908438715e-06) - actor/pg_clipfrac_lower:np.float64(1.2056327250320464e-05) - actor/grad_norm:np.float64(0.14924942329525948) - perf/mfu/actor:np.float64(0.07587062526530655) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.75049591064453) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003285691374912858 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003285691374912858 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:438.24609375 - response_length/max:949.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:438.24609375 - response_length_non_aborted/max:949.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:381.46875 - prompt_length/max:480.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016895495355129242 - timing_s/agent_loop/generate_sequences/min:np.float64(2.157773807644844) - timing_s/agent_loop/generate_sequences/max:np.float64(7.804611839354038) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.670612991540111) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.804611839354038) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:949 - timing_s/gen:13.501951225101948 - timing_s/reward:0.06336017325520515 - timing_s/old_log_prob:2.371345652267337 - timing_s/adv:0.12070532329380512 - timing_s/update_actor:9.9235181491822 - timing_s/step:26.06695833429694 - timing_s/stop_profile:0.00011040270328521729 - timing_per_token_ms/update_actor:0.04728930196372691 - timing_per_token_ms/adv:0.0005752063326795481 - timing_per_token_ms/gen:0.12034789978787913 - perf/total_num_tokens:209847 - perf/time_per_step:26.06695833429694 - perf/throughput:1006.2882927728238 (TaskRunner pid=2146199) Training Progress: 56%|█████▌ | 56/100 [28:57<20:42, 28.24s/it] (TaskRunner pid=2146199) step:57 - global_seqlen/min:22959 - global_seqlen/max:29561 - global_seqlen/minmax_diff:6602 - global_seqlen/balanced_min:26322 - global_seqlen/balanced_max:26323 - global_seqlen/mean:26322.625 - actor/entropy:0.05216638371348381 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6357130408287048 - training/rollout_probs_diff_mean:0.002540565561503172 - training/rollout_probs_diff_std:0.013406551443040371 - training/rollout_actor_probs_pearson_corr:0.9938746690750122 - rollout_corr/rollout_is_mean:0.9999613761901855 - rollout_corr/rollout_is_ratio_fraction_high:9.433516970602795e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001981038658414036 - rollout_corr/rollout_is_max:5.376122951507568 - rollout_corr/rollout_is_min:0.25212863087654114 - rollout_corr/rollout_is_std:0.03450770303606987 - rollout_corr/rollout_is_eff_sample_size:0.9988103968292807 - rollout_corr/rollout_is_seq_mean:1.0000181198120117 - rollout_corr/rollout_is_seq_std:0.002236654283478856 - rollout_corr/rollout_is_seq_max:1.0119788646697998 - rollout_corr/rollout_is_seq_min:0.9929762482643127 - rollout_corr/rollout_is_seq_max_deviation:0.011978864669799805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0550304455682635) - rollout_corr/training_log_ppl:np.float64(0.053291174070182024) - rollout_corr/kl:np.float64(0.0007571155535259777) - rollout_corr/k3_kl:np.float64(0.0009619000965201963) - rollout_corr/rollout_ppl:np.float64(1.0542305344715714) - rollout_corr/rollout_log_ppl:np.float64(0.052534058328092215) - rollout_corr/log_ppl_diff:np.float64(0.0007571157420898089) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018027576970780501) - rollout_corr/log_ppl_diff_max:np.float64(0.01066737249493599) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055869147181510925) - rollout_corr/ppl_ratio:np.float64(1.0007602223195136) - rollout_corr/chi2_token:np.float64(0.003015274414792657) - rollout_corr/chi2_seq:np.float64(1.4729875740595162) - actor/pg_loss:np.float64(0.0003496753051877022) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005416328058345243) - actor/ppo_kl:np.float64(9.268780859983394e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4019697979092598) - perf/mfu/actor:np.float64(0.07667169391184367) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.74153518676758) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00010376868885941803 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:-0.00010376868885941803 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:414.08203125 - response_length/max:801.0 - response_length/min:200.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:414.08203125 - response_length_non_aborted/max:801.0 - response_length_non_aborted/min:200.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:408.5 - prompt_length/max:499.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.818188846111298e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.395354215055704) - timing_s/agent_loop/generate_sequences/max:np.float64(6.992354471236467) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.578056915801426) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.992354471236467) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:801 - timing_s/gen:12.825032241642475 - timing_s/reward:0.06562882661819458 - timing_s/old_log_prob:2.3428254537284374 - timing_s/adv:0.143313717097044 - timing_s/update_actor:9.901038555428386 - timing_s/step:25.364522205665708 - timing_s/stop_profile:0.00011902116239070892 - timing_per_token_ms/update_actor:0.04701772028544069 - timing_per_token_ms/adv:0.0006805633798730369 - timing_per_token_ms/gen:0.12098516335684614 - perf/total_num_tokens:210581 - perf/time_per_step:25.364522205665708 - perf/throughput:1037.7733428828508 (TaskRunner pid=2146199) Training Progress: 57%|█████▋ | 57/100 [29:23<19:38, 27.40s/it] (TaskRunner pid=2146199) step:58 - global_seqlen/min:23813 - global_seqlen/max:31584 - global_seqlen/minmax_diff:7771 - global_seqlen/balanced_min:26740 - global_seqlen/balanced_max:26743 - global_seqlen/mean:26741.625 - actor/entropy:0.04503406956791878 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44297531247138977 - training/rollout_probs_diff_mean:0.001968941418454051 - training/rollout_probs_diff_std:0.011529196053743362 - training/rollout_actor_probs_pearson_corr:0.9949560761451721 - rollout_corr/rollout_is_mean:0.9999918341636658 - rollout_corr/rollout_is_ratio_fraction_high:8.61734661157243e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012064285692758858 - rollout_corr/rollout_is_max:2.3352396488189697 - rollout_corr/rollout_is_min:0.26035764813423157 - rollout_corr/rollout_is_std:0.029946811497211456 - rollout_corr/rollout_is_eff_sample_size:0.9991038730691704 - rollout_corr/rollout_is_seq_mean:1.0000433921813965 - rollout_corr/rollout_is_seq_std:0.001880717114545405 - rollout_corr/rollout_is_seq_max:1.0093436241149902 - rollout_corr/rollout_is_seq_min:0.9919431209564209 - rollout_corr/rollout_is_seq_max_deviation:0.009343624114990234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0515558468177915) - rollout_corr/training_log_ppl:np.float64(0.04978281667899864) - rollout_corr/kl:np.float64(0.0008949635980393111) - rollout_corr/k3_kl:np.float64(0.0006857588285384963) - rollout_corr/rollout_ppl:np.float64(1.050603509414941) - rollout_corr/rollout_log_ppl:np.float64(0.048887852818552346) - rollout_corr/log_ppl_diff:np.float64(0.0008949638604462962) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017536951518195565) - rollout_corr/log_ppl_diff_max:np.float64(0.01226063072681427) - rollout_corr/log_ppl_diff_min:np.float64(-0.006492689251899719) - rollout_corr/ppl_ratio:np.float64(1.0008981772698462) - rollout_corr/chi2_token:np.float64(0.000994114438071847) - rollout_corr/chi2_seq:np.float64(0.4707074877806008) - actor/pg_loss:np.float64(-5.066883750259876e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028918130146848853) - actor/ppo_kl:np.float64(0.00038876773286822264) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15863015316426754) - perf/mfu/actor:np.float64(0.07820272206962162) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.78412246704102) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00048257142771035433 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00048257142771035433 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.30078125 - response_length/max:991.0 - response_length/min:194.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.30078125 - response_length_non_aborted/max:991.0 - response_length_non_aborted/min:194.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:382.375 - prompt_length/max:482.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.19018280506134e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2879326082766056) - timing_s/agent_loop/generate_sequences/max:np.float64(8.235209224745631) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.9104437457208405) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.235209224745631) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:991 - timing_s/gen:13.980025133118033 - timing_s/reward:0.06406913883984089 - timing_s/old_log_prob:2.2960866168141365 - timing_s/adv:0.12312664091587067 - timing_s/update_actor:9.704679196700454 - timing_s/step:26.253764620050788 - timing_s/stop_profile:2.9783695936203003e-05 - timing_per_token_ms/update_actor:0.04536317069690255 - timing_per_token_ms/adv:0.0005755383270270163 - timing_per_token_ms/gen:0.12047072371164663 - perf/total_num_tokens:213933 - perf/time_per_step:26.253764620050788 - perf/throughput:1018.5824923400364 (TaskRunner pid=2146199) Training Progress: 58%|█████▊ | 58/100 [29:49<18:56, 27.06s/it] (TaskRunner pid=2146199) step:59 - global_seqlen/min:21733 - global_seqlen/max:30427 - global_seqlen/minmax_diff:8694 - global_seqlen/balanced_min:25465 - global_seqlen/balanced_max:25475 - global_seqlen/mean:25472.75 - actor/entropy:0.05039264261722565 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5450959205627441 - training/rollout_probs_diff_mean:0.0021395166404545307 - training/rollout_probs_diff_std:0.011858855374157429 - training/rollout_actor_probs_pearson_corr:0.9952424764633179 - rollout_corr/rollout_is_mean:0.9999675154685974 - rollout_corr/rollout_is_ratio_fraction_high:4.703580270870589e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015992173575796187 - rollout_corr/rollout_is_max:2.271881103515625 - rollout_corr/rollout_is_min:0.18539360165596008 - rollout_corr/rollout_is_std:0.03195911645889282 - rollout_corr/rollout_is_eff_sample_size:0.9989795380042602 - rollout_corr/rollout_is_seq_mean:1.0000231266021729 - rollout_corr/rollout_is_seq_std:0.001936701126396656 - rollout_corr/rollout_is_seq_max:1.007285237312317 - rollout_corr/rollout_is_seq_min:0.9922624826431274 - rollout_corr/rollout_is_seq_max_deviation:0.007737517356872559 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0550377136096358) - rollout_corr/training_log_ppl:np.float64(0.052757243649466545) - rollout_corr/kl:np.float64(0.0007581577460911149) - rollout_corr/k3_kl:np.float64(0.0007770038147141278) - rollout_corr/rollout_ppl:np.float64(1.054213949944824) - rollout_corr/rollout_log_ppl:np.float64(0.051999084589624545) - rollout_corr/log_ppl_diff:np.float64(0.0007581590598420007) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016631113194307545) - rollout_corr/log_ppl_diff_max:np.float64(0.011454001069068909) - rollout_corr/log_ppl_diff_min:np.float64(-0.005804732441902161) - rollout_corr/ppl_ratio:np.float64(1.0007612542249262) - rollout_corr/chi2_token:np.float64(0.0015565590001642704) - rollout_corr/chi2_seq:np.float64(0.4730744904372841) - actor/pg_loss:np.float64(1.940352376550436e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039659078493059496) - actor/ppo_kl:np.float64(0.00021726011601685968) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2132607400417328) - perf/mfu/actor:np.float64(0.07411381237442019) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.73226928710938) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0019613930489867926 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0019613930489867926 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:415.2421875 - response_length/max:1164.0 - response_length/min:193.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:415.2421875 - response_length_non_aborted/max:1164.0 - response_length_non_aborted/min:193.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:380.78125 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7937814593315125e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5911886040121317) - timing_s/agent_loop/generate_sequences/max:np.float64(8.663472402840853) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.082937864332052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.663472402840853) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:1164 - timing_s/gen:14.485273379832506 - timing_s/reward:0.06313352473080158 - timing_s/old_log_prob:2.322603428736329 - timing_s/adv:0.12487081997096539 - timing_s/update_actor:9.94132635742426 - timing_s/step:27.02480658888817 - timing_s/stop_profile:0.0001088511198759079 - timing_per_token_ms/update_actor:0.04878412400223896 - timing_per_token_ms/adv:0.0006127666819000961 - timing_per_token_ms/gen:0.13626529491291325 - perf/total_num_tokens:203782 - perf/time_per_step:27.02480658888817 - perf/throughput:942.5691879132141 (TaskRunner pid=2146199) Training Progress: 59%|█████▉ | 59/100 [30:16<18:29, 27.07s/it] (TaskRunner pid=2146199) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2146199) validation generation end (TaskRunner pid=2146199) [prompt] system (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) To calculate the volume of the unit cell for a crystal structure with orthogonal lattice parameters (α = β = γ = 90.00º), we use the formula for the volume of a unit cell in a cubic or orthorhombic system: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = a \times b \times c $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given: (TaskRunner pid=2146199) - $ a = 7.82 $ Å (TaskRunner pid=2146199) - $ b = 7.82 $ Å (TaskRunner pid=2146199) - $ c = 17.88 $ Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Substituting the values: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Then multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 61.1524 \times 17.88 \approx 1094.31 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Therefore, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2146577) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2146199) step:60 - global_seqlen/min:22324 - global_seqlen/max:29522 - global_seqlen/minmax_diff:7198 - global_seqlen/balanced_min:26684 - global_seqlen/balanced_max:26689 - global_seqlen/mean:26686.0 - actor/entropy:0.05162710323929787 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5994813442230225 - training/rollout_probs_diff_mean:0.002199057023972273 - training/rollout_probs_diff_std:0.011910058557987213 - training/rollout_actor_probs_pearson_corr:0.9950743317604065 - rollout_corr/rollout_is_mean:0.9999393224716187 - rollout_corr/rollout_is_ratio_fraction_high:5.1888750022044405e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001643143768887967 - rollout_corr/rollout_is_max:2.546652317047119 - rollout_corr/rollout_is_min:0.3058864176273346 - rollout_corr/rollout_is_std:0.03131287917494774 - rollout_corr/rollout_is_eff_sample_size:0.9990201070974829 - rollout_corr/rollout_is_seq_mean:0.9999314546585083 - rollout_corr/rollout_is_seq_std:0.0016388982767239213 - rollout_corr/rollout_is_seq_max:1.006493091583252 - rollout_corr/rollout_is_seq_min:0.9942454695701599 - rollout_corr/rollout_is_seq_max_deviation:0.006493091583251953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0566336568444967) - rollout_corr/training_log_ppl:np.float64(0.05417421268612088) - rollout_corr/kl:np.float64(0.0006475147506446177) - rollout_corr/k3_kl:np.float64(0.0006050457191193459) - rollout_corr/rollout_ppl:np.float64(1.0559430145658553) - rollout_corr/rollout_log_ppl:np.float64(0.053526697716733906) - rollout_corr/log_ppl_diff:np.float64(0.0006475149693869753) - rollout_corr/log_ppl_abs_diff:np.float64(0.001334731154202018) - rollout_corr/log_ppl_diff_max:np.float64(0.005721639841794968) - rollout_corr/log_ppl_diff_min:np.float64(-0.003946289420127869) - rollout_corr/ppl_ratio:np.float64(1.0006490682717413) - rollout_corr/chi2_token:np.float64(0.0011737903114408255) - rollout_corr/chi2_seq:np.float64(0.29386569769121706) - actor/pg_loss:np.float64(1.2919190339744091e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010246875945085776) - actor/ppo_kl:np.float64(4.286962526833804e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1441601449623704) - perf/mfu/actor:np.float64(0.07813690259122516) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.65783309936523) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7659574468085106) - val-aux/sciknoweval/reward/std@16:np.float64(0.06572326778442847) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7910957446808511) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.04845734779229193) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7411595744680852) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.06408244632173159) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7662446808510638) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.06548697716699092) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8093297872340427) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.03171909176319926) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.716531914893617) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.05771009946399257) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7697872340425533) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.04814177644558915) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8216702127659574) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.018530557466029343) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6918085106382978) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.050328059993951034) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.770436170212766) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.029727463536568386) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8278510638297872) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.007931960723589085) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6687978723404255) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.036459684761228635) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7699042553191489) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.017102662541135523) - val-aux/sciknoweval/score/mean@16:np.float64(0.7659574468085106) - val-aux/sciknoweval/score/std@16:np.float64(0.06572326778442847) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7910957446808511) - val-aux/sciknoweval/score/best@2/std:np.float64(0.04845734779229193) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7411595744680852) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.06408244632173159) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7662446808510638) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.06548697716699092) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8093297872340427) - val-aux/sciknoweval/score/best@4/std:np.float64(0.03171909176319926) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.716531914893617) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.05771009946399257) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7697872340425533) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.04814177644558915) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8216702127659574) - val-aux/sciknoweval/score/best@8/std:np.float64(0.018530557466029343) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6918085106382978) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.050328059993951034) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.770436170212766) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.029727463536568386) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8278510638297872) - val-aux/sciknoweval/score/best@16/std:np.float64(0.007931960723589085) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6687978723404255) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.036459684761228635) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7699042553191489) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.017102662541135523) - val-core/sciknoweval/acc/mean@16:np.float64(0.7659574468085106) - val-aux/sciknoweval/acc/std@16:np.float64(0.06572326778442847) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7910957446808511) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.04845734779229193) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7411595744680852) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.06408244632173159) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7662446808510638) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.06548697716699092) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8093297872340427) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.03171909176319926) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.716531914893617) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.05771009946399257) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7697872340425533) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.04814177644558915) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8216702127659574) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.018530557466029343) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6918085106382978) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.050328059993951034) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.770436170212766) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.029727463536568386) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8278510638297872) - val-core/sciknoweval/acc/best@16/std:np.float64(0.007931960723589085) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6687978723404255) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.036459684761228635) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7699042553191489) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.017102662541135523) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003263586200773716 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003263586200773716 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:451.6875 - response_length/max:1017.0 - response_length/min:207.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:451.6875 - response_length_non_aborted/max:1017.0 - response_length_non_aborted/min:207.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:382.25 - prompt_length/max:495.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.750016033649445e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.481598872691393) - timing_s/agent_loop/generate_sequences/max:np.float64(8.343909841030836) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.909151073465182) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.343909841030836) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:1017 - timing_s/gen:13.881776552647352 - timing_s/reward:0.06600731238722801 - timing_s/old_log_prob:2.3319669235497713 - timing_s/adv:0.12174169160425663 - timing_s/update_actor:9.779073609039187 - timing_s/step:26.266387056559324 - timing_s/testing:44.38668565824628 - timing_s/save_checkpoint:6.61843335442245 - timing_s/stop_profile:0.0001239832490682602 - timing_per_token_ms/update_actor:0.04580619804878582 - timing_per_token_ms/adv:0.0005702507476029408 - timing_per_token_ms/gen:0.12005134004987679 - perf/total_num_tokens:213488 - perf/time_per_step:26.266387056559324 - perf/throughput:1015.9752821176786 (TaskRunner pid=2146199) Training Progress: 60%|██████ | 60/100 [31:33<28:05, 42.14s/it] (TaskRunner pid=2146199) step:61 - global_seqlen/min:22651 - global_seqlen/max:30181 - global_seqlen/minmax_diff:7530 - global_seqlen/balanced_min:27064 - global_seqlen/balanced_max:27068 - global_seqlen/mean:27066.375 - actor/entropy:0.04668615758419037 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9814402461051941 - training/rollout_probs_diff_mean:0.0021403066348284483 - training/rollout_probs_diff_std:0.011851292103528976 - training/rollout_actor_probs_pearson_corr:0.9946837425231934 - rollout_corr/rollout_is_mean:1.0000090599060059 - rollout_corr/rollout_is_ratio_fraction_high:4.512187297223136e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.317062070593238e-05 - rollout_corr/rollout_is_max:4.31656551361084 - rollout_corr/rollout_is_min:0.0025121995713561773 - rollout_corr/rollout_is_std:0.030438372865319252 - rollout_corr/rollout_is_eff_sample_size:0.9990741250219439 - rollout_corr/rollout_is_seq_mean:1.0000107288360596 - rollout_corr/rollout_is_seq_std:0.0016845143400132656 - rollout_corr/rollout_is_seq_max:1.005863904953003 - rollout_corr/rollout_is_seq_min:0.9939930438995361 - rollout_corr/rollout_is_seq_max_deviation:0.006006956100463867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.050461934413761) - rollout_corr/training_log_ppl:np.float64(0.04892735888097377) - rollout_corr/kl:np.float64(0.000603220537530369) - rollout_corr/k3_kl:np.float64(0.0006021562215039467) - rollout_corr/rollout_ppl:np.float64(1.0498226466588676) - rollout_corr/rollout_log_ppl:np.float64(0.0483241376150545) - rollout_corr/log_ppl_diff:np.float64(0.0006032212659192737) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014146280736895278) - rollout_corr/log_ppl_diff_max:np.float64(0.006837852299213409) - rollout_corr/log_ppl_diff_min:np.float64(-0.0043100640177726746) - rollout_corr/ppl_ratio:np.float64(1.0006050020456314) - rollout_corr/chi2_token:np.float64(0.001273707253858447) - rollout_corr/chi2_seq:np.float64(0.7202077922411263) - actor/pg_loss:np.float64(8.88691283762455e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002183186552429106) - actor/ppo_kl:np.float64(8.91062679491128e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17633902095258236) - perf/mfu/actor:np.float64(0.07987819575197866) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.01884841918945) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001191217452287674 - critic/advantages/max:0.625 - critic/advantages/min:-0.625 - critic/returns/mean:-0.001191217452287674 - critic/returns/max:0.625 - critic/returns/min:-0.625 - response_length/mean:432.85546875 - response_length/max:849.0 - response_length/min:194.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:432.85546875 - response_length_non_aborted/max:849.0 - response_length_non_aborted/min:194.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.96875 - prompt_length/max:509.0 - prompt_length/min:269.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014674849808216095 - timing_s/agent_loop/generate_sequences/min:np.float64(2.280557092279196) - timing_s/agent_loop/generate_sequences/max:np.float64(7.401343552395701) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.785547939638491) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.401343552395701) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:470 - timing_s/agent_loop/slowest/response_length:849 - timing_s/gen:13.05488439835608 - timing_s/reward:0.06190898269414902 - timing_s/old_log_prob:2.3691947367042303 - timing_s/adv:0.11887696012854576 - timing_s/update_actor:9.792532823979855 - timing_s/step:25.496616369113326 - timing_s/stop_profile:0.00011237896978855133 - timing_per_token_ms/update_actor:0.04522462291302333 - timing_per_token_ms/adv:0.0005490066555299046 - timing_per_token_ms/gen:0.11781217025706907 - perf/total_num_tokens:216531 - perf/time_per_step:25.496616369113326 - perf/throughput:1061.567331451411 (TaskRunner pid=2146199) Training Progress: 61%|██████ | 61/100 [31:59<24:09, 37.17s/it] (TaskRunner pid=2146199) step:62 - global_seqlen/min:19889 - global_seqlen/max:31968 - global_seqlen/minmax_diff:12079 - global_seqlen/balanced_min:27571 - global_seqlen/balanced_max:27574 - global_seqlen/mean:27572.5 - actor/entropy:0.051550526171922684 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39407843351364136 - training/rollout_probs_diff_mean:0.002012582030147314 - training/rollout_probs_diff_std:0.010950378142297268 - training/rollout_actor_probs_pearson_corr:0.9959352612495422 - rollout_corr/rollout_is_mean:0.9998511672019958 - rollout_corr/rollout_is_ratio_fraction_high:4.124187398701906e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011547725443961099 - rollout_corr/rollout_is_max:3.03580904006958 - rollout_corr/rollout_is_min:0.20092754065990448 - rollout_corr/rollout_is_std:0.02988305129110813 - rollout_corr/rollout_is_eff_sample_size:0.9991075619504816 - rollout_corr/rollout_is_seq_mean:0.9998239874839783 - rollout_corr/rollout_is_seq_std:0.0016010386170819402 - rollout_corr/rollout_is_seq_max:1.0069009065628052 - rollout_corr/rollout_is_seq_min:0.9938949346542358 - rollout_corr/rollout_is_seq_max_deviation:0.006900906562805176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0581231056712568) - rollout_corr/training_log_ppl:np.float64(0.05531671060361987) - rollout_corr/kl:np.float64(0.0006843379073320222) - rollout_corr/k3_kl:np.float64(0.0005994963409978737) - rollout_corr/rollout_ppl:np.float64(1.0573846632614732) - rollout_corr/rollout_log_ppl:np.float64(0.054632372406558716) - rollout_corr/log_ppl_diff:np.float64(0.0006843381970611517) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014233539905035286) - rollout_corr/log_ppl_diff_max:np.float64(0.007586754858493805) - rollout_corr/log_ppl_diff_min:np.float64(-0.006864540278911591) - rollout_corr/ppl_ratio:np.float64(1.0006862331647426) - rollout_corr/chi2_token:np.float64(0.0011065129656344652) - rollout_corr/chi2_seq:np.float64(0.39669181080535054) - actor/pg_loss:np.float64(0.000123962527140975) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002980156982630433) - actor/ppo_kl:np.float64(-3.147265968017621e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18734022229909897) - perf/mfu/actor:np.float64(0.08001500507219289) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.9618034362793) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.8125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021177702583372593 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0021177702583372593 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:473.578125 - response_length/max:869.0 - response_length/min:190.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:473.578125 - response_length_non_aborted/max:869.0 - response_length_non_aborted/min:190.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.0625 - prompt_length/max:541.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011656433343887329 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2676097843796015) - timing_s/agent_loop/generate_sequences/max:np.float64(7.916997352614999) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.074865577968012) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.916997352614999) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:445 - timing_s/agent_loop/slowest/response_length:866 - timing_s/gen:13.702079916372895 - timing_s/reward:0.06428892910480499 - timing_s/old_log_prob:2.4779190849512815 - timing_s/adv:0.13680999353528023 - timing_s/update_actor:9.911134421825409 - timing_s/step:26.380057008937 - timing_s/stop_profile:0.00011780485510826111 - timing_per_token_ms/update_actor:0.044932153512673 - timing_per_token_ms/adv:0.0006202284592224147 - timing_per_token_ms/gen:0.11301989439088138 - perf/total_num_tokens:220580 - perf/time_per_step:26.380057008937 - perf/throughput:1045.2024417786142 (TaskRunner pid=2146199) Training Progress: 62%|██████▏ | 62/100 [32:25<21:30, 33.95s/it] (TaskRunner pid=2146199) step:63 - global_seqlen/min:22750 - global_seqlen/max:31743 - global_seqlen/minmax_diff:8993 - global_seqlen/balanced_min:27867 - global_seqlen/balanced_max:27870 - global_seqlen/mean:27868.75 - actor/entropy:0.05611368641257286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8753384351730347 - training/rollout_probs_diff_mean:0.00235021417029202 - training/rollout_probs_diff_std:0.012638549320399761 - training/rollout_actor_probs_pearson_corr:0.9949365258216858 - rollout_corr/rollout_is_mean:0.9998043775558472 - rollout_corr/rollout_is_ratio_fraction_high:3.3713735319906846e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00022756772523280233 - rollout_corr/rollout_is_max:3.568155288696289 - rollout_corr/rollout_is_min:0.024168552830815315 - rollout_corr/rollout_is_std:0.03174109756946564 - rollout_corr/rollout_is_eff_sample_size:0.9989931003256733 - rollout_corr/rollout_is_seq_mean:0.999756932258606 - rollout_corr/rollout_is_seq_std:0.0016375579871237278 - rollout_corr/rollout_is_seq_max:1.0047188997268677 - rollout_corr/rollout_is_seq_min:0.9944717288017273 - rollout_corr/rollout_is_seq_max_deviation:0.005528271198272705 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.059029346331954) - rollout_corr/training_log_ppl:np.float64(0.056891138083301485) - rollout_corr/kl:np.float64(0.001084191916399746) - rollout_corr/k3_kl:np.float64(0.0009494085434624822) - rollout_corr/rollout_ppl:np.float64(1.0578639972954988) - rollout_corr/rollout_log_ppl:np.float64(0.05580694597938418) - rollout_corr/log_ppl_diff:np.float64(0.001084192103917303) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017548447949593537) - rollout_corr/log_ppl_diff_max:np.float64(0.016530685126781464) - rollout_corr/log_ppl_diff_min:np.float64(-0.006118997931480408) - rollout_corr/ppl_ratio:np.float64(1.0010877882596105) - rollout_corr/chi2_token:np.float64(0.0015552681870758533) - rollout_corr/chi2_seq:np.float64(0.7069385580252856) - actor/pg_loss:np.float64(7.042405195534229e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005038880426582182) - actor/ppo_kl:np.float64(0.00020799350089895086) - actor/pg_clipfrac_lower:np.float64(7.141225069062784e-06) - actor/grad_norm:np.float64(0.2414776235818863) - perf/mfu/actor:np.float64(0.08123417216512625) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.89482116699219) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003714832244440913 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003714832244440913 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:463.4609375 - response_length/max:734.0 - response_length/min:212.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.4609375 - response_length_non_aborted/max:734.0 - response_length_non_aborted/min:212.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.4375 - prompt_length/max:483.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001302771270275116 - timing_s/agent_loop/generate_sequences/min:np.float64(2.6069273613393307) - timing_s/agent_loop/generate_sequences/max:np.float64(7.248260343447328) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.240580555146153) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.248260343447328) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:425 - timing_s/agent_loop/slowest/response_length:721 - timing_s/gen:13.166410610079765 - timing_s/reward:0.06019227020442486 - timing_s/old_log_prob:2.3198466766625643 - timing_s/adv:0.12805710546672344 - timing_s/update_actor:9.80109771154821 - timing_s/step:25.577948296442628 - timing_s/stop_profile:0.00010663829743862152 - timing_per_token_ms/update_actor:0.0439609675332954 - timing_per_token_ms/adv:0.0005743758935488829 - timing_per_token_ms/gen:0.11097222502300765 - perf/total_num_tokens:222950 - perf/time_per_step:25.577948296442628 - perf/throughput:1089.5615894209925 (TaskRunner pid=2146199) Training Progress: 63%|██████▎ | 63/100 [32:51<19:23, 31.46s/it] (TaskRunner pid=2146199) step:64 - global_seqlen/min:22456 - global_seqlen/max:31666 - global_seqlen/minmax_diff:9210 - global_seqlen/balanced_min:27417 - global_seqlen/balanced_max:27430 - global_seqlen/mean:27426.875 - actor/entropy:0.05738339573144913 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9152472019195557 - training/rollout_probs_diff_mean:0.002285720780491829 - training/rollout_probs_diff_std:0.01237183902412653 - training/rollout_actor_probs_pearson_corr:0.9951571822166443 - rollout_corr/rollout_is_mean:0.9999684691429138 - rollout_corr/rollout_is_ratio_fraction_high:6.70786394039169e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020123591821175069 - rollout_corr/rollout_is_max:3.6475942134857178 - rollout_corr/rollout_is_min:0.0008141319849528372 - rollout_corr/rollout_is_std:0.03171761706471443 - rollout_corr/rollout_is_eff_sample_size:0.9989951228086639 - rollout_corr/rollout_is_seq_mean:1.0000218152999878 - rollout_corr/rollout_is_seq_std:0.001857783179730177 - rollout_corr/rollout_is_seq_max:1.0079878568649292 - rollout_corr/rollout_is_seq_min:0.995285153388977 - rollout_corr/rollout_is_seq_max_deviation:0.0079878568649292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0603100107982755) - rollout_corr/training_log_ppl:np.float64(0.05805522078753711) - rollout_corr/kl:np.float64(0.0008025542520773143) - rollout_corr/k3_kl:np.float64(0.0007265918333985155) - rollout_corr/rollout_ppl:np.float64(1.0594583237543702) - rollout_corr/rollout_log_ppl:np.float64(0.05725266630270198) - rollout_corr/log_ppl_diff:np.float64(0.0008025544848351274) - rollout_corr/log_ppl_abs_diff:np.float64(0.00158304631622741) - rollout_corr/log_ppl_diff_max:np.float64(0.01399629469960928) - rollout_corr/log_ppl_diff_min:np.float64(-0.006409764289855957) - rollout_corr/ppl_ratio:np.float64(1.0008053586352617) - rollout_corr/chi2_token:np.float64(0.0011819545179605484) - rollout_corr/chi2_seq:np.float64(0.5894116314593703) - actor/pg_loss:np.float64(5.361693911254406e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022254843497648835) - actor/ppo_kl:np.float64(0.00014225537064760374) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1416674293577671) - perf/mfu/actor:np.float64(0.07974473268887006) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.9585189819336) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006120926118455827 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006120926118455827 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:465.87109375 - response_length/max:1351.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:465.87109375 - response_length_non_aborted/max:1351.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.21875 - prompt_length/max:473.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014016218483448029 - timing_s/agent_loop/generate_sequences/min:np.float64(2.30619840323925) - timing_s/agent_loop/generate_sequences/max:np.float64(10.024490017443895) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.053645237327146) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.024490017443895) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:1351 - timing_s/gen:15.539944503456354 - timing_s/reward:0.06522806733846664 - timing_s/old_log_prob:2.5436307955533266 - timing_s/adv:0.12606195732951164 - timing_s/update_actor:9.743870848789811 - timing_s/step:28.10589674115181 - timing_s/stop_profile:0.0001069139689207077 - timing_per_token_ms/update_actor:0.044408408034044215 - timing_per_token_ms/adv:0.0005745366421143114 - timing_per_token_ms/gen:0.1302997954391249 - perf/total_num_tokens:219415 - perf/time_per_step:28.10589674115181 - perf/throughput:975.8405950393461 (TaskRunner pid=2146199) Training Progress: 64%|██████▍ | 64/100 [33:19<18:16, 30.46s/it] (TaskRunner pid=2146199) step:65 - global_seqlen/min:23956 - global_seqlen/max:32040 - global_seqlen/minmax_diff:8084 - global_seqlen/balanced_min:27551 - global_seqlen/balanced_max:27554 - global_seqlen/mean:27552.875 - actor/entropy:0.057285018265247345 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9857746362686157 - training/rollout_probs_diff_mean:0.0022721809800714254 - training/rollout_probs_diff_std:0.011948187835514545 - training/rollout_actor_probs_pearson_corr:0.9954981207847595 - rollout_corr/rollout_is_mean:0.999977707862854 - rollout_corr/rollout_is_ratio_fraction_high:4.223008727421984e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.290619345847517e-05 - rollout_corr/rollout_is_max:2.934908866882324 - rollout_corr/rollout_is_min:0.00012515588605310768 - rollout_corr/rollout_is_std:0.030582938343286514 - rollout_corr/rollout_is_eff_sample_size:0.9990656768995193 - rollout_corr/rollout_is_seq_mean:0.9999144077301025 - rollout_corr/rollout_is_seq_std:0.0015106865903362632 - rollout_corr/rollout_is_seq_max:1.0071814060211182 - rollout_corr/rollout_is_seq_min:0.9931866526603699 - rollout_corr/rollout_is_seq_max_deviation:0.007181406021118164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0590817909687757) - rollout_corr/training_log_ppl:np.float64(0.056948647466015245) - rollout_corr/kl:np.float64(0.0009208322066633912) - rollout_corr/k3_kl:np.float64(0.0007086626972592214) - rollout_corr/rollout_ppl:np.float64(1.0580947631970048) - rollout_corr/rollout_log_ppl:np.float64(0.056027814742265036) - rollout_corr/log_ppl_diff:np.float64(0.000920832723750209) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015465557053175871) - rollout_corr/log_ppl_diff_max:np.float64(0.04472583904862404) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034879669547080994) - rollout_corr/ppl_ratio:np.float64(1.0009265248663723) - rollout_corr/chi2_token:np.float64(0.0007443218491971493) - rollout_corr/chi2_seq:np.float64(0.32490317872725427) - actor/pg_loss:np.float64(8.13810620456934e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021921267580182757) - actor/ppo_kl:np.float64(0.00017427348481557914) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15104441531002522) - perf/mfu/actor:np.float64(0.07975940101080513) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.91822576522827) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017124300356954336 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017124300356954336 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:462.49609375 - response_length/max:844.0 - response_length/min:167.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:462.49609375 - response_length_non_aborted/max:844.0 - response_length_non_aborted/min:167.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:398.53125 - prompt_length/max:490.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.055621922016144e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0171640664339066) - timing_s/agent_loop/generate_sequences/max:np.float64(7.7834068443626165) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.158651270903647) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.7834068443626165) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:821 - timing_s/gen:13.61933626793325 - timing_s/reward:0.06556718051433563 - timing_s/old_log_prob:2.3319720793515444 - timing_s/adv:0.12269252166152 - timing_s/update_actor:9.97441573254764 - timing_s/step:26.200398560613394 - timing_s/stop_profile:2.7783215045928955e-05 - timing_per_token_ms/update_actor:0.04525124752202647 - timing_per_token_ms/adv:0.0005566230459685241 - timing_per_token_ms/gen:0.11502914946860404 - perf/total_num_tokens:220423 - perf/time_per_step:26.200398560613394 - perf/throughput:1051.6204528819558 (TaskRunner pid=2146199) Training Progress: 65%|██████▌ | 65/100 [33:45<17:01, 29.19s/it] (TaskRunner pid=2146199) step:66 - global_seqlen/min:17960 - global_seqlen/max:31792 - global_seqlen/minmax_diff:13832 - global_seqlen/balanced_min:27059 - global_seqlen/balanced_max:27063 - global_seqlen/mean:27060.75 - actor/entropy:0.04720494896173477 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9590364098548889 - training/rollout_probs_diff_mean:0.0019200732931494713 - training/rollout_probs_diff_std:0.011346276849508286 - training/rollout_actor_probs_pearson_corr:0.9953361749649048 - rollout_corr/rollout_is_mean:1.000046968460083 - rollout_corr/rollout_is_ratio_fraction_high:7.685082528041676e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001280847063753754 - rollout_corr/rollout_is_max:2.691246747970581 - rollout_corr/rollout_is_min:0.004978310316801071 - rollout_corr/rollout_is_std:0.028717413544654846 - rollout_corr/rollout_is_eff_sample_size:0.9991761087007662 - rollout_corr/rollout_is_seq_mean:1.0001097917556763 - rollout_corr/rollout_is_seq_std:0.001836672076024115 - rollout_corr/rollout_is_seq_max:1.008939504623413 - rollout_corr/rollout_is_seq_min:0.9934665560722351 - rollout_corr/rollout_is_seq_max_deviation:0.008939504623413086 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0522325839847326) - rollout_corr/training_log_ppl:np.float64(0.050349305447525694) - rollout_corr/kl:np.float64(0.0005636716403429887) - rollout_corr/k3_kl:np.float64(0.0005898027524544602) - rollout_corr/rollout_ppl:np.float64(1.0516338841989636) - rollout_corr/rollout_log_ppl:np.float64(0.049785633285864606) - rollout_corr/log_ppl_diff:np.float64(0.0005636721616610885) - rollout_corr/log_ppl_abs_diff:np.float64(0.001345603548543295) - rollout_corr/log_ppl_diff_max:np.float64(0.011321118101477623) - rollout_corr/log_ppl_diff_min:np.float64(-0.00591813400387764) - rollout_corr/ppl_ratio:np.float64(1.000565622933209) - rollout_corr/chi2_token:np.float64(0.001125197159126401) - rollout_corr/chi2_seq:np.float64(0.4393501377198845) - actor/pg_loss:np.float64(-2.727378159761429e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.9025590770907e-05) - actor/ppo_kl:np.float64(0.0001293788432247256) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14534942619502544) - perf/mfu/actor:np.float64(0.07916224655585877) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.96170043945312) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002114464994519949 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002114464994519949 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.4609375 - response_length/max:887.0 - response_length/min:171.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:457.4609375 - response_length_non_aborted/max:887.0 - response_length_non_aborted/min:171.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.1875 - prompt_length/max:475.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.833672523498535e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.202094092965126) - timing_s/agent_loop/generate_sequences/max:np.float64(7.72814385779202) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.951475875473989) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.72814385779202) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:887 - timing_s/gen:13.406499216333032 - timing_s/reward:0.05297527275979519 - timing_s/old_log_prob:2.50131325237453 - timing_s/adv:0.14099153131246567 - timing_s/update_actor:9.807900903746486 - timing_s/step:25.949846172705293 - timing_s/stop_profile:0.00010638311505317688 - timing_per_token_ms/update_actor:0.04530501235066695 - timing_per_token_ms/adv:0.0006512732061771462 - timing_per_token_ms/gen:0.1144778346540264 - perf/total_num_tokens:216486 - perf/time_per_step:25.949846172705293 - perf/throughput:1042.809649810687 (TaskRunner pid=2146199) Training Progress: 66%|██████▌ | 66/100 [34:11<15:59, 28.23s/it] (TaskRunner pid=2146199) step:67 - global_seqlen/min:23785 - global_seqlen/max:31165 - global_seqlen/minmax_diff:7380 - global_seqlen/balanced_min:26311 - global_seqlen/balanced_max:26318 - global_seqlen/mean:26314.875 - actor/entropy:0.047155965119600296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4626723527908325 - training/rollout_probs_diff_mean:0.0019961653742939234 - training/rollout_probs_diff_std:0.011302574537694454 - training/rollout_actor_probs_pearson_corr:0.9952737092971802 - rollout_corr/rollout_is_mean:0.9999139904975891 - rollout_corr/rollout_is_ratio_fraction_high:1.7877733625937253e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.044980495469645e-05 - rollout_corr/rollout_is_max:2.404759168624878 - rollout_corr/rollout_is_min:0.3903655409812927 - rollout_corr/rollout_is_std:0.0293070450425148 - rollout_corr/rollout_is_eff_sample_size:0.999141596140458 - rollout_corr/rollout_is_seq_mean:0.9999265670776367 - rollout_corr/rollout_is_seq_std:0.0017951095942407846 - rollout_corr/rollout_is_seq_max:1.0052658319473267 - rollout_corr/rollout_is_seq_min:0.9931451082229614 - rollout_corr/rollout_is_seq_max_deviation:0.006854891777038574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.052351213991642) - rollout_corr/training_log_ppl:np.float64(0.050615647623999394) - rollout_corr/kl:np.float64(0.0005430876942667806) - rollout_corr/k3_kl:np.float64(0.0005947562525854266) - rollout_corr/rollout_ppl:np.float64(1.0517675406299531) - rollout_corr/rollout_log_ppl:np.float64(0.0500725592373783) - rollout_corr/log_ppl_diff:np.float64(0.0005430883866210934) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013831468240823597) - rollout_corr/log_ppl_diff_max:np.float64(0.011988319456577301) - rollout_corr/log_ppl_diff_min:np.float64(-0.00460275262594223) - rollout_corr/ppl_ratio:np.float64(1.0005452106706798) - rollout_corr/chi2_token:np.float64(0.0013179946690797806) - rollout_corr/chi2_seq:np.float64(0.34062383603304625) - actor/pg_loss:np.float64(2.9104645363986492e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002862602395907743) - actor/ppo_kl:np.float64(-4.0856137324851716e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15908449701964855) - perf/mfu/actor:np.float64(0.07687577173257971) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.89622116088867) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:2 - critic/score/mean:0.78515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008000286179594696 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008000286179594696 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:436.99609375 - response_length/max:1148.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:436.99609375 - response_length_non_aborted/max:1148.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.34375 - prompt_length/max:472.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010082870721817017 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2343768943101168) - timing_s/agent_loop/generate_sequences/max:np.float64(8.78978717699647) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.616361525913817) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.78978717699647) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:1148 - timing_s/gen:14.525874989107251 - timing_s/reward:0.0617762915790081 - timing_s/old_log_prob:2.3239715341478586 - timing_s/adv:0.12177818268537521 - timing_s/update_actor:9.727137768641114 - timing_s/step:26.84919297322631 - timing_s/stop_profile:2.8196722269058228e-05 - timing_per_token_ms/update_actor:0.04620551004251927 - timing_per_token_ms/adv:0.0005784664694653462 - timing_per_token_ms/gen:0.12984486586431918 - perf/total_num_tokens:210519 - perf/time_per_step:26.84919297322631 - perf/throughput:980.0992911124322 (TaskRunner pid=2146199) Training Progress: 67%|██████▋ | 67/100 [34:38<15:18, 27.82s/it] (TaskRunner pid=2146199) step:68 - global_seqlen/min:16575 - global_seqlen/max:30930 - global_seqlen/minmax_diff:14355 - global_seqlen/balanced_min:24717 - global_seqlen/balanced_max:24721 - global_seqlen/mean:24719.0 - actor/entropy:0.0701860785484314 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4863182306289673 - training/rollout_probs_diff_mean:0.0026776501908898354 - training/rollout_probs_diff_std:0.012804234400391579 - training/rollout_actor_probs_pearson_corr:0.9959306716918945 - rollout_corr/rollout_is_mean:0.9997715950012207 - rollout_corr/rollout_is_ratio_fraction_high:4.82476461911574e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001447429385734722 - rollout_corr/rollout_is_max:3.4113123416900635 - rollout_corr/rollout_is_min:0.2689007818698883 - rollout_corr/rollout_is_std:0.03389512374997139 - rollout_corr/rollout_is_eff_sample_size:0.9988520226061409 - rollout_corr/rollout_is_seq_mean:0.9998180270195007 - rollout_corr/rollout_is_seq_std:0.0020656553097069263 - rollout_corr/rollout_is_seq_max:1.014003872871399 - rollout_corr/rollout_is_seq_min:0.9946730732917786 - rollout_corr/rollout_is_seq_max_deviation:0.014003872871398926 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0806802995502949) - rollout_corr/training_log_ppl:np.float64(0.07548296367531293) - rollout_corr/kl:np.float64(0.0008378513675566523) - rollout_corr/k3_kl:np.float64(0.0008507416575582738) - rollout_corr/rollout_ppl:np.float64(1.079740488436073) - rollout_corr/rollout_log_ppl:np.float64(0.07464511207217583) - rollout_corr/log_ppl_diff:np.float64(0.0008378516031370964) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016117767117975745) - rollout_corr/log_ppl_diff_max:np.float64(0.014756649732589722) - rollout_corr/log_ppl_diff_min:np.float64(-0.010821301490068436) - rollout_corr/ppl_ratio:np.float64(1.0008407135028392) - rollout_corr/chi2_token:np.float64(0.00187430321238935) - rollout_corr/chi2_seq:np.float64(1.5505575174465775) - actor/pg_loss:np.float64(0.00015119847375899553) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006360013753692328) - actor/ppo_kl:np.float64(-4.278251520162257e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3528257682919502) - perf/mfu/actor:np.float64(0.07310701196950539) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.04170989990234) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:2 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0052083334885537624 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0052083334885537624 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:404.8125 - response_length/max:842.0 - response_length/min:178.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:404.8125 - response_length_non_aborted/max:842.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:367.65625 - prompt_length/max:491.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.606026828289032e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.128714656457305) - timing_s/agent_loop/generate_sequences/max:np.float64(7.13719742000103) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.395532650698442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.13719742000103) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:474 - timing_s/agent_loop/slowest/response_length:783 - timing_s/gen:12.949182905256748 - timing_s/reward:0.06127143278717995 - timing_s/old_log_prob:2.3618092574179173 - timing_s/adv:0.12222523614764214 - timing_s/update_actor:9.656971430405974 - timing_s/step:25.24130703508854 - timing_s/stop_profile:0.00011776387691497803 - timing_per_token_ms/update_actor:0.048833748485001285 - timing_per_token_ms/adv:0.0006180733249102013 - timing_per_token_ms/gen:0.12495351730408318 - perf/total_num_tokens:197752 - perf/time_per_step:25.24130703508854 - perf/throughput:979.3074489224165 (TaskRunner pid=2146199) Training Progress: 68%|██████▊ | 68/100 [35:03<14:25, 27.06s/it] (TaskRunner pid=2146199) step:69 - global_seqlen/min:19722 - global_seqlen/max:29972 - global_seqlen/minmax_diff:10250 - global_seqlen/balanced_min:25702 - global_seqlen/balanced_max:25705 - global_seqlen/mean:25703.625 - actor/entropy:0.0606297142803669 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47360923886299133 - training/rollout_probs_diff_mean:0.0022091970313340425 - training/rollout_probs_diff_std:0.011148248799145222 - training/rollout_actor_probs_pearson_corr:0.9963864088058472 - rollout_corr/rollout_is_mean:1.000080943107605 - rollout_corr/rollout_is_ratio_fraction_high:4.433056528796442e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.866113057592884e-05 - rollout_corr/rollout_is_max:3.0322513580322266 - rollout_corr/rollout_is_min:0.04621234908699989 - rollout_corr/rollout_is_std:0.030091755092144012 - rollout_corr/rollout_is_eff_sample_size:0.9990955434373779 - rollout_corr/rollout_is_seq_mean:1.0000951290130615 - rollout_corr/rollout_is_seq_std:0.0017333582509309053 - rollout_corr/rollout_is_seq_max:1.0068395137786865 - rollout_corr/rollout_is_seq_min:0.9950804114341736 - rollout_corr/rollout_is_seq_max_deviation:0.0068395137786865234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.066784041468054) - rollout_corr/training_log_ppl:np.float64(0.06333305252519494) - rollout_corr/kl:np.float64(0.000674074653694845) - rollout_corr/k3_kl:np.float64(0.0007290815497000835) - rollout_corr/rollout_ppl:np.float64(1.0660569407045841) - rollout_corr/rollout_log_ppl:np.float64(0.06265897721277724) - rollout_corr/log_ppl_diff:np.float64(0.000674075312417699) - rollout_corr/log_ppl_abs_diff:np.float64(0.001608990081876982) - rollout_corr/log_ppl_diff_max:np.float64(0.013620145618915558) - rollout_corr/log_ppl_diff_min:np.float64(-0.005113471299409866) - rollout_corr/ppl_ratio:np.float64(1.000676882918924) - rollout_corr/chi2_token:np.float64(0.0015482765156775713) - rollout_corr/chi2_seq:np.float64(0.6858308578375727) - actor/pg_loss:np.float64(9.150512050837278e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003587279111343378) - actor/ppo_kl:np.float64(0.00023110838921211636) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2257436253130436) - perf/mfu/actor:np.float64(0.0743879293156566) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.93959045410156) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:2 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003151903161779046 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003151903161779046 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:440.58203125 - response_length/max:953.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.58203125 - response_length_non_aborted/max:953.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.65625 - prompt_length/max:461.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015556439757347107 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0235330890864134) - timing_s/agent_loop/generate_sequences/max:np.float64(7.893370267003775) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.778758992346411) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.893370267003775) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:280 - timing_s/agent_loop/slowest/response_length:953 - timing_s/gen:13.722657777369022 - timing_s/reward:0.06291156262159348 - timing_s/old_log_prob:2.3050945717841387 - timing_s/adv:0.1402856558561325 - timing_s/update_actor:9.821962960064411 - timing_s/step:26.139795992523432 - timing_s/stop_profile:0.00011573918163776398 - timing_per_token_ms/update_actor:0.04776545604007417 - timing_per_token_ms/adv:0.0006822270003556527 - timing_per_token_ms/gen:0.12166663218371493 - perf/total_num_tokens:205629 - perf/time_per_step:26.139795992523432 - perf/throughput:983.3139098465731 (TaskRunner pid=2146199) Training Progress: 69%|██████▉ | 69/100 [35:30<13:51, 26.81s/it] (TaskRunner pid=2146199) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2146199) validation generation end (TaskRunner pid=2146199) [prompt] system (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) To calculate the volume of the unit cell for a crystal structure with orthogonal lattice parameters (α = β = γ = 90.00º), we use the formula for the volume of a rectangular unit cell: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = a \times b \times c $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given: (TaskRunner pid=2146199) - $ a = 7.82 $ Å (TaskRunner pid=2146199) - $ b = 7.82 $ Å (TaskRunner pid=2146199) - $ c = 17.88 $ Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Substituting the values: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Then multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Thus, the volume of the unit cell is approximately 1094.31 ų, which corresponds to option B. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2146577) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2146199) step:70 - global_seqlen/min:20947 - global_seqlen/max:32527 - global_seqlen/minmax_diff:11580 - global_seqlen/balanced_min:26070 - global_seqlen/balanced_max:26075 - global_seqlen/mean:26072.125 - actor/entropy:0.055282797664403915 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6418517827987671 - training/rollout_probs_diff_mean:0.0022460578475147486 - training/rollout_probs_diff_std:0.012188022024929523 - training/rollout_actor_probs_pearson_corr:0.9952324628829956 - rollout_corr/rollout_is_mean:1.0000709295272827 - rollout_corr/rollout_is_ratio_fraction_high:9.168844553641975e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00019254573271609843 - rollout_corr/rollout_is_max:5.015303611755371 - rollout_corr/rollout_is_min:0.05495893210172653 - rollout_corr/rollout_is_std:0.03124046139419079 - rollout_corr/rollout_is_eff_sample_size:0.9990249851282543 - rollout_corr/rollout_is_seq_mean:1.0000910758972168 - rollout_corr/rollout_is_seq_std:0.0019268683390691876 - rollout_corr/rollout_is_seq_max:1.0109930038452148 - rollout_corr/rollout_is_seq_min:0.9951149821281433 - rollout_corr/rollout_is_seq_max_deviation:0.010993003845214844 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0628072144463658) - rollout_corr/training_log_ppl:np.float64(0.05956334700840671) - rollout_corr/kl:np.float64(0.0006800062605432089) - rollout_corr/k3_kl:np.float64(0.0008513780118963865) - rollout_corr/rollout_ppl:np.float64(1.0620776806026697) - rollout_corr/rollout_log_ppl:np.float64(0.05888334027986275) - rollout_corr/log_ppl_diff:np.float64(0.0006800067285439582) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016670058003001031) - rollout_corr/log_ppl_diff_max:np.float64(0.009437952190637589) - rollout_corr/log_ppl_diff_min:np.float64(-0.004882164299488068) - rollout_corr/ppl_ratio:np.float64(1.0006827421020716) - rollout_corr/chi2_token:np.float64(0.0023069086018949747) - rollout_corr/chi2_seq:np.float64(0.9402522228192538) - actor/pg_loss:np.float64(3.209500573575497e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018685764280235162) - actor/ppo_kl:np.float64(0.00014930208785557397) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18278604745864868) - perf/mfu/actor:np.float64(0.07634229767616413) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.95454406738281) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.75) - val-aux/sciknoweval/reward/std@16:np.float64(0.03238652516099717) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.761968085106383) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.019849305768571204) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7382765957446809) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.03448672246647901) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7499893617021277) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.031894489195372695) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7675531914893617) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.009579224366797518) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7241170212765957) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.03401831698562356) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7547021276595743) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.02312934593323447) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7701489361702128) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.005536783555435272) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7094255319148935) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.029971268887346983) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7577021276595743) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.015575326468239798) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7729468085106382) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.005050122248183056) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6965106382978724) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.023040989796328896) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7604787234042553) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.011493118504791225) - val-aux/sciknoweval/score/mean@16:np.float64(0.75) - val-aux/sciknoweval/score/std@16:np.float64(0.03238652516099717) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.761968085106383) - val-aux/sciknoweval/score/best@2/std:np.float64(0.019849305768571204) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7382765957446809) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.03448672246647901) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7499893617021277) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.031894489195372695) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7675531914893617) - val-aux/sciknoweval/score/best@4/std:np.float64(0.009579224366797518) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7241170212765957) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.03401831698562356) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7547021276595743) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.02312934593323447) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7701489361702128) - val-aux/sciknoweval/score/best@8/std:np.float64(0.005536783555435272) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7094255319148935) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.029971268887346983) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7577021276595743) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.015575326468239798) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7729468085106382) - val-aux/sciknoweval/score/best@16/std:np.float64(0.005050122248183056) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6965106382978724) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.023040989796328896) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7604787234042553) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.011493118504791225) - val-core/sciknoweval/acc/mean@16:np.float64(0.75) - val-aux/sciknoweval/acc/std@16:np.float64(0.03238652516099717) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.761968085106383) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.019849305768571204) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7382765957446809) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.03448672246647901) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7499893617021277) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.031894489195372695) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7675531914893617) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.009579224366797518) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7241170212765957) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.03401831698562356) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7547021276595743) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.02312934593323447) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7701489361702128) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.005536783555435272) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7094255319148935) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.029971268887346983) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7577021276595743) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.015575326468239798) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7729468085106382) - val-core/sciknoweval/acc/best@16/std:np.float64(0.005050122248183056) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6965106382978724) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.023040989796328896) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7604787234042553) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.011493118504791225) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:2 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0018647137330844998 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0018647137330844998 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:426.03515625 - response_length/max:908.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.03515625 - response_length_non_aborted/max:908.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.71875 - prompt_length/max:512.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013644061982631683 - timing_s/agent_loop/generate_sequences/min:np.float64(2.052416967228055) - timing_s/agent_loop/generate_sequences/max:np.float64(7.514533661305904) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.487358166057675) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.514533661305904) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:430 - timing_s/agent_loop/slowest/response_length:887 - timing_s/gen:13.315912859514356 - timing_s/reward:0.061586445197463036 - timing_s/old_log_prob:2.41913210041821 - timing_s/adv:0.12963471002876759 - timing_s/update_actor:9.890127301216125 - timing_s/step:25.907122230157256 - timing_s/testing:43.71618625335395 - timing_s/save_checkpoint:6.669953780248761 - timing_s/stop_profile:3.4768134355545044e-05 - timing_per_token_ms/update_actor:0.047417151944922624 - timing_per_token_ms/adv:0.0006215196787218514 - timing_per_token_ms/gen:0.12209153128422827 - perf/total_num_tokens:208577 - perf/time_per_step:25.907122230157256 - perf/throughput:1006.3690119024749 (TaskRunner pid=2146199) Training Progress: 70%|███████ | 70/100 [36:46<20:50, 41.67s/it] (TaskRunner pid=2146199) step:71 - global_seqlen/min:21049 - global_seqlen/max:32648 - global_seqlen/minmax_diff:11599 - global_seqlen/balanced_min:27121 - global_seqlen/balanced_max:27129 - global_seqlen/mean:27126.0 - actor/entropy:0.05311896279454231 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7945970892906189 - training/rollout_probs_diff_mean:0.0022116340696811676 - training/rollout_probs_diff_std:0.01325868908315897 - training/rollout_actor_probs_pearson_corr:0.9943668842315674 - rollout_corr/rollout_is_mean:0.9999369382858276 - rollout_corr/rollout_is_ratio_fraction_high:7.694938540225849e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002051983610726893 - rollout_corr/rollout_is_max:4.062183856964111 - rollout_corr/rollout_is_min:0.015028823167085648 - rollout_corr/rollout_is_std:0.03132810443639755 - rollout_corr/rollout_is_eff_sample_size:0.9990195122189405 - rollout_corr/rollout_is_seq_mean:0.9999502897262573 - rollout_corr/rollout_is_seq_std:0.001774327247403562 - rollout_corr/rollout_is_seq_max:1.007266879081726 - rollout_corr/rollout_is_seq_min:0.9942275285720825 - rollout_corr/rollout_is_seq_max_deviation:0.007266879081726074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.059027730487287) - rollout_corr/training_log_ppl:np.float64(0.05673689844479668) - rollout_corr/kl:np.float64(0.0009565068957515521) - rollout_corr/k3_kl:np.float64(0.0007575751719830492) - rollout_corr/rollout_ppl:np.float64(1.0579949030652642) - rollout_corr/rollout_log_ppl:np.float64(0.055780390748623176) - rollout_corr/log_ppl_diff:np.float64(0.0009565076961735031) - rollout_corr/log_ppl_abs_diff:np.float64(0.001629676780794398) - rollout_corr/log_ppl_diff_max:np.float64(0.011695615947246552) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037643127143383026) - rollout_corr/ppl_ratio:np.float64(1.0009593770373613) - rollout_corr/chi2_token:np.float64(0.0011274551507085562) - rollout_corr/chi2_seq:np.float64(0.463244428858161) - actor/pg_loss:np.float64(-6.601936183869839e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00035762523793891887) - actor/ppo_kl:np.float64(0.00024033917571308905) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17101709358394146) - perf/mfu/actor:np.float64(0.079708120775552) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(105.6991958618164) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:2 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013369955122470856 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013369955122470856 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:456.875 - response_length/max:1142.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:456.875 - response_length_non_aborted/max:1142.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.8125 - prompt_length/max:482.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.7909637689590454e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1379325203597546) - timing_s/agent_loop/generate_sequences/max:np.float64(8.938765021041036) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.965891423125868) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.938765021041036) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:1142 - timing_s/gen:14.64490002579987 - timing_s/reward:0.06017265282571316 - timing_s/old_log_prob:2.492539495229721 - timing_s/adv:0.14441856928169727 - timing_s/update_actor:9.828959498554468 - timing_s/step:27.27580350264907 - timing_s/stop_profile:2.894364297389984e-05 - timing_per_token_ms/update_actor:0.04529307444220705 - timing_per_token_ms/adv:0.0006654988262262095 - timing_per_token_ms/gen:0.12521289351744075 - perf/total_num_tokens:217008 - perf/time_per_step:27.27580350264907 - perf/throughput:994.5078243933485 (TaskRunner pid=2146199) Training Progress: 71%|███████ | 71/100 [37:13<18:03, 37.36s/it] (TaskRunner pid=2146199) step:72 - global_seqlen/min:25515 - global_seqlen/max:31708 - global_seqlen/minmax_diff:6193 - global_seqlen/balanced_min:28279 - global_seqlen/balanced_max:28287 - global_seqlen/mean:28284.875 - actor/entropy:0.04064812511205673 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3175737261772156 - training/rollout_probs_diff_mean:0.0017629670910537243 - training/rollout_probs_diff_std:0.010362716391682625 - training/rollout_actor_probs_pearson_corr:0.9954261183738708 - rollout_corr/rollout_is_mean:0.9999600052833557 - rollout_corr/rollout_is_ratio_fraction_high:6.426787149393931e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.426787149393931e-05 - rollout_corr/rollout_is_max:4.470643520355225 - rollout_corr/rollout_is_min:0.2894100844860077 - rollout_corr/rollout_is_std:0.026937464252114296 - rollout_corr/rollout_is_eff_sample_size:0.9992746611342851 - rollout_corr/rollout_is_seq_mean:0.999939501285553 - rollout_corr/rollout_is_seq_std:0.0016795923002064228 - rollout_corr/rollout_is_seq_max:1.0073479413986206 - rollout_corr/rollout_is_seq_min:0.994812548160553 - rollout_corr/rollout_is_seq_max_deviation:0.0073479413986206055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0445621483959258) - rollout_corr/training_log_ppl:np.float64(0.043173339750865125) - rollout_corr/kl:np.float64(0.0006804254709642521) - rollout_corr/k3_kl:np.float64(0.0005955293768487024) - rollout_corr/rollout_ppl:np.float64(1.0438406970351934) - rollout_corr/rollout_log_ppl:np.float64(0.04249291388714482) - rollout_corr/log_ppl_diff:np.float64(0.000680425863720302) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013459154388328898) - rollout_corr/log_ppl_diff_max:np.float64(0.009998984634876251) - rollout_corr/log_ppl_diff_min:np.float64(-0.00490541011095047) - rollout_corr/ppl_ratio:np.float64(1.0006824743468314) - rollout_corr/chi2_token:np.float64(0.0010534131433814764) - rollout_corr/chi2_seq:np.float64(0.4315157514065504) - actor/pg_loss:np.float64(9.28947702050209e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00013625994870380964) - actor/ppo_kl:np.float64(0.00017442086651797695) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08401121105998755) - perf/mfu/actor:np.float64(0.08172431157053298) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.2209701538086) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:2 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0087665393948555 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0087665393948555 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:486.24609375 - response_length/max:1185.0 - response_length/min:207.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:486.24609375 - response_length_non_aborted/max:1185.0 - response_length_non_aborted/min:207.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.65625 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6345252990722656e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.631248328834772) - timing_s/agent_loop/generate_sequences/max:np.float64(8.524673823267221) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.798065633855003) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.524673823267221) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:1185 - timing_s/gen:14.341018304228783 - timing_s/reward:0.06013658083975315 - timing_s/old_log_prob:2.4432556722313166 - timing_s/adv:0.13638384267687798 - timing_s/update_actor:9.870010945945978 - timing_s/step:26.938693527132273 - timing_s/stop_profile:0.00011367537081241608 - timing_per_token_ms/update_actor:0.043618766858373854 - timing_per_token_ms/adv:0.0006027242593297565 - timing_per_token_ms/gen:0.1152083347731648 - perf/total_num_tokens:226279 - perf/time_per_step:26.938693527132273 - perf/throughput:1049.972040088428 (TaskRunner pid=2146199) Training Progress: 72%|███████▏ | 72/100 [37:40<15:58, 34.24s/it] (TaskRunner pid=2146199) step:73 - global_seqlen/min:19762 - global_seqlen/max:33822 - global_seqlen/minmax_diff:14060 - global_seqlen/balanced_min:26127 - global_seqlen/balanced_max:26131 - global_seqlen/mean:26129.0 - actor/entropy:0.043429240584373474 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3298608958721161 - training/rollout_probs_diff_mean:0.0020465648267418146 - training/rollout_probs_diff_std:0.011387963779270649 - training/rollout_actor_probs_pearson_corr:0.9948678016662598 - rollout_corr/rollout_is_mean:0.9998932480812073 - rollout_corr/rollout_is_ratio_fraction_high:1.8364798961556517e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001285535836359486 - rollout_corr/rollout_is_max:3.219790458679199 - rollout_corr/rollout_is_min:0.12566468119621277 - rollout_corr/rollout_is_std:0.02950972504913807 - rollout_corr/rollout_is_eff_sample_size:0.9991298148123003 - rollout_corr/rollout_is_seq_mean:0.9998681545257568 - rollout_corr/rollout_is_seq_std:0.002070891670882702 - rollout_corr/rollout_is_seq_max:1.009402871131897 - rollout_corr/rollout_is_seq_min:0.9930364489555359 - rollout_corr/rollout_is_seq_max_deviation:0.009402871131896973 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0516510414890945) - rollout_corr/training_log_ppl:np.float64(0.04964522701811802) - rollout_corr/kl:np.float64(0.000772072547538194) - rollout_corr/k3_kl:np.float64(0.0006971475974069108) - rollout_corr/rollout_ppl:np.float64(1.0508179422467947) - rollout_corr/rollout_log_ppl:np.float64(0.04887315421456151) - rollout_corr/log_ppl_diff:np.float64(0.0007720728035565116) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017214380141012953) - rollout_corr/log_ppl_diff_max:np.float64(0.01277586817741394) - rollout_corr/log_ppl_diff_min:np.float64(-0.008681826293468475) - rollout_corr/ppl_ratio:np.float64(1.000775532098487) - rollout_corr/chi2_token:np.float64(0.001193895237520337) - rollout_corr/chi2_seq:np.float64(0.5024361328687519) - actor/pg_loss:np.float64(-5.034718196839094e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022883839392306982) - actor/ppo_kl:np.float64(6.373504457801715e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.09376490861177444) - perf/mfu/actor:np.float64(0.07697481577025608) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.0593490600586) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:2 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:6.42767918179743e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:6.42767918179743e-05 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:425.40625 - response_length/max:844.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:425.40625 - response_length_non_aborted/max:844.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.125 - prompt_length/max:494.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013961084187030792 - timing_s/agent_loop/generate_sequences/min:np.float64(1.131607135757804) - timing_s/agent_loop/generate_sequences/max:np.float64(7.315270917490125) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.17285737318889) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.315270917490125) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:446 - timing_s/agent_loop/slowest/response_length:835 - timing_s/gen:13.126717433333397 - timing_s/reward:0.06403303891420364 - timing_s/old_log_prob:2.3909898437559605 - timing_s/adv:0.11876309476792812 - timing_s/update_actor:9.792982151731849 - timing_s/step:25.580546267330647 - timing_s/stop_profile:0.0001176781952381134 - timing_per_token_ms/update_actor:0.04684920084834786 - timing_per_token_ms/adv:0.0005681574819545722 - timing_per_token_ms/gen:0.12053475935992615 - perf/total_num_tokens:209032 - perf/time_per_step:25.580546267330647 - perf/throughput:1021.440266636127 (TaskRunner pid=2146199) Training Progress: 73%|███████▎ | 73/100 [38:06<14:14, 31.66s/it] (TaskRunner pid=2146199) step:74 - global_seqlen/min:19231 - global_seqlen/max:31259 - global_seqlen/minmax_diff:12028 - global_seqlen/balanced_min:25833 - global_seqlen/balanced_max:25836 - global_seqlen/mean:25834.875 - actor/entropy:0.04889968782663345 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.32331767678260803 - training/rollout_probs_diff_mean:0.0021929459180682898 - training/rollout_probs_diff_std:0.011764825321733952 - training/rollout_actor_probs_pearson_corr:0.9950502514839172 - rollout_corr/rollout_is_mean:1.0000125169754028 - rollout_corr/rollout_is_ratio_fraction_high:1.8561313481768593e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.568393680732697e-05 - rollout_corr/rollout_is_max:2.6149532794952393 - rollout_corr/rollout_is_min:0.3683270215988159 - rollout_corr/rollout_is_std:0.03110661916434765 - rollout_corr/rollout_is_eff_sample_size:0.9990334325625089 - rollout_corr/rollout_is_seq_mean:1.0000407695770264 - rollout_corr/rollout_is_seq_std:0.00158528087195009 - rollout_corr/rollout_is_seq_max:1.0063660144805908 - rollout_corr/rollout_is_seq_min:0.9958335757255554 - rollout_corr/rollout_is_seq_max_deviation:0.00636601448059082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0508666662499309) - rollout_corr/training_log_ppl:np.float64(0.04927112740006123) - rollout_corr/kl:np.float64(0.0006713545835967949) - rollout_corr/k3_kl:np.float64(0.0006095749308947518) - rollout_corr/rollout_ppl:np.float64(1.0501518324017525) - rollout_corr/rollout_log_ppl:np.float64(0.04859977240084845) - rollout_corr/log_ppl_diff:np.float64(0.000671354999212781) - rollout_corr/log_ppl_abs_diff:np.float64(0.001417454841430299) - rollout_corr/log_ppl_diff_max:np.float64(0.010202184319496155) - rollout_corr/log_ppl_diff_min:np.float64(-0.005531676113605499) - rollout_corr/ppl_ratio:np.float64(1.0006733126938343) - rollout_corr/chi2_token:np.float64(0.0011140683200210333) - rollout_corr/chi2_seq:np.float64(0.4470801525749266) - actor/pg_loss:np.float64(2.268119715154171e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021880191070522415) - actor/ppo_kl:np.float64(0.00022397367132942492) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14421727322041988) - perf/mfu/actor:np.float64(0.07530633279305796) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.15338516235352) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:2 - critic/score/mean:0.83203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011658824514597654 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011658824514597654 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:420.90234375 - response_length/max:818.0 - response_length/min:181.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:420.90234375 - response_length_non_aborted/max:818.0 - response_length_non_aborted/min:181.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:386.4375 - prompt_length/max:481.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.980238974094391e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.145769277587533) - timing_s/agent_loop/generate_sequences/max:np.float64(7.110788274556398) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.553270738790161) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.110788274556398) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:406 - timing_s/agent_loop/slowest/response_length:806 - timing_s/gen:12.860746312886477 - timing_s/reward:0.06258940324187279 - timing_s/old_log_prob:2.351754317060113 - timing_s/adv:0.13163398392498493 - timing_s/update_actor:9.94028976559639 - timing_s/step:25.43317300453782 - timing_s/stop_profile:8.191727101802826e-05 - timing_per_token_ms/update_actor:0.048095306081393806 - timing_per_token_ms/adv:0.0006369006233095038 - timing_per_token_ms/gen:0.11935616665169212 - perf/total_num_tokens:206679 - perf/time_per_step:25.43317300453782 - perf/throughput:1015.7944113143297 (TaskRunner pid=2146199) Training Progress: 74%|███████▍ | 74/100 [38:31<12:54, 29.81s/it] (TaskRunner pid=2146199) step:75 - global_seqlen/min:21131 - global_seqlen/max:32161 - global_seqlen/minmax_diff:11030 - global_seqlen/balanced_min:26273 - global_seqlen/balanced_max:26274 - global_seqlen/mean:26273.5 - actor/entropy:0.043091464787721634 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6083944439888 - training/rollout_probs_diff_mean:0.0019931308925151825 - training/rollout_probs_diff_std:0.011996152810752392 - training/rollout_actor_probs_pearson_corr:0.994069516658783 - rollout_corr/rollout_is_mean:1.0000470876693726 - rollout_corr/rollout_is_ratio_fraction_high:5.489880277309567e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012809720647055656 - rollout_corr/rollout_is_max:3.944427013397217 - rollout_corr/rollout_is_min:0.022593684494495392 - rollout_corr/rollout_is_std:0.02930501103401184 - rollout_corr/rollout_is_eff_sample_size:0.9991420721595567 - rollout_corr/rollout_is_seq_mean:1.0000299215316772 - rollout_corr/rollout_is_seq_std:0.0017850594595074654 - rollout_corr/rollout_is_seq_max:1.0075738430023193 - rollout_corr/rollout_is_seq_min:0.9937546849250793 - rollout_corr/rollout_is_seq_max_deviation:0.007573843002319336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0484930062666535) - rollout_corr/training_log_ppl:np.float64(0.046609767647169065) - rollout_corr/kl:np.float64(0.001400476434017861) - rollout_corr/k3_kl:np.float64(0.001197970774150292) - rollout_corr/rollout_ppl:np.float64(1.046985640656203) - rollout_corr/rollout_log_ppl:np.float64(0.04520929075533786) - rollout_corr/log_ppl_diff:np.float64(0.0014004768918312038) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021917948970440193) - rollout_corr/log_ppl_diff_max:np.float64(0.0172557532787323) - rollout_corr/log_ppl_diff_min:np.float64(-0.004435785114765167) - rollout_corr/ppl_ratio:np.float64(1.0014077180530876) - rollout_corr/chi2_token:np.float64(0.0012966087087988853) - rollout_corr/chi2_seq:np.float64(0.4293694163206965) - actor/pg_loss:np.float64(-1.452502328902483e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000278825466921262) - actor/ppo_kl:np.float64(0.0008897074202973698) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16844428330659866) - perf/mfu/actor:np.float64(0.07704986430708059) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.06644821166992) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:2 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0003488361544441432 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0003488361544441432 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:426.921875 - response_length/max:931.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:426.921875 - response_length_non_aborted/max:931.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.125 - prompt_length/max:493.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.071400225162506e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.007368851453066) - timing_s/agent_loop/generate_sequences/max:np.float64(7.569498650729656) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.56827202666318) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.569498650729656) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:931 - timing_s/gen:13.064472615718842 - timing_s/reward:0.06342996843159199 - timing_s/old_log_prob:2.3551226314157248 - timing_s/adv:0.12257699482142925 - timing_s/update_actor:9.887287450954318 - timing_s/step:25.578531136736274 - timing_s/stop_profile:0.00011863000690937042 - timing_per_token_ms/update_actor:0.04704020900790872 - timing_per_token_ms/adv:0.0005831778922746743 - timing_per_token_ms/gen:0.11953731852028365 - perf/total_num_tokens:210188 - perf/time_per_step:25.578531136736274 - perf/throughput:1027.1700067352813 (TaskRunner pid=2146199) Training Progress: 75%|███████▌ | 75/100 [38:57<11:53, 28.55s/it] (TaskRunner pid=2146199) step:76 - global_seqlen/min:18730 - global_seqlen/max:27703 - global_seqlen/minmax_diff:8973 - global_seqlen/balanced_min:23824 - global_seqlen/balanced_max:23838 - global_seqlen/mean:23834.125 - actor/entropy:0.05598592758178711 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417161464691162 - training/rollout_probs_diff_mean:0.0025872846599668264 - training/rollout_probs_diff_std:0.01490830723196268 - training/rollout_actor_probs_pearson_corr:0.9929959774017334 - rollout_corr/rollout_is_mean:0.9999797940254211 - rollout_corr/rollout_is_ratio_fraction_high:0.00014821559307165444 - rollout_corr/rollout_is_ratio_fraction_low:0.0002964311861433089 - rollout_corr/rollout_is_max:5.697662830352783 - rollout_corr/rollout_is_min:0.09278460592031479 - rollout_corr/rollout_is_std:0.03521384671330452 - rollout_corr/rollout_is_eff_sample_size:0.9987612828747 - rollout_corr/rollout_is_seq_mean:1.000093936920166 - rollout_corr/rollout_is_seq_std:0.0033086398616433144 - rollout_corr/rollout_is_seq_max:1.0240411758422852 - rollout_corr/rollout_is_seq_min:0.991833508014679 - rollout_corr/rollout_is_seq_max_deviation:0.024041175842285156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0572347743436694) - rollout_corr/training_log_ppl:np.float64(0.05463178321770101) - rollout_corr/kl:np.float64(0.0010483782867591884) - rollout_corr/k3_kl:np.float64(0.0011095134246019711) - rollout_corr/rollout_ppl:np.float64(1.056108265183866) - rollout_corr/rollout_log_ppl:np.float64(0.053583404513119604) - rollout_corr/log_ppl_diff:np.float64(0.0010483787045814097) - rollout_corr/log_ppl_abs_diff:np.float64(0.002085324020299595) - rollout_corr/log_ppl_diff_max:np.float64(0.01443926990032196) - rollout_corr/log_ppl_diff_min:np.float64(-0.00988314300775528) - rollout_corr/ppl_ratio:np.float64(1.001053398475051) - rollout_corr/chi2_token:np.float64(0.0023742960765957832) - rollout_corr/chi2_seq:np.float64(0.7714955767150968) - actor/pg_loss:np.float64(3.385043237358332e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.600557637488237e-05) - actor/ppo_kl:np.float64(0.0002009261775430815) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08636107575148344) - perf/mfu/actor:np.float64(0.06980476315940443) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.08797836303711) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:2 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0019492997089400887 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0019492997089400887 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:368.97265625 - response_length/max:967.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:368.97265625 - response_length_non_aborted/max:967.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:375.84375 - prompt_length/max:471.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.778406143188477e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.122740125283599) - timing_s/agent_loop/generate_sequences/max:np.float64(6.961374973878264) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6424222300047404) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.961374973878264) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:899 - timing_s/gen:12.683735959231853 - timing_s/reward:0.06086129881441593 - timing_s/old_log_prob:2.336681555956602 - timing_s/adv:0.14603652246296406 - timing_s/update_actor:9.831658877432346 - timing_s/step:25.146581349894404 - timing_s/stop_profile:0.00012607499957084656 - timing_per_token_ms/update_actor:0.05156293170733322 - timing_per_token_ms/adv:0.0007659003763666805 - timing_per_token_ms/gen:0.1342805293332612 - perf/total_num_tokens:190673 - perf/time_per_step:25.146581349894404 - perf/throughput:947.8077623501727 (TaskRunner pid=2146199) Training Progress: 76%|███████▌ | 76/100 [39:22<11:01, 27.55s/it] (TaskRunner pid=2146199) step:77 - global_seqlen/min:22806 - global_seqlen/max:31182 - global_seqlen/minmax_diff:8376 - global_seqlen/balanced_min:27635 - global_seqlen/balanced_max:27636 - global_seqlen/mean:27635.5 - actor/entropy:0.04000265151262283 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6044727563858032 - training/rollout_probs_diff_mean:0.0019838158041238785 - training/rollout_probs_diff_std:0.012652820907533169 - training/rollout_actor_probs_pearson_corr:0.993073582649231 - rollout_corr/rollout_is_mean:1.0000358819961548 - rollout_corr/rollout_is_ratio_fraction_high:5.142798545421101e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002999965799972415 - rollout_corr/rollout_is_max:2.9354820251464844 - rollout_corr/rollout_is_min:0.04595488682389259 - rollout_corr/rollout_is_std:0.030058063566684723 - rollout_corr/rollout_is_eff_sample_size:0.9990973283467841 - rollout_corr/rollout_is_seq_mean:1.000033974647522 - rollout_corr/rollout_is_seq_std:0.0019012537086382508 - rollout_corr/rollout_is_seq_max:1.009982943534851 - rollout_corr/rollout_is_seq_min:0.991452157497406 - rollout_corr/rollout_is_seq_max_deviation:0.009982943534851074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.04476328054443) - rollout_corr/training_log_ppl:np.float64(0.04319791453963262) - rollout_corr/kl:np.float64(0.0007020482703978814) - rollout_corr/k3_kl:np.float64(0.000983930625448437) - rollout_corr/rollout_ppl:np.float64(1.0440163351595402) - rollout_corr/rollout_log_ppl:np.float64(0.042495865676755784) - rollout_corr/log_ppl_diff:np.float64(0.0007020488628768362) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016647392440063413) - rollout_corr/log_ppl_diff_max:np.float64(0.017813581973314285) - rollout_corr/log_ppl_diff_min:np.float64(-0.008897464722394943) - rollout_corr/ppl_ratio:np.float64(1.0007062209770083) - rollout_corr/chi2_token:np.float64(0.0027417431119829416) - rollout_corr/chi2_seq:np.float64(1.2524716129992157) - actor/pg_loss:np.float64(0.00010078214108943939) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003275459280303039) - actor/ppo_kl:np.float64(2.000482497521716e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15676945121958852) - perf/mfu/actor:np.float64(0.08082957892246295) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.05048370361328) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:2 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005196369020268321 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005196369020268321 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:455.734375 - response_length/max:767.0 - response_length/min:185.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:455.734375 - response_length_non_aborted/max:767.0 - response_length_non_aborted/min:185.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.875 - prompt_length/max:510.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001513119786977768 - timing_s/agent_loop/generate_sequences/min:np.float64(2.195318879559636) - timing_s/agent_loop/generate_sequences/max:np.float64(7.278248289600015) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.894568861309381) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.278248289600015) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:767 - timing_s/gen:12.90355608984828 - timing_s/reward:0.06591372936964035 - timing_s/old_log_prob:2.3179823830723763 - timing_s/adv:0.12202087230980396 - timing_s/update_actor:9.944258518517017 - timing_s/step:25.445413818582892 - timing_s/stop_profile:0.00011137686669826508 - timing_per_token_ms/update_actor:0.044979548581159275 - timing_per_token_ms/adv:0.0005519208640598323 - timing_per_token_ms/gen:0.11060064533418144 - perf/total_num_tokens:221084 - perf/time_per_step:25.445413818582892 - perf/throughput:1086.0699769723406 (TaskRunner pid=2146199) Training Progress: 77%|███████▋ | 77/100 [39:48<10:19, 26.94s/it] (TaskRunner pid=2146199) step:78 - global_seqlen/min:24161 - global_seqlen/max:28614 - global_seqlen/minmax_diff:4453 - global_seqlen/balanced_min:26166 - global_seqlen/balanced_max:26169 - global_seqlen/mean:26167.5 - actor/entropy:0.039375629276037216 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4292469620704651 - training/rollout_probs_diff_mean:0.002033015713095665 - training/rollout_probs_diff_std:0.011736562475562096 - training/rollout_actor_probs_pearson_corr:0.9937998652458191 - rollout_corr/rollout_is_mean:0.9998176097869873 - rollout_corr/rollout_is_ratio_fraction_high:3.8510417653014883e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010590364399831742 - rollout_corr/rollout_is_max:2.463862657546997 - rollout_corr/rollout_is_min:0.1985011249780655 - rollout_corr/rollout_is_std:0.030386436730623245 - rollout_corr/rollout_is_eff_sample_size:0.9990773377257155 - rollout_corr/rollout_is_seq_mean:0.9997347593307495 - rollout_corr/rollout_is_seq_std:0.0018866725731641054 - rollout_corr/rollout_is_seq_max:1.0046619176864624 - rollout_corr/rollout_is_seq_min:0.992652416229248 - rollout_corr/rollout_is_seq_max_deviation:0.007347583770751953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.048908544704318) - rollout_corr/training_log_ppl:np.float64(0.04742776556486206) - rollout_corr/kl:np.float64(0.006579041265613483) - rollout_corr/k3_kl:np.float64(0.005242740941159951) - rollout_corr/rollout_ppl:np.float64(1.0419986243359745) - rollout_corr/rollout_log_ppl:np.float64(0.04084872375096893) - rollout_corr/log_ppl_diff:np.float64(0.006579041817531106) - rollout_corr/log_ppl_abs_diff:np.float64(0.00686174491966085) - rollout_corr/log_ppl_diff_max:np.float64(0.04815289378166199) - rollout_corr/log_ppl_diff_min:np.float64(-0.006678592413663864) - rollout_corr/ppl_ratio:np.float64(1.0066258441656828) - rollout_corr/chi2_token:np.float64(0.0026945623103529215) - rollout_corr/chi2_seq:np.float64(-0.540671911323443) - actor/pg_loss:np.float64(0.0004199024988338351) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00043760340440712753) - actor/ppo_kl:np.float64(0.005793062266178595) - actor/pg_clipfrac_lower:np.float64(1.0416666555101983e-05) - actor/grad_norm:np.float64(0.23354287445545197) - perf/mfu/actor:np.float64(0.07763082635623188) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.0602035522461) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:2 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005824700347147882 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005824700347147882 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:405.734375 - response_length/max:795.0 - response_length/min:167.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:405.734375 - response_length_non_aborted/max:795.0 - response_length_non_aborted/min:167.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.0 - prompt_length/max:493.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.744265556335449e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.012940440326929) - timing_s/agent_loop/generate_sequences/max:np.float64(6.968817979097366) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.388390846201219) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.968817979097366) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:490 - timing_s/agent_loop/slowest/response_length:795 - timing_s/gen:12.832705829292536 - timing_s/reward:0.060555024072527885 - timing_s/old_log_prob:2.3405001666396856 - timing_s/adv:0.11683694832026958 - timing_s/update_actor:9.718437040224671 - timing_s/step:25.161354549229145 - timing_s/stop_profile:0.00011271797120571136 - timing_per_token_ms/update_actor:0.04642417617380659 - timing_per_token_ms/adv:0.0005581205136155039 - timing_per_token_ms/gen:0.123548213398665 - perf/total_num_tokens:209340 - perf/time_per_step:25.161354549229145 - perf/throughput:1039.9877299452337 (TaskRunner pid=2146199) Training Progress: 78%|███████▊ | 78/100 [40:13<09:41, 26.42s/it] (TaskRunner pid=2146199) step:79 - global_seqlen/min:19579 - global_seqlen/max:29853 - global_seqlen/minmax_diff:10274 - global_seqlen/balanced_min:26733 - global_seqlen/balanced_max:26736 - global_seqlen/mean:26734.375 - actor/entropy:0.039511386305093765 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.440033495426178 - training/rollout_probs_diff_mean:0.0021660877391695976 - training/rollout_probs_diff_std:0.013013363815844059 - training/rollout_actor_probs_pearson_corr:0.992502748966217 - rollout_corr/rollout_is_mean:1.0000076293945312 - rollout_corr/rollout_is_ratio_fraction_high:0.00012592306302394718 - rollout_corr/rollout_is_ratio_fraction_low:0.0003417911648284644 - rollout_corr/rollout_is_max:3.7972800731658936 - rollout_corr/rollout_is_min:0.06315048784017563 - rollout_corr/rollout_is_std:0.032747622579336166 - rollout_corr/rollout_is_eff_sample_size:0.9989288610095047 - rollout_corr/rollout_is_seq_mean:1.000064492225647 - rollout_corr/rollout_is_seq_std:0.0018735870253294706 - rollout_corr/rollout_is_seq_max:1.006730318069458 - rollout_corr/rollout_is_seq_min:0.9940834045410156 - rollout_corr/rollout_is_seq_max_deviation:0.006730318069458008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0477381232194602) - rollout_corr/training_log_ppl:np.float64(0.046206099918890686) - rollout_corr/kl:np.float64(0.0015904441828880067) - rollout_corr/k3_kl:np.float64(0.0017963379038548055) - rollout_corr/rollout_ppl:np.float64(1.0460580410435796) - rollout_corr/rollout_log_ppl:np.float64(0.04461565541259915) - rollout_corr/log_ppl_diff:np.float64(0.0015904445062915329) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026327134437451605) - rollout_corr/log_ppl_diff_max:np.float64(0.017743483185768127) - rollout_corr/log_ppl_diff_min:np.float64(-0.008445635437965393) - rollout_corr/ppl_ratio:np.float64(1.0015971844550222) - rollout_corr/chi2_token:np.float64(0.005007386673241854) - rollout_corr/chi2_seq:np.float64(1.1444842799101025) - actor/pg_loss:np.float64(2.9988354071974754e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00046516470229107654) - actor/ppo_kl:np.float64(0.0009375673238452009) - actor/pg_clipfrac_lower:np.float64(8.699888894625474e-06) - actor/grad_norm:np.float64(0.20429369434714317) - perf/mfu/actor:np.float64(0.0788278515974222) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.04590606689453) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024521267041563988 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024521267041563988 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:434.29296875 - response_length/max:807.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.29296875 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.15625 - prompt_length/max:494.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002267640084028244 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9041874445974827) - timing_s/agent_loop/generate_sequences/max:np.float64(7.262142397463322) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.604671682907792) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.262142397463322) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:422 - timing_s/agent_loop/slowest/response_length:787 - timing_s/gen:12.953388631343842 - timing_s/reward:0.054294243454933167 - timing_s/old_log_prob:2.4946513511240482 - timing_s/adv:0.13977370783686638 - timing_s/update_actor:9.76483209617436 - timing_s/step:25.513385014608502 - timing_s/stop_profile:0.00012118369340896606 - timing_per_token_ms/update_actor:0.045656725172060125 - timing_per_token_ms/adv:0.0006535299022179608 - timing_per_token_ms/gen:0.11650931049338312 - perf/total_num_tokens:213875 - perf/time_per_step:25.513385014608502 - perf/throughput:1047.8568400348438 (TaskRunner pid=2146199) Training Progress: 79%|███████▉ | 79/100 [40:39<09:12, 26.31s/it] (TaskRunner pid=2146199) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2146199) validation generation end (TaskRunner pid=2146199) [prompt] system (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) To calculate the volume of the unit cell for a crystal structure with orthogonal lattice parameters (α = β = γ = 90.00º), we can use the formula for the volume of a unit cell in a rectangular system: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = a \times b \times c $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given: (TaskRunner pid=2146199) - $ a = 7.82 $ Å (TaskRunner pid=2146199) - $ b = 7.82 $ Å (TaskRunner pid=2146199) - $ c = 17.88 $ Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Substituting the values: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Now multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) $$ 61.1524 \times 17.88 = 1094.31 $$ (TaskRunner pid=2146199) (TaskRunner pid=2146199) Thus, the volume of the unit cell is approximately 1094.31 ų, which matches option B. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2146577) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2146199) step:80 - global_seqlen/min:21874 - global_seqlen/max:30571 - global_seqlen/minmax_diff:8697 - global_seqlen/balanced_min:26237 - global_seqlen/balanced_max:26387 - global_seqlen/mean:26256.125 - actor/entropy:0.0638868659734726 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5829912424087524 - training/rollout_probs_diff_mean:0.0025715488009154797 - training/rollout_probs_diff_std:0.012625768780708313 - training/rollout_actor_probs_pearson_corr:0.995551586151123 - rollout_corr/rollout_is_mean:1.0000125169754028 - rollout_corr/rollout_is_ratio_fraction_high:5.3578605729853734e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020538465469144285 - rollout_corr/rollout_is_max:5.0501933097839355 - rollout_corr/rollout_is_min:0.10832185298204422 - rollout_corr/rollout_is_std:0.03361879289150238 - rollout_corr/rollout_is_eff_sample_size:0.9988709337480308 - rollout_corr/rollout_is_seq_mean:0.9999619722366333 - rollout_corr/rollout_is_seq_std:0.002099543111398816 - rollout_corr/rollout_is_seq_max:1.0109219551086426 - rollout_corr/rollout_is_seq_min:0.9940289855003357 - rollout_corr/rollout_is_seq_max_deviation:0.010921955108642578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0671630883589387) - rollout_corr/training_log_ppl:np.float64(0.0639981363692641) - rollout_corr/kl:np.float64(0.0047144165605832455) - rollout_corr/k3_kl:np.float64(0.0047130617831641075) - rollout_corr/rollout_ppl:np.float64(1.0621191035024822) - rollout_corr/rollout_log_ppl:np.float64(0.05928371967456769) - rollout_corr/log_ppl_diff:np.float64(0.0047144166946964106) - rollout_corr/log_ppl_abs_diff:np.float64(0.005513946727660368) - rollout_corr/log_ppl_diff_max:np.float64(0.04823169857263565) - rollout_corr/log_ppl_diff_min:np.float64(-0.006100788712501526) - rollout_corr/ppl_ratio:np.float64(1.0047575796488672) - rollout_corr/chi2_token:np.float64(0.0166023017372936) - rollout_corr/chi2_seq:np.float64(1.432770494138822) - actor/pg_loss:np.float64(4.944263491779566e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003110653242401895) - actor/ppo_kl:np.float64(0.004017419689404278) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10630176775157452) - perf/mfu/actor:np.float64(0.07701855460115742) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(102.9047737121582) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7506648936170213) - val-aux/sciknoweval/reward/std@16:np.float64(0.03015784969549897) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7594042553191489) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.01805225873808336) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7415212765957447) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.0313750322681755) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7504148936170213) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.029905567478505344) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7642340425531914) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.014518524350686481) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7304148936170212) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.03420878991120481) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7537978723404254) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.016541612613916686) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7705744680851063) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.015426442115850293) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7156063829787234) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.03504241495138197) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.755031914893617) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.005153746942478447) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.777904255319149) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.013765504978824115) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6993085106382977) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.02963911392080788) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7552659574468086) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0010077223043701727) - val-aux/sciknoweval/score/mean@16:np.float64(0.7506648936170213) - val-aux/sciknoweval/score/std@16:np.float64(0.03015784969549897) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7594042553191489) - val-aux/sciknoweval/score/best@2/std:np.float64(0.01805225873808336) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7415212765957447) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.0313750322681755) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7504148936170213) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.029905567478505344) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7642340425531914) - val-aux/sciknoweval/score/best@4/std:np.float64(0.014518524350686481) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7304148936170212) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.03420878991120481) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7537978723404254) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.016541612613916686) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7705744680851063) - val-aux/sciknoweval/score/best@8/std:np.float64(0.015426442115850293) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7156063829787234) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.03504241495138197) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.755031914893617) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.005153746942478447) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.777904255319149) - val-aux/sciknoweval/score/best@16/std:np.float64(0.013765504978824115) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6993085106382977) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.02963911392080788) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7552659574468086) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0010077223043701727) - val-core/sciknoweval/acc/mean@16:np.float64(0.7506648936170213) - val-aux/sciknoweval/acc/std@16:np.float64(0.03015784969549897) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7594042553191489) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.01805225873808336) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7415212765957447) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.0313750322681755) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7504148936170213) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.029905567478505344) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7642340425531914) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.014518524350686481) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7304148936170212) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.03420878991120481) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7537978723404254) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.016541612613916686) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7705744680851063) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.015426442115850293) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7156063829787234) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.03504241495138197) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.755031914893617) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.005153746942478447) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.777904255319149) - val-core/sciknoweval/acc/best@16/std:np.float64(0.013765504978824115) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6993085106382977) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.02963911392080788) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7552659574468086) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0010077223043701727) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00048220742610283196 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00048220742610283196 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:437.44140625 - response_length/max:1243.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.44140625 - response_length_non_aborted/max:1243.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.0625 - prompt_length/max:488.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015719980001449585 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1565769854933023) - timing_s/agent_loop/generate_sequences/max:np.float64(9.246533636003733) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.727572656847769) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.246533636003733) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:451 - timing_s/agent_loop/slowest/response_length:1243 - timing_s/gen:15.147192286327481 - timing_s/reward:0.062169790267944336 - timing_s/old_log_prob:2.3167877048254013 - timing_s/adv:0.11924800090491772 - timing_s/update_actor:9.813526827841997 - timing_s/step:27.548170138150454 - timing_s/testing:42.80566253513098 - timing_s/save_checkpoint:6.577129703015089 - timing_s/stop_profile:0.00010807253420352936 - timing_per_token_ms/update_actor:0.04672017875753751 - timing_per_token_ms/adv:0.000567715156486904 - timing_per_token_ms/gen:0.1352609035703664 - perf/total_num_tokens:210049 - perf/time_per_step:27.548170138150454 - perf/throughput:953.0986946983768 (TaskRunner pid=2146199) Training Progress: 80%|████████ | 80/100 [41:56<13:50, 41.52s/it] (TaskRunner pid=2146199) step:81 - global_seqlen/min:19238 - global_seqlen/max:31745 - global_seqlen/minmax_diff:12507 - global_seqlen/balanced_min:24323 - global_seqlen/balanced_max:24330 - global_seqlen/mean:24328.625 - actor/entropy:0.0454610176384449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48276346921920776 - training/rollout_probs_diff_mean:0.0022187037393450737 - training/rollout_probs_diff_std:0.012337523512542248 - training/rollout_actor_probs_pearson_corr:0.9940638542175293 - rollout_corr/rollout_is_mean:0.9997979402542114 - rollout_corr/rollout_is_ratio_fraction_high:8.195629925467074e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.000256113416980952 - rollout_corr/rollout_is_max:5.381066799163818 - rollout_corr/rollout_is_min:0.12561804056167603 - rollout_corr/rollout_is_std:0.03206803649663925 - rollout_corr/rollout_is_eff_sample_size:0.9989722811238648 - rollout_corr/rollout_is_seq_mean:0.9997701048851013 - rollout_corr/rollout_is_seq_std:0.002187376841902733 - rollout_corr/rollout_is_seq_max:1.0106942653656006 - rollout_corr/rollout_is_seq_min:0.9915710091590881 - rollout_corr/rollout_is_seq_max_deviation:0.010694265365600586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0507443067617714) - rollout_corr/training_log_ppl:np.float64(0.048668023951904615) - rollout_corr/kl:np.float64(0.001065947673820844) - rollout_corr/k3_kl:np.float64(0.0008570477055087622) - rollout_corr/rollout_ppl:np.float64(1.0496200053021312) - rollout_corr/rollout_log_ppl:np.float64(0.0476020756377693) - rollout_corr/log_ppl_diff:np.float64(0.001065948314135312) - rollout_corr/log_ppl_abs_diff:np.float64(0.001843757692768122) - rollout_corr/log_ppl_diff_max:np.float64(0.014116194099187851) - rollout_corr/log_ppl_diff_min:np.float64(-0.005465984344482422) - rollout_corr/ppl_ratio:np.float64(1.0010698894038796) - rollout_corr/chi2_token:np.float64(0.0012246051337569952) - rollout_corr/chi2_seq:np.float64(0.16860972251743078) - actor/pg_loss:np.float64(2.4694367311894894e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.842122456873767e-05) - actor/ppo_kl:np.float64(0.00015889055194495683) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10860276641324162) - perf/mfu/actor:np.float64(0.07228432106096218) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(105.0451431274414) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00021641585044562817 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00021641585044562817 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:381.30078125 - response_length/max:876.0 - response_length/min:163.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.30078125 - response_length_non_aborted/max:876.0 - response_length_non_aborted/min:163.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:378.96875 - prompt_length/max:524.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019592605531215668 - timing_s/agent_loop/generate_sequences/min:np.float64(1.974859308451414) - timing_s/agent_loop/generate_sequences/max:np.float64(7.210609203204513) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.151048744679429) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.210609203204513) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:876 - timing_s/gen:12.861397430300713 - timing_s/reward:0.057336365804076195 - timing_s/old_log_prob:2.3250329848378897 - timing_s/adv:0.12175748124718666 - timing_s/update_actor:9.648430464789271 - timing_s/step:25.115377575159073 - timing_s/stop_profile:0.00011594220995903015 - timing_per_token_ms/update_actor:0.049573447249840835 - timing_per_token_ms/adv:0.0006255875601641413 - timing_per_token_ms/gen:0.1317590631401628 - perf/total_num_tokens:194629 - perf/time_per_step:25.115377575159073 - perf/throughput:968.674467552611 (TaskRunner pid=2146199) Training Progress: 81%|████████ | 81/100 [42:21<11:35, 36.61s/it] (TaskRunner pid=2146199) step:82 - global_seqlen/min:21542 - global_seqlen/max:27173 - global_seqlen/minmax_diff:5631 - global_seqlen/balanced_min:24565 - global_seqlen/balanced_max:24569 - global_seqlen/mean:24566.125 - actor/entropy:0.05494064837694168 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36353588104248047 - training/rollout_probs_diff_mean:0.002280291635543108 - training/rollout_probs_diff_std:0.012044516392052174 - training/rollout_actor_probs_pearson_corr:0.9955347180366516 - rollout_corr/rollout_is_mean:1.0001684427261353 - rollout_corr/rollout_is_ratio_fraction_high:8.193279791157693e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016386559582315385 - rollout_corr/rollout_is_max:3.3646881580352783 - rollout_corr/rollout_is_min:0.2920839190483093 - rollout_corr/rollout_is_std:0.03323004022240639 - rollout_corr/rollout_is_eff_sample_size:0.9988973391818019 - rollout_corr/rollout_is_seq_mean:1.0001329183578491 - rollout_corr/rollout_is_seq_std:0.0018537776777520776 - rollout_corr/rollout_is_seq_max:1.0058927536010742 - rollout_corr/rollout_is_seq_min:0.9943650364875793 - rollout_corr/rollout_is_seq_max_deviation:0.005892753601074219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0548055358231068) - rollout_corr/training_log_ppl:np.float64(0.052793717570239096) - rollout_corr/kl:np.float64(0.0005393300840426107) - rollout_corr/k3_kl:np.float64(0.0006593122632807535) - rollout_corr/rollout_ppl:np.float64(1.054230005480349) - rollout_corr/rollout_log_ppl:np.float64(0.05225438714660413) - rollout_corr/log_ppl_diff:np.float64(0.000539330423634965) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016479908917972352) - rollout_corr/log_ppl_diff_max:np.float64(0.010518059134483337) - rollout_corr/log_ppl_diff_min:np.float64(-0.005966216325759888) - rollout_corr/ppl_ratio:np.float64(1.0005417650099844) - rollout_corr/chi2_token:np.float64(0.0016422346234321594) - rollout_corr/chi2_seq:np.float64(0.77241958421655) - actor/pg_loss:np.float64(-2.377619966864586e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.7041629841260146e-05) - actor/ppo_kl:np.float64(0.00011113984009725186) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.04635968059301376) - perf/mfu/actor:np.float64(0.07218304547408795) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(104.85897064208984) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006196167669259012 - critic/advantages/max:0.75 - critic/advantages/min:-0.25 - critic/returns/mean:0.0006196167669259012 - critic/returns/max:0.75 - critic/returns/min:-0.25 - response_length/mean:381.41015625 - response_length/max:940.0 - response_length/min:176.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:381.41015625 - response_length_non_aborted/max:940.0 - response_length_non_aborted/min:176.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:386.28125 - prompt_length/max:510.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011840462684631348 - timing_s/agent_loop/generate_sequences/min:np.float64(2.11877216771245) - timing_s/agent_loop/generate_sequences/max:np.float64(7.320236181840301) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.159982757504622) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.320236181840301) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:940 - timing_s/gen:13.001283392310143 - timing_s/reward:0.06118536740541458 - timing_s/old_log_prob:2.2628217302262783 - timing_s/adv:0.12807857058942318 - timing_s/update_actor:9.75557561405003 - timing_s/step:25.297244949266315 - timing_s/stop_profile:0.00014123320579528809 - timing_per_token_ms/update_actor:0.049639369324883505 - timing_per_token_ms/adv:0.0006517031613116801 - timing_per_token_ms/gen:0.1331539352557854 - perf/total_num_tokens:196529 - perf/time_per_step:25.297244949266315 - perf/throughput:971.0988310888171 (TaskRunner pid=2146199) Training Progress: 82%|████████▏ | 82/100 [42:47<09:58, 33.23s/it] (TaskRunner pid=2146199) step:83 - global_seqlen/min:23474 - global_seqlen/max:30918 - global_seqlen/minmax_diff:7444 - global_seqlen/balanced_min:25578 - global_seqlen/balanced_max:27547 - global_seqlen/mean:25825.375 - actor/entropy:0.055972497910261154 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4917345941066742 - training/rollout_probs_diff_mean:0.002354572294279933 - training/rollout_probs_diff_std:0.012616433203220367 - training/rollout_actor_probs_pearson_corr:0.9948427677154541 - rollout_corr/rollout_is_mean:0.9999601244926453 - rollout_corr/rollout_is_ratio_fraction_high:3.758444654522464e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001221494603669271 - rollout_corr/rollout_is_max:2.4553167819976807 - rollout_corr/rollout_is_min:0.15462268888950348 - rollout_corr/rollout_is_std:0.032183993607759476 - rollout_corr/rollout_is_eff_sample_size:0.9989653812370077 - rollout_corr/rollout_is_seq_mean:1.0000340938568115 - rollout_corr/rollout_is_seq_std:0.0017893940676003695 - rollout_corr/rollout_is_seq_max:1.0072091817855835 - rollout_corr/rollout_is_seq_min:0.995140016078949 - rollout_corr/rollout_is_seq_max_deviation:0.007209181785583496 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0560307195410132) - rollout_corr/training_log_ppl:np.float64(0.0537358048659371) - rollout_corr/kl:np.float64(0.0006470137497549544) - rollout_corr/k3_kl:np.float64(0.0007146234917954075) - rollout_corr/rollout_ppl:np.float64(1.055338675621897) - rollout_corr/rollout_log_ppl:np.float64(0.05308879098993202) - rollout_corr/log_ppl_diff:np.float64(0.0006470138760050759) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016727128022466786) - rollout_corr/log_ppl_diff_max:np.float64(0.008520536124706268) - rollout_corr/log_ppl_diff_min:np.float64(-0.0060235485434532166) - rollout_corr/ppl_ratio:np.float64(1.000649485271424) - rollout_corr/chi2_token:np.float64(0.0015931350644677877) - rollout_corr/chi2_seq:np.float64(0.7568402329925448) - actor/pg_loss:np.float64(5.573965609073639e-07) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003066948038394912) - actor/ppo_kl:np.float64(7.250778660417012e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14149587228894234) - perf/mfu/actor:np.float64(0.07499097891987316) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.13691711425781) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002314967103302479 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002314967103302479 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:415.73046875 - response_length/max:3235.0 - response_length/min:183.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:415.73046875 - response_length_non_aborted/max:3235.0 - response_length_non_aborted/min:183.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.3125 - prompt_length/max:492.0 - prompt_length/min:267.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014868751168251038 - timing_s/agent_loop/generate_sequences/min:np.float64(2.101010534912348) - timing_s/agent_loop/generate_sequences/max:np.float64(17.99409211240709) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.248143444368907) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.99409211240709) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:3235 - timing_s/gen:23.69477784819901 - timing_s/reward:0.05949152447283268 - timing_s/old_log_prob:2.282221419736743 - timing_s/adv:0.10885190404951572 - timing_s/update_actor:9.896594805642962 - timing_s/step:36.136148773133755 - timing_s/stop_profile:0.00011920183897018433 - timing_per_token_ms/update_actor:0.047901505813773095 - timing_per_token_ms/adv:0.0005268650699627582 - timing_per_token_ms/gen:0.2226387838443159 - perf/total_num_tokens:206603 - perf/time_per_step:36.136148773133755 - perf/throughput:714.6687147580171 (TaskRunner pid=2146199) Training Progress: 83%|████████▎ | 83/100 [43:23<09:39, 34.12s/it] (TaskRunner pid=2146199) step:84 - global_seqlen/min:20092 - global_seqlen/max:31501 - global_seqlen/minmax_diff:11409 - global_seqlen/balanced_min:26920 - global_seqlen/balanced_max:26923 - global_seqlen/mean:26921.75 - actor/entropy:0.047285933047533035 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9702563285827637 - training/rollout_probs_diff_mean:0.001998181687667966 - training/rollout_probs_diff_std:0.01176747027784586 - training/rollout_actor_probs_pearson_corr:0.9947652220726013 - rollout_corr/rollout_is_mean:0.9999570250511169 - rollout_corr/rollout_is_ratio_fraction_high:1.739039726089686e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001391231780871749 - rollout_corr/rollout_is_max:2.072388172149658 - rollout_corr/rollout_is_min:0.02932487055659294 - rollout_corr/rollout_is_std:0.028965409845113754 - rollout_corr/rollout_is_eff_sample_size:0.9991617083426337 - rollout_corr/rollout_is_seq_mean:0.9999341368675232 - rollout_corr/rollout_is_seq_std:0.0016290530329570174 - rollout_corr/rollout_is_seq_max:1.0047682523727417 - rollout_corr/rollout_is_seq_min:0.994337260723114 - rollout_corr/rollout_is_seq_max_deviation:0.005662739276885986 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0506100784987211) - rollout_corr/training_log_ppl:np.float64(0.04899685260079423) - rollout_corr/kl:np.float64(0.0008145442976399409) - rollout_corr/k3_kl:np.float64(0.0006395985890463862) - rollout_corr/rollout_ppl:np.float64(1.0497489608824253) - rollout_corr/rollout_log_ppl:np.float64(0.04818230797081924) - rollout_corr/log_ppl_diff:np.float64(0.0008145446299749892) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015282427848433144) - rollout_corr/log_ppl_diff_max:np.float64(0.027048081159591675) - rollout_corr/log_ppl_diff_min:np.float64(-0.00547654926776886) - rollout_corr/ppl_ratio:np.float64(1.0008179298602045) - rollout_corr/chi2_token:np.float64(0.0007157814688980579) - rollout_corr/chi2_seq:np.float64(0.3016389023978263) - actor/pg_loss:np.float64(-1.5655532479286194e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017637754081079038) - actor/ppo_kl:np.float64(0.00023511595035863309) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13140143640339375) - perf/mfu/actor:np.float64(0.0787529927798204) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.21229934692383) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006771385669708252 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006771385669708252 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:449.2421875 - response_length/max:916.0 - response_length/min:179.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:449.2421875 - response_length_non_aborted/max:916.0 - response_length_non_aborted/min:179.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:392.0625 - prompt_length/max:519.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.996257722377777e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1541482768952847) - timing_s/agent_loop/generate_sequences/max:np.float64(7.856898242607713) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.82572743754281) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.856898242607713) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:916 - timing_s/gen:13.612378211691976 - timing_s/reward:0.06359268724918365 - timing_s/old_log_prob:2.305539906024933 - timing_s/adv:0.10970183275640011 - timing_s/update_actor:9.885056579485536 - timing_s/step:26.060026755556464 - timing_s/stop_profile:0.00012900307774543762 - timing_per_token_ms/update_actor:0.04589716762230137 - timing_per_token_ms/adv:0.0005093550417246284 - timing_per_token_ms/gen:0.11836233076267304 - perf/total_num_tokens:215374 - perf/time_per_step:26.060026755556464 - perf/throughput:1033.0668595441791 (TaskRunner pid=2146199) Training Progress: 84%|████████▍ | 84/100 [43:49<08:27, 31.72s/it] (TaskRunner pid=2146199) step:85 - global_seqlen/min:20697 - global_seqlen/max:28926 - global_seqlen/minmax_diff:8229 - global_seqlen/balanced_min:25509 - global_seqlen/balanced_max:25513 - global_seqlen/mean:25511.25 - actor/entropy:0.0537082739174366 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5848106145858765 - training/rollout_probs_diff_mean:0.0022274358198046684 - training/rollout_probs_diff_std:0.011979250237345695 - training/rollout_actor_probs_pearson_corr:0.9954252243041992 - rollout_corr/rollout_is_mean:0.9999368786811829 - rollout_corr/rollout_is_ratio_fraction_high:8.67637136252597e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011568495392566547 - rollout_corr/rollout_is_max:2.736992835998535 - rollout_corr/rollout_is_min:0.1523718386888504 - rollout_corr/rollout_is_std:0.031525351107120514 - rollout_corr/rollout_is_eff_sample_size:0.9990067819879318 - rollout_corr/rollout_is_seq_mean:0.9999983310699463 - rollout_corr/rollout_is_seq_std:0.001847455045208335 - rollout_corr/rollout_is_seq_max:1.007943034172058 - rollout_corr/rollout_is_seq_min:0.9942744970321655 - rollout_corr/rollout_is_seq_max_deviation:0.007943034172058105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0569829531013966) - rollout_corr/training_log_ppl:np.float64(0.05484875438924064) - rollout_corr/kl:np.float64(0.000564154028900532) - rollout_corr/k3_kl:np.float64(0.0007049430767054332) - rollout_corr/rollout_ppl:np.float64(1.0563891842029989) - rollout_corr/rollout_log_ppl:np.float64(0.05428459981158085) - rollout_corr/log_ppl_diff:np.float64(0.0005641545776597923) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015242881872836733) - rollout_corr/log_ppl_diff_max:np.float64(0.011014923453330994) - rollout_corr/log_ppl_diff_min:np.float64(-0.007752552628517151) - rollout_corr/ppl_ratio:np.float64(1.00056640827097) - rollout_corr/chi2_token:np.float64(0.0018217619508504868) - rollout_corr/chi2_seq:np.float64(0.5302525763399899) - actor/pg_loss:np.float64(3.4186523407697678e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(1.8920458387583494e-05) - actor/ppo_kl:np.float64(-1.462270548002742e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.062252964824438095) - perf/mfu/actor:np.float64(0.07532675620457446) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.11812210083008) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008869179873727262 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0008869179873727262 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:405.1953125 - response_length/max:861.0 - response_length/min:173.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:405.1953125 - response_length_non_aborted/max:861.0 - response_length_non_aborted/min:173.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:392.03125 - prompt_length/max:541.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.737986743450165e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.18395003862679) - timing_s/agent_loop/generate_sequences/max:np.float64(7.46437212638557) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.468181933181768) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.46437212638557) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:541 - timing_s/agent_loop/slowest/response_length:849 - timing_s/gen:13.435584550723433 - timing_s/reward:0.0603086743503809 - timing_s/old_log_prob:2.4273746870458126 - timing_s/adv:0.1293876525014639 - timing_s/update_actor:9.722496332600713 - timing_s/step:25.865453267470002 - timing_s/stop_profile:8.542463183403015e-05 - timing_per_token_ms/update_actor:0.04763827886031022 - timing_per_token_ms/adv:0.0006339735043434949 - timing_per_token_ms/gen:0.12952457872094317 - perf/total_num_tokens:204090 - perf/time_per_step:25.865453267470002 - perf/throughput:986.3059323257455 (TaskRunner pid=2146199) Training Progress: 85%|████████▌ | 85/100 [44:15<07:29, 29.97s/it] (TaskRunner pid=2146199) step:86 - global_seqlen/min:21728 - global_seqlen/max:27725 - global_seqlen/minmax_diff:5997 - global_seqlen/balanced_min:24688 - global_seqlen/balanced_max:24691 - global_seqlen/mean:24688.625 - actor/entropy:0.061135634779930115 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3990497589111328 - training/rollout_probs_diff_mean:0.0024492167867720127 - training/rollout_probs_diff_std:0.011708705686032772 - training/rollout_actor_probs_pearson_corr:0.9958972334861755 - rollout_corr/rollout_is_mean:1.000069499015808 - rollout_corr/rollout_is_ratio_fraction_high:4.9842001317301765e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.974720210768282e-05 - rollout_corr/rollout_is_max:2.7921876907348633 - rollout_corr/rollout_is_min:0.047019168734550476 - rollout_corr/rollout_is_std:0.03246060386300087 - rollout_corr/rollout_is_eff_sample_size:0.9989476561025407 - rollout_corr/rollout_is_seq_mean:1.0000441074371338 - rollout_corr/rollout_is_seq_std:0.0018616501474753022 - rollout_corr/rollout_is_seq_max:1.0080980062484741 - rollout_corr/rollout_is_seq_min:0.9922611713409424 - rollout_corr/rollout_is_seq_max_deviation:0.008098006248474121 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0576213751919568) - rollout_corr/training_log_ppl:np.float64(0.055652573246334214) - rollout_corr/kl:np.float64(0.0005706517937165145) - rollout_corr/k3_kl:np.float64(0.0005937302741383377) - rollout_corr/rollout_ppl:np.float64(1.0570157296024263) - rollout_corr/rollout_log_ppl:np.float64(0.05508192085471819) - rollout_corr/log_ppl_diff:np.float64(0.0005706523916160222) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015123811426747125) - rollout_corr/log_ppl_diff_max:np.float64(0.008686698973178864) - rollout_corr/log_ppl_diff_min:np.float64(-0.006528489291667938) - rollout_corr/ppl_ratio:np.float64(1.0005727738607675) - rollout_corr/chi2_token:np.float64(0.0013023677747696638) - rollout_corr/chi2_seq:np.float64(0.8290172575507313) - actor/pg_loss:np.float64(-4.644633736461401e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020940781405442976) - actor/ppo_kl:np.float64(7.144228803923625e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13857212755829096) - perf/mfu/actor:np.float64(0.07375008490475707) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.19477462768555) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.84765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005968579789623618 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:0.0005968579789623618 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:391.86328125 - response_length/max:909.0 - response_length/min:187.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:391.86328125 - response_length_non_aborted/max:909.0 - response_length_non_aborted/min:187.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.65625 - prompt_length/max:483.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.8963894844055176e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1699759010225534) - timing_s/agent_loop/generate_sequences/max:np.float64(7.213053761050105) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.171432902039669) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.213053761050105) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:880 - timing_s/gen:12.947935050353408 - timing_s/reward:0.06206372380256653 - timing_s/old_log_prob:2.3176186848431826 - timing_s/adv:0.12088934518396854 - timing_s/update_actor:9.630010453984141 - timing_s/step:25.16466488316655 - timing_s/stop_profile:0.00011617317795753479 - timing_per_token_ms/update_actor:0.04875732474967794 - timing_per_token_ms/adv:0.0006120700584984408 - timing_per_token_ms/gen:0.12907019797595032 - perf/total_num_tokens:197509 - perf/time_per_step:25.16466488316655 - perf/throughput:981.0830032755576 (TaskRunner pid=2146199) Training Progress: 86%|████████▌ | 86/100 [44:40<06:39, 28.55s/it] (TaskRunner pid=2146199) step:87 - global_seqlen/min:22273 - global_seqlen/max:32024 - global_seqlen/minmax_diff:9751 - global_seqlen/balanced_min:27519 - global_seqlen/balanced_max:27528 - global_seqlen/mean:27525.75 - actor/entropy:0.0445718877017498 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975857734680176 - training/rollout_probs_diff_mean:0.002079889178276062 - training/rollout_probs_diff_std:0.01153971441090107 - training/rollout_actor_probs_pearson_corr:0.9948481321334839 - rollout_corr/rollout_is_mean:0.9999972581863403 - rollout_corr/rollout_is_ratio_fraction_high:2.6194924430456012e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013970627333037555 - rollout_corr/rollout_is_max:2.508166790008545 - rollout_corr/rollout_is_min:0.16280917823314667 - rollout_corr/rollout_is_std:0.03028131276369095 - rollout_corr/rollout_is_eff_sample_size:0.9990840011771853 - rollout_corr/rollout_is_seq_mean:0.9999658465385437 - rollout_corr/rollout_is_seq_std:0.0017042295075953007 - rollout_corr/rollout_is_seq_max:1.0062042474746704 - rollout_corr/rollout_is_seq_min:0.988897979259491 - rollout_corr/rollout_is_seq_max_deviation:0.011102020740509033 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0508045055903494) - rollout_corr/training_log_ppl:np.float64(0.04917257259876351) - rollout_corr/kl:np.float64(0.0008119303084285212) - rollout_corr/k3_kl:np.float64(0.0006376066653928092) - rollout_corr/rollout_ppl:np.float64(1.049935205373913) - rollout_corr/rollout_log_ppl:np.float64(0.04836064190021716) - rollout_corr/log_ppl_diff:np.float64(0.0008119306985463481) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015615168049407657) - rollout_corr/log_ppl_diff_max:np.float64(0.019536092877388) - rollout_corr/log_ppl_diff_min:np.float64(-0.004697389900684357) - rollout_corr/ppl_ratio:np.float64(1.0008148858323693) - rollout_corr/chi2_token:np.float64(0.0008333791047334671) - rollout_corr/chi2_seq:np.float64(0.3704268264118582) - actor/pg_loss:np.float64(-1.9594677723944187e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019473570864647627) - actor/ppo_kl:np.float64(0.0002214926234174186) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15005829744040966) - perf/mfu/actor:np.float64(0.0816836689600733) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.16437530517578) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005369959399104118 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005369959399104118 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:447.3671875 - response_length/max:904.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:447.3671875 - response_length_non_aborted/max:904.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.8125 - prompt_length/max:518.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012892857193946838 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2074314299970865) - timing_s/agent_loop/generate_sequences/max:np.float64(7.719810716807842) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.875570193362364) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.719810716807842) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:518 - timing_s/agent_loop/slowest/response_length:904 - timing_s/gen:13.502020623534918 - timing_s/reward:0.06480852328240871 - timing_s/old_log_prob:2.3401190657168627 - timing_s/adv:0.1175947766751051 - timing_s/update_actor:9.74976965226233 - timing_s/step:25.86268744431436 - timing_s/stop_profile:0.00012781284749507904 - timing_per_token_ms/update_actor:0.04427567664942068 - timing_per_token_ms/adv:0.0005340216736833015 - timing_per_token_ms/gen:0.11789480662500147 - perf/total_num_tokens:220206 - perf/time_per_step:25.86268744431436 - perf/throughput:1064.3035476984526 (TaskRunner pid=2146199) Training Progress: 87%|████████▋ | 87/100 [45:06<06:00, 27.76s/it] (TaskRunner pid=2146199) step:88 - global_seqlen/min:20782 - global_seqlen/max:32487 - global_seqlen/minmax_diff:11705 - global_seqlen/balanced_min:26121 - global_seqlen/balanced_max:26124 - global_seqlen/mean:26122.125 - actor/entropy:0.04978610947728157 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6780092716217041 - training/rollout_probs_diff_mean:0.00200673402287066 - training/rollout_probs_diff_std:0.011237187311053276 - training/rollout_actor_probs_pearson_corr:0.9956738948822021 - rollout_corr/rollout_is_mean:0.9999000430107117 - rollout_corr/rollout_is_ratio_fraction_high:5.4422263929154724e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.070376836461946e-05 - rollout_corr/rollout_is_max:2.5243659019470215 - rollout_corr/rollout_is_min:0.07763990014791489 - rollout_corr/rollout_is_std:0.028913918882608414 - rollout_corr/rollout_is_eff_sample_size:0.9991645645790151 - rollout_corr/rollout_is_seq_mean:0.9999350905418396 - rollout_corr/rollout_is_seq_std:0.001755621051415801 - rollout_corr/rollout_is_seq_max:1.007818579673767 - rollout_corr/rollout_is_seq_min:0.9955093264579773 - rollout_corr/rollout_is_seq_max_deviation:0.00781857967376709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0563168628141284) - rollout_corr/training_log_ppl:np.float64(0.05398160306322097) - rollout_corr/kl:np.float64(0.0006752784820314162) - rollout_corr/k3_kl:np.float64(0.0006548747212740125) - rollout_corr/rollout_ppl:np.float64(1.0555871645919979) - rollout_corr/rollout_log_ppl:np.float64(0.05330632410823455) - rollout_corr/log_ppl_diff:np.float64(0.0006752789549864247) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015276311451088986) - rollout_corr/log_ppl_diff_max:np.float64(0.006414309144020081) - rollout_corr/log_ppl_diff_min:np.float64(-0.006607219576835632) - rollout_corr/ppl_ratio:np.float64(1.0006774859502912) - rollout_corr/chi2_token:np.float64(0.001272396882995963) - rollout_corr/chi2_seq:np.float64(0.44137590448372066) - actor/pg_loss:np.float64(0.0001823698403313756) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005461941095745715) - actor/ppo_kl:np.float64(0.00012440625442877717) - actor/pg_clipfrac_lower:np.float64(4.689375600719359e-06) - actor/grad_norm:np.float64(0.21767354384064674) - perf/mfu/actor:np.float64(0.07690475459693276) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.15730285644531) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0031417517457157373 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0031417517457157373 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:430.66015625 - response_length/max:850.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:430.66015625 - response_length_non_aborted/max:850.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.65625 - prompt_length/max:490.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018705613911151886 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9101635981351137) - timing_s/agent_loop/generate_sequences/max:np.float64(7.360999694094062) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.570031204355473) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.360999694094062) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:807 - timing_s/gen:13.09954946115613 - timing_s/reward:0.062071667984128 - timing_s/old_log_prob:2.397555310279131 - timing_s/adv:0.14476189203560352 - timing_s/update_actor:9.617085754871368 - timing_s/step:25.405381184071302 - timing_s/stop_profile:2.738460898399353e-05 - timing_per_token_ms/update_actor:0.0460198287604443 - timing_per_token_ms/adv:0.0006927168637486591 - timing_per_token_ms/gen:0.11881785287082994 - perf/total_num_tokens:208977 - perf/time_per_step:25.405381184071302 - perf/throughput:1028.2122834818192 (TaskRunner pid=2146199) Training Progress: 88%|████████▊ | 88/100 [45:31<05:24, 27.07s/it] (TaskRunner pid=2146199) step:89 - global_seqlen/min:24649 - global_seqlen/max:33401 - global_seqlen/minmax_diff:8752 - global_seqlen/balanced_min:29323 - global_seqlen/balanced_max:29325 - global_seqlen/mean:29323.75 - actor/entropy:0.04022124782204628 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5830117464065552 - training/rollout_probs_diff_mean:0.0017969151958823204 - training/rollout_probs_diff_std:0.010503474622964859 - training/rollout_actor_probs_pearson_corr:0.9952396750450134 - rollout_corr/rollout_is_mean:1.0000041723251343 - rollout_corr/rollout_is_ratio_fraction_high:2.3543814677395858e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.847938104532659e-05 - rollout_corr/rollout_is_max:4.0074310302734375 - rollout_corr/rollout_is_min:0.307452917098999 - rollout_corr/rollout_is_std:0.027917563915252686 - rollout_corr/rollout_is_eff_sample_size:0.9992209785938259 - rollout_corr/rollout_is_seq_mean:1.0000067949295044 - rollout_corr/rollout_is_seq_std:0.0015692247543483973 - rollout_corr/rollout_is_seq_max:1.009533405303955 - rollout_corr/rollout_is_seq_min:0.9947223663330078 - rollout_corr/rollout_is_seq_max_deviation:0.009533405303955078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0462127816863358) - rollout_corr/training_log_ppl:np.float64(0.044784407255974656) - rollout_corr/kl:np.float64(0.0006013624857392763) - rollout_corr/k3_kl:np.float64(0.0006099964043713157) - rollout_corr/rollout_ppl:np.float64(1.0455811796709895) - rollout_corr/rollout_log_ppl:np.float64(0.044183044255987625) - rollout_corr/log_ppl_diff:np.float64(0.0006013629999870318) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014206745136107202) - rollout_corr/log_ppl_diff_max:np.float64(0.008299320936203003) - rollout_corr/log_ppl_diff_min:np.float64(-0.006630688905715942) - rollout_corr/ppl_ratio:np.float64(1.00060333032161) - rollout_corr/chi2_token:np.float64(0.00131711782887578) - rollout_corr/chi2_seq:np.float64(2.751286949031055) - actor/pg_loss:np.float64(4.6454835683107376e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.8379607556416886e-05) - actor/ppo_kl:np.float64(0.00016049884379754076) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.049162840005010366) - perf/mfu/actor:np.float64(0.08483849544280975) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.06477737426758) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:3 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005621085874736309 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005621085874736309 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:497.7421875 - response_length/max:839.0 - response_length/min:198.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:497.7421875 - response_length_non_aborted/max:839.0 - response_length_non_aborted/min:198.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:418.625 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.437007009983063e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3461192548274994) - timing_s/agent_loop/generate_sequences/max:np.float64(7.896998358890414) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.441400952440745) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.896998358890414) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:836 - timing_s/gen:13.736420588567853 - timing_s/reward:0.06631769984960556 - timing_s/old_log_prob:2.330074939876795 - timing_s/adv:0.12180916219949722 - timing_s/update_actor:9.916611259803176 - timing_s/step:26.255432656034827 - timing_s/stop_profile:5.4018571972846985e-05 - timing_per_token_ms/update_actor:0.04227209710474946 - timing_per_token_ms/adv:0.0005192427733471044 - timing_per_token_ms/gen:0.10780258188199725 - perf/total_num_tokens:234590 - perf/time_per_step:26.255432656034827 - perf/throughput:1116.864093772986 (TaskRunner pid=2146199) Training Progress: 89%|████████▉ | 89/100 [45:58<04:55, 26.83s/it] (TaskRunner pid=2146199) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2146199) validation generation end (TaskRunner pid=2146199) [prompt] system (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) To calculate the volume of the unit cell for a crystal structure with orthorhombic symmetry (since all angles α, β, and γ are 90.00º), we can use the formula for the volume of a unit cell in an orthorhombic system: (TaskRunner pid=2146199) (TaskRunner pid=2146199) V = a × b × c (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given: (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Substituting these values into the formula: (TaskRunner pid=2146199) (TaskRunner pid=2146199) V = 7.82 × 7.82 × 17.88 (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate 7.82 × 7.82: (TaskRunner pid=2146199) (TaskRunner pid=2146199) 7.82 × 7.82 = 61.1524 (TaskRunner pid=2146199) (TaskRunner pid=2146199) Now multiply that by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) 61.1524 × 17.88 ≈ 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Therefore, the correct answer is B: 1094.31 ų. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2146577) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2146199) step:90 - global_seqlen/min:21572 - global_seqlen/max:33078 - global_seqlen/minmax_diff:11506 - global_seqlen/balanced_min:27047 - global_seqlen/balanced_max:27051 - global_seqlen/mean:27049.375 - actor/entropy:0.05337677150964737 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5838292837142944 - training/rollout_probs_diff_mean:0.002151863183826208 - training/rollout_probs_diff_std:0.011963930912315845 - training/rollout_actor_probs_pearson_corr:0.9951996207237244 - rollout_corr/rollout_is_mean:1.0000139474868774 - rollout_corr/rollout_is_ratio_fraction_high:3.4801068977685645e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013050400593783706 - rollout_corr/rollout_is_max:7.123044967651367 - rollout_corr/rollout_is_min:0.13723966479301453 - rollout_corr/rollout_is_std:0.030356017872691154 - rollout_corr/rollout_is_eff_sample_size:0.9990791225700723 - rollout_corr/rollout_is_seq_mean:1.0000704526901245 - rollout_corr/rollout_is_seq_std:0.0032896019984036684 - rollout_corr/rollout_is_seq_max:1.034042239189148 - rollout_corr/rollout_is_seq_min:0.9943957328796387 - rollout_corr/rollout_is_seq_max_deviation:0.03404223918914795 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0565867288969457) - rollout_corr/training_log_ppl:np.float64(0.05451778970382293) - rollout_corr/kl:np.float64(0.0005373297904114338) - rollout_corr/k3_kl:np.float64(0.000734658154001977) - rollout_corr/rollout_ppl:np.float64(1.05601063137874) - rollout_corr/rollout_log_ppl:np.float64(0.0539804592008295) - rollout_corr/log_ppl_diff:np.float64(0.0005373305029934272) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014802939986111596) - rollout_corr/log_ppl_diff_max:np.float64(0.011161405593156815) - rollout_corr/log_ppl_diff_min:np.float64(-0.010649912059307098) - rollout_corr/ppl_ratio:np.float64(1.0005397493951023) - rollout_corr/chi2_token:np.float64(0.0029833028092980385) - rollout_corr/chi2_seq:np.float64(1.778731806203723) - actor/pg_loss:np.float64(9.338895324617624e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(2.1003947040298954e-05) - actor/ppo_kl:np.float64(-6.227661563396047e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10093431454151869) - perf/mfu/actor:np.float64(0.07837184889663078) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.1535758972168) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7566489361702128) - val-aux/sciknoweval/reward/std@16:np.float64(0.051779916379202574) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7787446808510639) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.039970469659672915) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7347021276595744) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.04786711484565726) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7571702127659574) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.05225740433309325) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7945638297872339) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.024899347755593753) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7152978723404255) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.03827400172973456) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7601808510638297) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.042683415031958986) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8042446808510638) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.012364483567494838) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6999255319148937) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.02654025168353534) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7619148936170211) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.029941536785489036) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8078404255319148) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.004754004717463307) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6893936170212766) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.016073646699142997) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7625851063829788) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.01878738114715779) - val-aux/sciknoweval/score/mean@16:np.float64(0.7566489361702128) - val-aux/sciknoweval/score/std@16:np.float64(0.051779916379202574) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7787446808510639) - val-aux/sciknoweval/score/best@2/std:np.float64(0.039970469659672915) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7347021276595744) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.04786711484565726) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7571702127659574) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.05225740433309325) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7945638297872339) - val-aux/sciknoweval/score/best@4/std:np.float64(0.024899347755593753) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7152978723404255) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.03827400172973456) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7601808510638297) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.042683415031958986) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8042446808510638) - val-aux/sciknoweval/score/best@8/std:np.float64(0.012364483567494838) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6999255319148937) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.02654025168353534) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7619148936170211) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.029941536785489036) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8078404255319148) - val-aux/sciknoweval/score/best@16/std:np.float64(0.004754004717463307) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6893936170212766) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.016073646699142997) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7625851063829788) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.01878738114715779) - val-core/sciknoweval/acc/mean@16:np.float64(0.7566489361702128) - val-aux/sciknoweval/acc/std@16:np.float64(0.051779916379202574) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7787446808510639) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.039970469659672915) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7347021276595744) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.04786711484565726) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7571702127659574) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.05225740433309325) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7945638297872339) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.024899347755593753) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7152978723404255) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.03827400172973456) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7601808510638297) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.042683415031958986) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8042446808510638) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.012364483567494838) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6999255319148937) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.02654025168353534) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7619148936170211) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.029941536785489036) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8078404255319148) - val-core/sciknoweval/acc/best@16/std:np.float64(0.004754004717463307) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6893936170212766) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.016073646699142997) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7625851063829788) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.01878738114715779) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9993351063829787) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0025751219057230166) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999893617021276) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0003362442687080662) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9987659574468085) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.003406648095255393) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9993723404255319) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0025066457616098187) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9976702127659575) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.004399666125856953) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9998297872340427) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0013348414588389894) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9957978723404256) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.005200539286727206) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9930744680851065) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.005070786835475184) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:3 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006492574466392398 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006492574466392398 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:448.98046875 - response_length/max:1085.0 - response_length/min:186.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:448.98046875 - response_length_non_aborted/max:1085.0 - response_length_non_aborted/min:186.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:509.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.03670084476471e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0495247840881348) - timing_s/agent_loop/generate_sequences/max:np.float64(8.571558777242899) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7728800085387775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.571558777242899) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:1085 - timing_s/gen:14.102520316839218 - timing_s/reward:0.06438948027789593 - timing_s/old_log_prob:2.4226504396647215 - timing_s/adv:0.1210464145988226 - timing_s/update_actor:9.881728129461408 - timing_s/step:26.677598802372813 - timing_s/testing:87.7413747496903 - timing_s/save_checkpoint:6.636503567919135 - timing_s/stop_profile:0.000112876296043396 - timing_per_token_ms/update_actor:0.04566523315909059 - timing_per_token_ms/adv:0.0005593771325530746 - timing_per_token_ms/gen:0.12269569351429209 - perf/total_num_tokens:216395 - perf/time_per_step:26.677598802372813 - perf/throughput:1013.9358943202234 (TaskRunner pid=2146199) Training Progress: 90%|█████████ | 90/100 [47:59<09:11, 55.12s/it] (TaskRunner pid=2146199) step:91 - global_seqlen/min:22592 - global_seqlen/max:31989 - global_seqlen/minmax_diff:9397 - global_seqlen/balanced_min:25799 - global_seqlen/balanced_max:32554 - global_seqlen/mean:26645.25 - actor/entropy:0.05074597895145416 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.466818243265152 - training/rollout_probs_diff_mean:0.0018669883720576763 - training/rollout_probs_diff_std:0.009989441372454166 - training/rollout_actor_probs_pearson_corr:0.9965559840202332 - rollout_corr/rollout_is_mean:0.9999288320541382 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.37644440960139e-05 - rollout_corr/rollout_is_max:1.919961929321289 - rollout_corr/rollout_is_min:0.16960850358009338 - rollout_corr/rollout_is_std:0.026654968038201332 - rollout_corr/rollout_is_eff_sample_size:0.999289898028118 - rollout_corr/rollout_is_seq_mean:0.999915361404419 - rollout_corr/rollout_is_seq_std:0.0015413021901622415 - rollout_corr/rollout_is_seq_max:1.0052435398101807 - rollout_corr/rollout_is_seq_min:0.9944308996200562 - rollout_corr/rollout_is_seq_max_deviation:0.005569100379943848 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0612004683353007) - rollout_corr/training_log_ppl:np.float64(0.05845046364629525) - rollout_corr/kl:np.float64(0.0005044721559075427) - rollout_corr/k3_kl:np.float64(0.00045321661817965264) - rollout_corr/rollout_ppl:np.float64(1.0606594067066908) - rollout_corr/rollout_log_ppl:np.float64(0.05794599100045161) - rollout_corr/log_ppl_diff:np.float64(0.0005044726458436344) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012553419364849105) - rollout_corr/log_ppl_diff_max:np.float64(0.00856219232082367) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035656802356243134) - rollout_corr/ppl_ratio:np.float64(1.0005058830138296) - rollout_corr/chi2_token:np.float64(0.0008079037070274353) - rollout_corr/chi2_seq:np.float64(0.5942554601933807) - actor/pg_loss:np.float64(-6.973976269364357e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.7351449666166445e-05) - actor/ppo_kl:np.float64(5.3711747192153325e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.01579522853717208) - perf/mfu/actor:np.float64(0.07329023449381297) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(104.8720932006836) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:3 - critic/score/mean:0.80859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.054961685091257095 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.054961685091257095 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:476.6015625 - response_length/max:8192.0 - response_length/min:187.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:476.6015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:187.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:356.0625 - prompt_length/max:477.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.0185248255729675e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0423638969659805) - timing_s/agent_loop/generate_sequences/max:np.float64(50.59291830845177) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7901106993012945) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.59291830845177) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.42935983836651 - timing_s/reward:0.05745197460055351 - timing_s/old_log_prob:2.7820449508726597 - timing_s/adv:0.12995323166251183 - timing_s/update_actor:10.38615631684661 - timing_s/step:69.8929017689079 - timing_s/stop_profile:0.00014711357653141022 - timing_per_token_ms/update_actor:0.04872423938997856 - timing_per_token_ms/adv:0.0006096453948757838 - timing_per_token_ms/gen:0.4624978267221253 - perf/total_num_tokens:213162 - perf/time_per_step:69.8929017689079 - perf/throughput:381.22970037929133 (TaskRunner pid=2146199) Training Progress: 91%|█████████ | 91/100 [49:09<08:56, 59.57s/it] (TaskRunner pid=2146199) step:92 - global_seqlen/min:24751 - global_seqlen/max:31005 - global_seqlen/minmax_diff:6254 - global_seqlen/balanced_min:28583 - global_seqlen/balanced_max:28599 - global_seqlen/mean:28590.25 - actor/entropy:0.07447073608636856 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6695266366004944 - training/rollout_probs_diff_mean:0.0025611515156924725 - training/rollout_probs_diff_std:0.01220319326967001 - training/rollout_actor_probs_pearson_corr:0.9964745044708252 - rollout_corr/rollout_is_mean:1.000080943107605 - rollout_corr/rollout_is_ratio_fraction_high:7.045230449875817e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011742050992324948 - rollout_corr/rollout_is_max:4.670137405395508 - rollout_corr/rollout_is_min:0.04595468193292618 - rollout_corr/rollout_is_std:0.033514026552438736 - rollout_corr/rollout_is_eff_sample_size:0.9988779512721494 - rollout_corr/rollout_is_seq_mean:1.0001102685928345 - rollout_corr/rollout_is_seq_std:0.001765890745446086 - rollout_corr/rollout_is_seq_max:1.0090281963348389 - rollout_corr/rollout_is_seq_min:0.9952805042266846 - rollout_corr/rollout_is_seq_max_deviation:0.009028196334838867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0732701481319964) - rollout_corr/training_log_ppl:np.float64(0.0694317512861744) - rollout_corr/kl:np.float64(0.00046238616268556854) - rollout_corr/k3_kl:np.float64(0.0006291608898933987) - rollout_corr/rollout_ppl:np.float64(1.0727753364481032) - rollout_corr/rollout_log_ppl:np.float64(0.06896936462908343) - rollout_corr/log_ppl_diff:np.float64(0.00046238665709097404) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013666152972291457) - rollout_corr/log_ppl_diff_max:np.float64(0.007326595485210419) - rollout_corr/log_ppl_diff_min:np.float64(-0.006338030099868774) - rollout_corr/ppl_ratio:np.float64(1.0004640747793019) - rollout_corr/chi2_token:np.float64(0.0016734306700527668) - rollout_corr/chi2_seq:np.float64(0.6637354781851172) - actor/pg_loss:np.float64(7.901119533926249e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00043912684168390115) - actor/ppo_kl:np.float64(-4.9294723233117566e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16749685630202293) - perf/mfu/actor:np.float64(0.083484074070933) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(104.75609970092773) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005313277943059802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005313277943059802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:499.0078125 - response_length/max:1275.0 - response_length/min:203.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:499.0078125 - response_length_non_aborted/max:1275.0 - response_length_non_aborted/min:203.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.4375 - prompt_length/max:476.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.717419743537903e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.4601905792951584) - timing_s/agent_loop/generate_sequences/max:np.float64(9.975601432844996) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.471915273170453) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.975601432844996) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:1275 - timing_s/gen:15.71469853632152 - timing_s/reward:0.06533286906778812 - timing_s/old_log_prob:2.3762769252061844 - timing_s/adv:0.11895527318120003 - timing_s/update_actor:9.833900161087513 - timing_s/step:28.20399871468544 - timing_s/stop_profile:2.664327621459961e-05 - timing_per_token_ms/update_actor:0.04299499025492744 - timing_per_token_ms/adv:0.0005200867130455314 - timing_per_token_ms/gen:0.12301519058382666 - perf/total_num_tokens:228722 - perf/time_per_step:28.20399871468544 - perf/throughput:1013.6949121726291 (TaskRunner pid=2146199) Training Progress: 92%|█████████▏| 92/100 [49:37<06:41, 50.16s/it] (TaskRunner pid=2146199) step:93 - global_seqlen/min:21194 - global_seqlen/max:30043 - global_seqlen/minmax_diff:8849 - global_seqlen/balanced_min:26263 - global_seqlen/balanced_max:26272 - global_seqlen/mean:26267.875 - actor/entropy:0.047844935208559036 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5599756240844727 - training/rollout_probs_diff_mean:0.0019041248597204685 - training/rollout_probs_diff_std:0.010661047883331776 - training/rollout_actor_probs_pearson_corr:0.9959314465522766 - rollout_corr/rollout_is_mean:0.9998970031738281 - rollout_corr/rollout_is_ratio_fraction_high:7.087737321853638e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.087737321853638e-05 - rollout_corr/rollout_is_max:2.549046516418457 - rollout_corr/rollout_is_min:0.17551067471504211 - rollout_corr/rollout_is_std:0.02885614149272442 - rollout_corr/rollout_is_eff_sample_size:0.9991676588535228 - rollout_corr/rollout_is_seq_mean:0.9999610185623169 - rollout_corr/rollout_is_seq_std:0.0016428149538114667 - rollout_corr/rollout_is_seq_max:1.006683349609375 - rollout_corr/rollout_is_seq_min:0.994486391544342 - rollout_corr/rollout_is_seq_max_deviation:0.006683349609375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0567301423288882) - rollout_corr/training_log_ppl:np.float64(0.05439375846617622) - rollout_corr/kl:np.float64(0.0006434451307271161) - rollout_corr/k3_kl:np.float64(0.0005920195093249703) - rollout_corr/rollout_ppl:np.float64(1.0560367717407644) - rollout_corr/rollout_log_ppl:np.float64(0.053750312803458655) - rollout_corr/log_ppl_diff:np.float64(0.0006434456627175678) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013332313756109215) - rollout_corr/log_ppl_diff_max:np.float64(0.007844395935535431) - rollout_corr/log_ppl_diff_min:np.float64(-0.00539582222700119) - rollout_corr/ppl_ratio:np.float64(1.0006451506633312) - rollout_corr/chi2_token:np.float64(0.0011210150551050901) - rollout_corr/chi2_seq:np.float64(0.10734880738891661) - actor/pg_loss:np.float64(2.6544439606368542e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.816855415716418e-05) - actor/ppo_kl:np.float64(9.917735489750612e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.02947139646857977) - perf/mfu/actor:np.float64(0.07661425337252582) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.23891067504883) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:3 - critic/score/mean:0.78515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008228420047089458 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:0.0008228420047089458 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:440.90234375 - response_length/max:1010.0 - response_length/min:170.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.90234375 - response_length_non_aborted/max:1010.0 - response_length_non_aborted/min:170.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.96875 - prompt_length/max:492.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.2862106561660767e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8822572939097881) - timing_s/agent_loop/generate_sequences/max:np.float64(8.475016556680202) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.550744398489769) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.475016556680202) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:436 - timing_s/agent_loop/slowest/response_length:1010 - timing_s/gen:14.08065290749073 - timing_s/reward:0.06197900325059891 - timing_s/old_log_prob:2.6011801548302174 - timing_s/adv:0.15368284285068512 - timing_s/update_actor:9.930420707911253 - timing_s/step:26.909868858754635 - timing_s/stop_profile:0.00012194924056529999 - timing_per_token_ms/update_actor:0.04725553888500332 - timing_per_token_ms/adv:0.0007313250636503957 - timing_per_token_ms/gen:0.12474996152679368 - perf/total_num_tokens:210143 - perf/time_per_step:26.909868858754635 - perf/throughput:976.1428098321716 (TaskRunner pid=2146199) Training Progress: 93%|█████████▎| 93/100 [50:04<05:02, 43.21s/it] (TaskRunner pid=2146199) step:94 - global_seqlen/min:23365 - global_seqlen/max:31374 - global_seqlen/minmax_diff:8009 - global_seqlen/balanced_min:28365 - global_seqlen/balanced_max:28377 - global_seqlen/mean:28373.625 - actor/entropy:0.05107166990637779 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6118495464324951 - training/rollout_probs_diff_mean:0.002033080905675888 - training/rollout_probs_diff_std:0.011409728787839413 - training/rollout_actor_probs_pearson_corr:0.9957208633422852 - rollout_corr/rollout_is_mean:0.999905526638031 - rollout_corr/rollout_is_ratio_fraction_high:3.147301686112769e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015736508066765964 - rollout_corr/rollout_is_max:6.195666790008545 - rollout_corr/rollout_is_min:0.23039421439170837 - rollout_corr/rollout_is_std:0.029570257291197777 - rollout_corr/rollout_is_eff_sample_size:0.9991261257666647 - rollout_corr/rollout_is_seq_mean:0.9998438954353333 - rollout_corr/rollout_is_seq_std:0.0018056193366646767 - rollout_corr/rollout_is_seq_max:1.010923981666565 - rollout_corr/rollout_is_seq_min:0.9918465614318848 - rollout_corr/rollout_is_seq_max_deviation:0.010923981666564941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0576600073836744) - rollout_corr/training_log_ppl:np.float64(0.055277571715123486) - rollout_corr/kl:np.float64(0.0006258701370072117) - rollout_corr/k3_kl:np.float64(0.000533808628354393) - rollout_corr/rollout_ppl:np.float64(1.056982469279319) - rollout_corr/rollout_log_ppl:np.float64(0.05465170104071149) - rollout_corr/log_ppl_diff:np.float64(0.0006258706744119991) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012843467411585152) - rollout_corr/log_ppl_diff_max:np.float64(0.00776304304599762) - rollout_corr/log_ppl_diff_min:np.float64(-0.005833819508552551) - rollout_corr/ppl_ratio:np.float64(1.0006276078056544) - rollout_corr/chi2_token:np.float64(0.0009046124760061502) - rollout_corr/chi2_seq:np.float64(2.0929555441252887) - actor/pg_loss:np.float64(0.00027054280508309603) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002680098132259445) - actor/ppo_kl:np.float64(-7.556408530295933e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1807025857269764) - perf/mfu/actor:np.float64(0.08282964649513413) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.09398651123047) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:3 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006392956129275262 - critic/advantages/max:0.875 - critic/advantages/min:-0.25 - critic/returns/mean:-0.0006392956129275262 - critic/returns/max:0.875 - critic/returns/min:-0.25 - response_length/mean:496.45703125 - response_length/max:1071.0 - response_length/min:211.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:496.45703125 - response_length_non_aborted/max:1071.0 - response_length_non_aborted/min:211.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.21875 - prompt_length/max:483.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.479746222496033e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.335531396791339) - timing_s/agent_loop/generate_sequences/max:np.float64(8.884080475196242) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.296686432004208) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.884080475196242) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:471 - timing_s/agent_loop/slowest/response_length:1071 - timing_s/gen:14.734076181426644 - timing_s/reward:0.06665491685271263 - timing_s/old_log_prob:2.328215977177024 - timing_s/adv:0.11762564443051815 - timing_s/update_actor:9.743917653337121 - timing_s/step:27.079343495890498 - timing_s/stop_profile:0.00013345107436180115 - timing_per_token_ms/update_actor:0.042926827526167 - timing_per_token_ms/adv:0.0005181997560697574 - timing_per_token_ms/gen:0.1159314531990483 - perf/total_num_tokens:226989 - perf/time_per_step:27.079343495890498 - perf/throughput:1047.795896688039 (TaskRunner pid=2146199) Training Progress: 94%|█████████▍| 94/100 [50:31<03:50, 38.38s/it] (TaskRunner pid=2146199) step:95 - global_seqlen/min:25115 - global_seqlen/max:33047 - global_seqlen/minmax_diff:7932 - global_seqlen/balanced_min:28616 - global_seqlen/balanced_max:28619 - global_seqlen/mean:28617.75 - actor/entropy:0.04661192372441292 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8254104256629944 - training/rollout_probs_diff_mean:0.0019973409362137318 - training/rollout_probs_diff_std:0.011340063065290451 - training/rollout_actor_probs_pearson_corr:0.9950777888298035 - rollout_corr/rollout_is_mean:1.0000083446502686 - rollout_corr/rollout_is_ratio_fraction_high:7.841046317480505e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.056941831251606e-05 - rollout_corr/rollout_is_max:2.3633062839508057 - rollout_corr/rollout_is_min:0.10317593812942505 - rollout_corr/rollout_is_std:0.029430845752358437 - rollout_corr/rollout_is_eff_sample_size:0.9991345749114386 - rollout_corr/rollout_is_seq_mean:1.0000441074371338 - rollout_corr/rollout_is_seq_std:0.0015897169942036271 - rollout_corr/rollout_is_seq_max:1.0052056312561035 - rollout_corr/rollout_is_seq_min:0.9953656196594238 - rollout_corr/rollout_is_seq_max_deviation:0.005205631256103516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.052196632605046) - rollout_corr/training_log_ppl:np.float64(0.050540514561362215) - rollout_corr/kl:np.float64(0.0006028899679608735) - rollout_corr/k3_kl:np.float64(0.0006437514416504087) - rollout_corr/rollout_ppl:np.float64(1.051551810465753) - rollout_corr/rollout_log_ppl:np.float64(0.04993762408957991) - rollout_corr/log_ppl_diff:np.float64(0.0006028904717823025) - rollout_corr/log_ppl_abs_diff:np.float64(0.001454217235732358) - rollout_corr/log_ppl_diff_max:np.float64(0.0101885125041008) - rollout_corr/log_ppl_diff_min:np.float64(-0.003857158124446869) - rollout_corr/ppl_ratio:np.float64(1.000604884698987) - rollout_corr/chi2_token:np.float64(0.0013503972440958023) - rollout_corr/chi2_seq:np.float64(1.4245592006482184) - actor/pg_loss:np.float64(8.295488078147173e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020260240671632346) - actor/ppo_kl:np.float64(0.00012206497116018511) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1737770214676857) - perf/mfu/actor:np.float64(0.08376044684646436) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.20892715454102) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:3 - critic/score/mean:0.76171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.76171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:3.136418672511354e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:3.136418672511354e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:498.1796875 - response_length/max:856.0 - response_length/min:180.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:498.1796875 - response_length_non_aborted/max:856.0 - response_length_non_aborted/min:180.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.125 - prompt_length/max:493.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014534778892993927 - timing_s/agent_loop/generate_sequences/min:np.float64(2.153837487101555) - timing_s/agent_loop/generate_sequences/max:np.float64(8.180927706882358) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.50552720228734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.180927706882358) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:438 - timing_s/agent_loop/slowest/response_length:848 - timing_s/gen:13.918872836977243 - timing_s/reward:0.07109526358544827 - timing_s/old_log_prob:2.4231134839355946 - timing_s/adv:0.15419645980000496 - timing_s/update_actor:9.866513896733522 - timing_s/step:26.522293388843536 - timing_s/stop_profile:0.00011077150702476501 - timing_per_token_ms/update_actor:0.04309612869955501 - timing_per_token_ms/adv:0.0006735175712626122 - timing_per_token_ms/gen:0.109138526486876 - perf/total_num_tokens:228942 - perf/time_per_step:26.522293388843536 - perf/throughput:1079.007368647762 (TaskRunner pid=2146199) Training Progress: 95%|█████████▌| 95/100 [50:58<02:54, 34.84s/it] (TaskRunner pid=2146199) step:96 - global_seqlen/min:22643 - global_seqlen/max:34496 - global_seqlen/minmax_diff:11853 - global_seqlen/balanced_min:29438 - global_seqlen/balanced_max:29502 - global_seqlen/mean:29458.25 - actor/entropy:0.04679719731211662 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49973881244659424 - training/rollout_probs_diff_mean:0.002006575698032975 - training/rollout_probs_diff_std:0.011437534354627132 - training/rollout_actor_probs_pearson_corr:0.9951053857803345 - rollout_corr/rollout_is_mean:1.0001201629638672 - rollout_corr/rollout_is_ratio_fraction_high:1.5229740711220074e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015991227701306343 - rollout_corr/rollout_is_max:2.2695932388305664 - rollout_corr/rollout_is_min:0.0005223815678618848 - rollout_corr/rollout_is_std:0.029386257752776146 - rollout_corr/rollout_is_eff_sample_size:0.9991375499964328 - rollout_corr/rollout_is_seq_mean:1.0001219511032104 - rollout_corr/rollout_is_seq_std:0.0016149127623066306 - rollout_corr/rollout_is_seq_max:1.0052374601364136 - rollout_corr/rollout_is_seq_min:0.9945542216300964 - rollout_corr/rollout_is_seq_max_deviation:0.0054457783699035645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0504484134726226) - rollout_corr/training_log_ppl:np.float64(0.04875225234536629) - rollout_corr/kl:np.float64(0.0006898636913366119) - rollout_corr/k3_kl:np.float64(0.0007324333878671041) - rollout_corr/rollout_ppl:np.float64(1.0497013847343624) - rollout_corr/rollout_log_ppl:np.float64(0.048062387919344474) - rollout_corr/log_ppl_diff:np.float64(0.0006898644260218134) - rollout_corr/log_ppl_abs_diff:np.float64(0.001569076217492693) - rollout_corr/log_ppl_diff_max:np.float64(0.010027572512626648) - rollout_corr/log_ppl_diff_min:np.float64(-0.005765251815319061) - rollout_corr/ppl_ratio:np.float64(1.0006924110930413) - rollout_corr/chi2_token:np.float64(0.0014178783167153597) - rollout_corr/chi2_seq:np.float64(0.7907166071236134) - actor/pg_loss:np.float64(-3.4949975088238716e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029350927343330113) - actor/ppo_kl:np.float64(0.0002765249004426096) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16087528318166733) - perf/mfu/actor:np.float64(0.08561085412592156) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.13811874389648) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:3 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007138940854929388 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0007138940854929388 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:512.9765625 - response_length/max:1581.0 - response_length/min:177.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:512.9765625 - response_length_non_aborted/max:1581.0 - response_length_non_aborted/min:177.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.59375 - prompt_length/max:505.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001514684408903122 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0514542553573847) - timing_s/agent_loop/generate_sequences/max:np.float64(11.520855454728007) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.558338761380583) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.520855454728007) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:1581 - timing_s/gen:17.156338008120656 - timing_s/reward:0.06802140548825264 - timing_s/old_log_prob:2.364218408241868 - timing_s/adv:0.11801940202713013 - timing_s/update_actor:9.785525593906641 - timing_s/step:29.579010501503944 - timing_s/stop_profile:0.00010866113007068634 - timing_per_token_ms/update_actor:0.04152285689877471 - timing_per_token_ms/adv:0.0005007909585053853 - timing_per_token_ms/gen:0.1306432890766258 - perf/total_num_tokens:235666 - perf/time_per_step:29.579010501503944 - perf/throughput:995.9173583072428 (TaskRunner pid=2146199) Training Progress: 96%|█████████▌| 96/100 [51:27<02:13, 33.28s/it] (TaskRunner pid=2146199) step:97 - global_seqlen/min:22013 - global_seqlen/max:34790 - global_seqlen/minmax_diff:12777 - global_seqlen/balanced_min:27093 - global_seqlen/balanced_max:33403 - global_seqlen/mean:28695.625 - actor/entropy:0.05618748813867569 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39973196387290955 - training/rollout_probs_diff_mean:0.0019156996859237552 - training/rollout_probs_diff_std:0.010508053936064243 - training/rollout_actor_probs_pearson_corr:0.996550440788269 - rollout_corr/rollout_is_mean:0.9999145269393921 - rollout_corr/rollout_is_ratio_fraction_high:7.371315405180212e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.845578122418374e-05 - rollout_corr/rollout_is_max:3.37223482131958 - rollout_corr/rollout_is_min:0.27226296067237854 - rollout_corr/rollout_is_std:0.02762783132493496 - rollout_corr/rollout_is_eff_sample_size:0.999237166069566 - rollout_corr/rollout_is_seq_mean:0.9998699426651001 - rollout_corr/rollout_is_seq_std:0.0017847311682999134 - rollout_corr/rollout_is_seq_max:1.006451964378357 - rollout_corr/rollout_is_seq_min:0.9932880401611328 - rollout_corr/rollout_is_seq_max_deviation:0.0067119598388671875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0567704434506595) - rollout_corr/training_log_ppl:np.float64(0.054321528661603224) - rollout_corr/kl:np.float64(0.0004643425212407237) - rollout_corr/k3_kl:np.float64(0.0005528200473072786) - rollout_corr/rollout_ppl:np.float64(1.0562733570113778) - rollout_corr/rollout_log_ppl:np.float64(0.05385718593788624) - rollout_corr/log_ppl_diff:np.float64(0.0004643427237169817) - rollout_corr/log_ppl_abs_diff:np.float64(0.001442504406441003) - rollout_corr/log_ppl_diff_max:np.float64(0.008554868400096893) - rollout_corr/log_ppl_diff_min:np.float64(-0.005728933960199356) - rollout_corr/ppl_ratio:np.float64(1.0004662966821343) - rollout_corr/chi2_token:np.float64(0.0013686688616871834) - rollout_corr/chi2_seq:np.float64(1.28147081611678) - actor/pg_loss:np.float64(-1.70036219060421e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001624026911031251) - actor/ppo_kl:np.float64(-0.00013354237297402705) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.07056355243548751) - perf/mfu/actor:np.float64(0.07932614276479677) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.1329116821289) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:3 - critic/score/mean:0.85546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.85546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.10374020785093307 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:-0.10374020785093307 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:529.92578125 - response_length/max:8192.0 - response_length/min:190.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:529.92578125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:190.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:366.8125 - prompt_length/max:490.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.939765393733978e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2069880440831184) - timing_s/agent_loop/generate_sequences/max:np.float64(50.93019938468933) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.171387562877499) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.93019938468933) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.85105039738119 - timing_s/reward:0.05547734349966049 - timing_s/old_log_prob:2.5362195391207933 - timing_s/adv:0.13003680482506752 - timing_s/update_actor:10.606128588318825 - timing_s/step:70.21276222355664 - timing_s/stop_profile:0.00011604651808738708 - timing_per_token_ms/update_actor:0.04620098267731938 - timing_per_token_ms/adv:0.0005664487392462594 - timing_per_token_ms/gen:0.41906701555628506 - perf/total_num_tokens:229565 - perf/time_per_step:70.21276222355664 - perf/throughput:408.69528688578663 (TaskRunner pid=2146199) Training Progress: 97%|█████████▋| 97/100 [52:38<02:13, 44.38s/it] (TaskRunner pid=2146199) step:98 - global_seqlen/min:22797 - global_seqlen/max:30863 - global_seqlen/minmax_diff:8066 - global_seqlen/balanced_min:26794 - global_seqlen/balanced_max:26799 - global_seqlen/mean:26796.375 - actor/entropy:0.04842422157526016 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7172131538391113 - training/rollout_probs_diff_mean:0.002257800195366144 - training/rollout_probs_diff_std:0.012311161495745182 - training/rollout_actor_probs_pearson_corr:0.9944987297058105 - rollout_corr/rollout_is_mean:1.0000731945037842 - rollout_corr/rollout_is_ratio_fraction_high:3.5939225199399516e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001707113260636106 - rollout_corr/rollout_is_max:2.2531673908233643 - rollout_corr/rollout_is_min:0.07730188965797424 - rollout_corr/rollout_is_std:0.031039481982588768 - rollout_corr/rollout_is_eff_sample_size:0.9990378348031789 - rollout_corr/rollout_is_seq_mean:1.0002034902572632 - rollout_corr/rollout_is_seq_std:0.00173987727612257 - rollout_corr/rollout_is_seq_max:1.0084892511367798 - rollout_corr/rollout_is_seq_min:0.9956490993499756 - rollout_corr/rollout_is_seq_max_deviation:0.008489251136779785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.050764275714755) - rollout_corr/training_log_ppl:np.float64(0.04918175351122045) - rollout_corr/kl:np.float64(0.00046318168666559245) - rollout_corr/k3_kl:np.float64(0.0006677604205691523) - rollout_corr/rollout_ppl:np.float64(1.0502702398225665) - rollout_corr/rollout_log_ppl:np.float64(0.048718571197241545) - rollout_corr/log_ppl_diff:np.float64(0.00046318231397890486) - rollout_corr/log_ppl_abs_diff:np.float64(0.001540178225695854) - rollout_corr/log_ppl_diff_max:np.float64(0.01113704964518547) - rollout_corr/log_ppl_diff_min:np.float64(-0.006780542433261871) - rollout_corr/ppl_ratio:np.float64(1.0004655125085264) - rollout_corr/chi2_token:np.float64(0.0017206836491823196) - rollout_corr/chi2_seq:np.float64(0.668095194734633) - actor/pg_loss:np.float64(1.3680546544492245e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001169136107819213) - actor/ppo_kl:np.float64(0.00010978662939820083) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08973986911587417) - perf/mfu/actor:np.float64(0.07893086665760501) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.16036605834961) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:3 - critic/score/mean:0.8828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018441316206008196 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018441316206008196 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:434.76171875 - response_length/max:940.0 - response_length/min:178.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.76171875 - response_length_non_aborted/max:940.0 - response_length_non_aborted/min:178.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:402.625 - prompt_length/max:486.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.54812902212143e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1515968181192875) - timing_s/agent_loop/generate_sequences/max:np.float64(7.847456922754645) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.73277190943918) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.847456922754645) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:940 - timing_s/gen:13.57878121174872 - timing_s/reward:0.06386242620646954 - timing_s/old_log_prob:2.3055368904024363 - timing_s/adv:0.11865591630339622 - timing_s/update_actor:9.867850305512547 - timing_s/step:26.019996501505375 - timing_s/stop_profile:0.00011219270527362823 - timing_per_token_ms/update_actor:0.046031647496688206 - timing_per_token_ms/adv:0.0005535073135050741 - timing_per_token_ms/gen:0.12200272429894896 - perf/total_num_tokens:214371 - perf/time_per_step:26.019996501505375 - perf/throughput:1029.8377633697878 (TaskRunner pid=2146199) Training Progress: 98%|█████████▊| 98/100 [53:04<01:17, 38.89s/it] (TaskRunner pid=2146199) step:99 - global_seqlen/min:23965 - global_seqlen/max:49458 - global_seqlen/minmax_diff:25493 - global_seqlen/balanced_min:28283 - global_seqlen/balanced_max:34713 - global_seqlen/mean:30399.0 - actor/entropy:0.06078997254371643 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7543654441833496 - training/rollout_probs_diff_mean:0.002177275251597166 - training/rollout_probs_diff_std:0.011563776060938835 - training/rollout_actor_probs_pearson_corr:0.9960559606552124 - rollout_corr/rollout_is_mean:1.0000057220458984 - rollout_corr/rollout_is_ratio_fraction_high:2.109111301251687e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013357705029193312 - rollout_corr/rollout_is_max:2.3837571144104004 - rollout_corr/rollout_is_min:0.1393641233444214 - rollout_corr/rollout_is_std:0.030002489686012268 - rollout_corr/rollout_is_eff_sample_size:0.9991006601947391 - rollout_corr/rollout_is_seq_mean:0.9999766945838928 - rollout_corr/rollout_is_seq_std:0.0018581986660137773 - rollout_corr/rollout_is_seq_max:1.0069513320922852 - rollout_corr/rollout_is_seq_min:0.9903984069824219 - rollout_corr/rollout_is_seq_max_deviation:0.009601593017578125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0606072330847383) - rollout_corr/training_log_ppl:np.float64(0.05775477383576799) - rollout_corr/kl:np.float64(0.0008794172163975045) - rollout_corr/k3_kl:np.float64(0.0008313798509078651) - rollout_corr/rollout_ppl:np.float64(1.059665482956916) - rollout_corr/rollout_log_ppl:np.float64(0.05687535672041122) - rollout_corr/log_ppl_diff:np.float64(0.0008794171153567731) - rollout_corr/log_ppl_abs_diff:np.float64(0.001662463510001544) - rollout_corr/log_ppl_diff_max:np.float64(0.012757256627082825) - rollout_corr/log_ppl_diff_min:np.float64(-0.008459538221359253) - rollout_corr/ppl_ratio:np.float64(1.000882558291778) - rollout_corr/chi2_token:np.float64(0.0015434159431606531) - rollout_corr/chi2_seq:np.float64(1.0315078594721854) - actor/pg_loss:np.float64(-6.383901927620173e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000246376960831185) - actor/ppo_kl:np.float64(0.0002717288461084877) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17105350457131863) - perf/mfu/actor:np.float64(0.08388365746018137) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.13896942138672) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:3 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.07076156139373779 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.07076156139373779 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:555.625 - response_length/max:8192.0 - response_length/min:188.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:555.625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:188.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:394.34375 - prompt_length/max:480.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.264448165893555e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.3103966992348433) - timing_s/agent_loop/generate_sequences/max:np.float64(51.27737538330257) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.5419215327638085) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.27737538330257) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.0158349853009 - timing_s/reward:0.06930895708501339 - timing_s/old_log_prob:2.538815476000309 - timing_s/adv:0.12096652761101723 - timing_s/update_actor:10.44528903067112 - timing_s/step:70.277354253456 - timing_s/stop_profile:0.0001178886741399765 - timing_per_token_ms/update_actor:0.04295079209295997 - timing_per_token_ms/adv:0.0004974116237829255 - timing_per_token_ms/gen:0.4008424844298432 - perf/total_num_tokens:243192 - perf/time_per_step:70.277354253456 - perf/throughput:432.5575474905571 (TaskRunner pid=2146199) Training Progress: 99%|█████████▉| 99/100 [54:14<00:48, 48.32s/it] (TaskRunner pid=2146199) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2146199) validation generation end (TaskRunner pid=2146199) [prompt] system (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) ... (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2146199) (TaskRunner pid=2146199) A (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) user (TaskRunner pid=2146199) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2146199) (TaskRunner pid=2146199) Lattice Parameters: (TaskRunner pid=2146199) (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) α = 90.00º (TaskRunner pid=2146199) β = 90.00º (TaskRunner pid=2146199) γ = 90.00º (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) A: 1025.67 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) B: 1094.31 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) C: 1200.45 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) D: 1150.78 ų (TaskRunner pid=2146199) (TaskRunner pid=2146199) Please reason step by step. (TaskRunner pid=2146199) assistant (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) [response] (TaskRunner pid=2146199) To calculate the volume of the unit cell for a triclinic crystal structure (where all angles are 90 degrees, indicating an orthorhombic structure), the formula for the unit cell volume is: (TaskRunner pid=2146199) (TaskRunner pid=2146199) V = a × b × c (TaskRunner pid=2146199) (TaskRunner pid=2146199) Given: (TaskRunner pid=2146199) a = 7.82 Å (TaskRunner pid=2146199) b = 7.82 Å (TaskRunner pid=2146199) c = 17.88 Å (TaskRunner pid=2146199) (TaskRunner pid=2146199) Now, calculate the volume: (TaskRunner pid=2146199) (TaskRunner pid=2146199) V = 7.82 × 7.82 × 17.88 (TaskRunner pid=2146199) (TaskRunner pid=2146199) First, calculate 7.82 × 7.82: (TaskRunner pid=2146199) (TaskRunner pid=2146199) 7.82 × 7.82 = 61.1524 (TaskRunner pid=2146199) (TaskRunner pid=2146199) Next, multiply by 17.88: (TaskRunner pid=2146199) (TaskRunner pid=2146199) 61.1524 × 17.88 ≈ 1094.31 (TaskRunner pid=2146199) (TaskRunner pid=2146199) Therefore, the volume of the unit cell is approximately 1094.31 ų, which matches option B. (TaskRunner pid=2146199) (TaskRunner pid=2146199) (TaskRunner pid=2146199) B (TaskRunner pid=2146199) (TaskRunner pid=2146199) [ground_truth] B (TaskRunner pid=2146199) [score] 1.0 (TaskRunner pid=2146199) [acc] 1.0 (TaskRunner pid=2146199) [pred] B (TaskRunner pid=2146199) [incorrect_format] 1 (TaskRunner pid=2146199) [feedback] (TaskRunner pid=2146199) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2146577) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2146199) step:100 - global_seqlen/min:21007 - global_seqlen/max:33657 - global_seqlen/minmax_diff:12650 - global_seqlen/balanced_min:26409 - global_seqlen/balanced_max:26415 - global_seqlen/mean:26412.125 - actor/entropy:0.05979691818356514 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37930381298065186 - training/rollout_probs_diff_mean:0.0023399998899549246 - training/rollout_probs_diff_std:0.011555762961506844 - training/rollout_actor_probs_pearson_corr:0.9959689378738403 - rollout_corr/rollout_is_mean:0.999850869178772 - rollout_corr/rollout_is_ratio_fraction_high:4.497656846069731e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001169390743598342 - rollout_corr/rollout_is_max:2.267052173614502 - rollout_corr/rollout_is_min:0.28976842761039734 - rollout_corr/rollout_is_std:0.03037368506193161 - rollout_corr/rollout_is_eff_sample_size:0.9990782896419121 - rollout_corr/rollout_is_seq_mean:0.9998504519462585 - rollout_corr/rollout_is_seq_std:0.0016542637022212148 - rollout_corr/rollout_is_seq_max:1.0056191682815552 - rollout_corr/rollout_is_seq_min:0.9947155714035034 - rollout_corr/rollout_is_seq_max_deviation:0.005619168281555176 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0646160240285099) - rollout_corr/training_log_ppl:np.float64(0.06111104129377054) - rollout_corr/kl:np.float64(0.0009433553804747419) - rollout_corr/k3_kl:np.float64(0.000666843313986476) - rollout_corr/rollout_ppl:np.float64(1.0636031506583095) - rollout_corr/rollout_log_ppl:np.float64(0.060167685403939686) - rollout_corr/log_ppl_diff:np.float64(0.0009433558898308547) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016063227940321667) - rollout_corr/log_ppl_diff_max:np.float64(0.009745059534907341) - rollout_corr/log_ppl_diff_min:np.float64(-0.004364762455224991) - rollout_corr/ppl_ratio:np.float64(1.0009459347929806) - rollout_corr/chi2_token:np.float64(0.0007009264081716537) - rollout_corr/chi2_seq:np.float64(0.7943548788316548) - actor/pg_loss:np.float64(4.458578769117594e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022653777296000044) - actor/ppo_kl:np.float64(0.00029493772322253253) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14011954190209508) - perf/mfu/actor:np.float64(0.07842677559658048) - perf/max_memory_allocated_gb:np.float64(123.45145273208618) - perf/max_memory_reserved_gb:np.float64(129.208984375) - perf/cpu_memory_used_gb:np.float64(103.06706619262695) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7626329787234043) - val-aux/sciknoweval/reward/std@16:np.float64(0.06729335213589911) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7876382978723404) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.041836077372629385) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7377765957446808) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.07115878526747459) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7628297872340426) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.06820989645423031) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.801627659574468) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.023418987057881077) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7114680851063832) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.07037925820603312) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.770308510638298) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.04686638157004486) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8101914893617022) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.014096974337829643) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6826808510638298) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06494348791272907) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7723617021276596) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.026416313160677763) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8149574468085106) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.00848440146040565) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6536170212765957) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05400056483953881) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7719148936170213) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.018618881588821194) - val-aux/sciknoweval/score/mean@16:np.float64(0.7626329787234043) - val-aux/sciknoweval/score/std@16:np.float64(0.06729335213589911) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7876382978723404) - val-aux/sciknoweval/score/best@2/std:np.float64(0.041836077372629385) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7377765957446808) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.07115878526747459) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7628297872340426) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.06820989645423031) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.801627659574468) - val-aux/sciknoweval/score/best@4/std:np.float64(0.023418987057881077) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7114680851063832) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.07037925820603312) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.770308510638298) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.04686638157004486) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8101914893617022) - val-aux/sciknoweval/score/best@8/std:np.float64(0.014096974337829643) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6826808510638298) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06494348791272907) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7723617021276596) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.026416313160677763) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8149574468085106) - val-aux/sciknoweval/score/best@16/std:np.float64(0.00848440146040565) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6536170212765957) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05400056483953881) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7719148936170213) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.018618881588821194) - val-core/sciknoweval/acc/mean@16:np.float64(0.7626329787234043) - val-aux/sciknoweval/acc/std@16:np.float64(0.06729335213589911) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7876382978723404) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.041836077372629385) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7377765957446808) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.07115878526747459) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7628297872340426) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.06820989645423031) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.801627659574468) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.023418987057881077) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7114680851063832) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.07037925820603312) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.770308510638298) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.04686638157004486) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8101914893617022) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.014096974337829643) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6826808510638298) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06494348791272907) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7723617021276596) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.026416313160677763) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8149574468085106) - val-core/sciknoweval/acc/best@16/std:np.float64(0.00848440146040565) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6536170212765957) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05400056483953881) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7719148936170213) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.018618881588821194) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9966755319148937) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.009302503118891777) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9995851063829787) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.00298506157652289) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9942765957446809) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.011671809930422904) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.996872340425532) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.009074902857638772) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999787234042554) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00047528314688699314) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9895957446808511) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.014164656923311533) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9992021276595746) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.004446232243099193) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9830851063829789) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.01468444636139687) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9998617021276596) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0012050425965809124) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.976063829787234) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.01202692239227068) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000756730732973665 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:0.000756730732973665 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:434.25390625 - response_length/max:1110.0 - response_length/min:197.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:434.25390625 - response_length_non_aborted/max:1110.0 - response_length_non_aborted/min:197.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.125 - prompt_length/max:479.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.438768029212952e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.1825265772640705) - timing_s/agent_loop/generate_sequences/max:np.float64(8.524729896336794) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.727881735328992) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.524729896336794) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:1110 - timing_s/gen:14.453305700793862 - timing_s/reward:0.06965606473386288 - timing_s/old_log_prob:2.3261021357029676 - timing_s/adv:0.12102740071713924 - timing_s/update_actor:9.65417823009193 - timing_s/step:26.7111904527992 - timing_s/testing:94.38914828561246 - timing_s/save_checkpoint:6.867023587226868 - timing_s/stop_profile:0.00014218315482139587 - timing_per_token_ms/update_actor:0.04569008660838502 - timing_per_token_ms/adv:0.0005727833368061982 - timing_per_token_ms/gen:0.1300120150473051 - perf/total_num_tokens:211297 - perf/time_per_step:26.7111904527992 - perf/throughput:988.8037392669685 (TaskRunner pid=2146199) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2146199) "np.float64(0.7626329787234043), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2146199) "np.float64(0.06729335213589911), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2146199) "np.float64(0.7876382978723404), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2146199) "np.float64(0.041836077372629385), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2146199) "np.float64(0.7377765957446808), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2146199) "np.float64(0.07115878526747459), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2146199) "np.float64(0.7628297872340426), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2146199) "np.float64(0.06820989645423031), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2146199) "np.float64(0.801627659574468), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2146199) "np.float64(0.023418987057881077), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2146199) "np.float64(0.7114680851063832), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2146199) "np.float64(0.07037925820603312), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2146199) "np.float64(0.770308510638298), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2146199) "np.float64(0.04686638157004486), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2146199) "np.float64(0.8101914893617022), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2146199) "np.float64(0.014096974337829643), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2146199) "np.float64(0.6826808510638298), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2146199) "np.float64(0.06494348791272907), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2146199) "np.float64(0.7723617021276596), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2146199) "np.float64(0.026416313160677763), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2146199) "np.float64(0.8149574468085106), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2146199) "np.float64(0.00848440146040565), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2146199) "np.float64(0.6536170212765957), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2146199) "np.float64(0.05400056483953881), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2146199) "np.float64(0.7719148936170213), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2146199) "np.float64(0.018618881588821194), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2146199) "np.float64(0.7626329787234043), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2146199) "np.float64(0.06729335213589911), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2146199) "np.float64(0.7876382978723404), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2146199) "np.float64(0.041836077372629385), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2146199) "np.float64(0.7377765957446808), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2146199) "np.float64(0.07115878526747459), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2146199) "np.float64(0.7628297872340426), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2146199) "np.float64(0.06820989645423031), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2146199) "np.float64(0.801627659574468), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2146199) "np.float64(0.023418987057881077), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2146199) "np.float64(0.7114680851063832), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2146199) "np.float64(0.07037925820603312), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2146199) "np.float64(0.770308510638298), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2146199) "np.float64(0.04686638157004486), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2146199) "np.float64(0.8101914893617022), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2146199) "np.float64(0.014096974337829643), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2146199) "np.float64(0.6826808510638298), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2146199) "np.float64(0.06494348791272907), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2146199) "np.float64(0.7723617021276596), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2146199) "np.float64(0.026416313160677763), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2146199) "np.float64(0.8149574468085106), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2146199) "np.float64(0.00848440146040565), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2146199) "np.float64(0.6536170212765957), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2146199) "np.float64(0.05400056483953881), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2146199) "np.float64(0.7719148936170213), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2146199) "np.float64(0.018618881588821194), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2146199) "np.float64(0.7626329787234043), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2146199) "np.float64(0.06729335213589911), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2146199) "np.float64(0.7876382978723404), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2146199) "np.float64(0.041836077372629385), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2146199) "np.float64(0.7377765957446808), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2146199) "np.float64(0.07115878526747459), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2146199) "np.float64(0.7628297872340426), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2146199) "np.float64(0.06820989645423031), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2146199) "np.float64(0.801627659574468), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2146199) "np.float64(0.023418987057881077), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2146199) "np.float64(0.7114680851063832), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2146199) "np.float64(0.07037925820603312), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2146199) "np.float64(0.770308510638298), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2146199) "np.float64(0.04686638157004486), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2146199) "np.float64(0.8101914893617022), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2146199) "np.float64(0.014096974337829643), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2146199) "np.float64(0.6826808510638298), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2146199) "np.float64(0.06494348791272907), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2146199) "np.float64(0.7723617021276596), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2146199) "np.float64(0.026416313160677763), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2146199) "np.float64(0.8149574468085106), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2146199) "np.float64(0.00848440146040565), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2146199) "np.float64(0.6536170212765957), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2146199) "np.float64(0.05400056483953881), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2146199) "np.float64(0.7719148936170213), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2146199) 'np.float64(0.018618881588821194), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/mean@16': " (TaskRunner pid=2146199) 'np.float64(0.9966755319148937), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2146199) 'np.float64(0.009302503118891777), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2146199) 'np.float64(0.9995851063829787), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2146199) 'np.float64(0.00298506157652289), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2146199) 'np.float64(0.9942765957446809), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2146199) 'np.float64(0.011671809930422904), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': " (TaskRunner pid=2146199) 'np.float64(0.996872340425532), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2146199) 'np.float64(0.009074902857638772), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2146199) 'np.float64(0.9999787234042554), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2146199) 'np.float64(0.00047528314688699314), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2146199) 'np.float64(0.9895957446808511), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2146199) 'np.float64(0.014164656923311533), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2146199) 'np.float64(0.9992021276595746), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2146199) 'np.float64(0.004446232243099193), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2146199) 'np.float64(0.9830851063829789), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2146199) 'np.float64(0.01468444636139687), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2146199) 'np.float64(0.9998617021276596), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2146199) 'np.float64(0.0012050425965809124), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2146199) 'np.float64(0.976063829787234), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2146199) 'np.float64(0.01202692239227068), ' (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2146199) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2146199) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2146199) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2146199) Training Progress: 100%|██████████| 100/100 [56:22<00:00, 72.24s/it] Training Progress: 100%|██████████| 100/100 [56:22<00:00, 33.83s/it] (TaskRunner pid=2146199) wandb: updating run metadata (TaskRunner pid=2146199) wandb: updating run metadata; uploading output.log; uploading wandb-summary.json (TaskRunner pid=2146199) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2146199) wandb: uploading output.log (TaskRunner pid=2146199) wandb: uploading history steps 98-99, summary, console lines 910-1038 (TaskRunner pid=2146199) wandb: (TaskRunner pid=2146199) wandb: Run history: (TaskRunner pid=2146199) wandb: actor/entropy ▃▃▂▁▃▃▃▂▂▃▁▃▅▄▃▂▄▃▄▃▂▄▂▄▄▃▅▃▆▄▂▅▃▄▆▄▂█▃▆ (TaskRunner pid=2146199) wandb: actor/grad_norm ▇▆▇▆▅█▅▅▄▆▄▄▅▄▃▄▃▂▅▅▅▄▄▃▄▃▃▄▃▄▃▂▄▂▂▃▄▁▃▂ (TaskRunner pid=2146199) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2146199) wandb: actor/lr ▁▃▄▅▇███████████████████████████████████ (TaskRunner pid=2146199) wandb: actor/pg_clipfrac ▅▆▆▅▃▆█▃▄▃▃▃▅▁▃▆▃▆▃▃▂▂▃▃▃▃▆▄▄▂▃▃▄▁▁▁▂▃▃▃ (TaskRunner pid=2146199) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▁▁▁▁▄█▃▁▁▄▁▁▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2146199) wandb: actor/pg_loss ▁▆▅▃▆▃▂▇▃▆▃▃▂▅▂▄▃▄▃▅▆▄▄▃▂█▂▃▂▃▃▃▃▄▃▇▄▂▃▂ (TaskRunner pid=2146199) wandb: actor/ppo_kl ▁▁▁▂▂▁▂▃▁▂▂▂▂▂▁▂▁▂▂▁▂▁▁▂▂▁▂▂▃▁█▁▂▂▂▂▁▂▂▂ (TaskRunner pid=2146199) wandb: critic/advantages/max ██████████▅▇▃▇▆█▁▆████▆▅█████▅███▆██▇▆██ (TaskRunner pid=2146199) wandb: critic/advantages/mean █▇▆▅▅█▃▅▄▄▆▄▇▆▁▅▃▄▃▇▂▃▄▅▂▇▅▁▂▄▃▄▄▅▄▄▄▅▄▅ (TaskRunner pid=2146199) wandb: +204 ... (TaskRunner pid=2146199) wandb: (TaskRunner pid=2146199) wandb: Run summary: (TaskRunner pid=2146199) wandb: actor/entropy 0.0598 (TaskRunner pid=2146199) wandb: actor/grad_norm 0.14012 (TaskRunner pid=2146199) wandb: actor/kl_loss 0 (TaskRunner pid=2146199) wandb: actor/lr 0.0 (TaskRunner pid=2146199) wandb: actor/pg_clipfrac 0.00023 (TaskRunner pid=2146199) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2146199) wandb: actor/pg_loss 4e-05 (TaskRunner pid=2146199) wandb: actor/ppo_kl 0.00029 (TaskRunner pid=2146199) wandb: critic/advantages/max 0.375 (TaskRunner pid=2146199) wandb: critic/advantages/mean 0.00076 (TaskRunner pid=2146199) wandb: +204 ... (TaskRunner pid=2146199) wandb: (TaskRunner pid=2146199) wandb: 🚀 View run qwen3gen-material-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/lnzvk3fv (TaskRunner pid=2146199) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2146199) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2146199) wandb: Find logs at: ./wandb/run-20260702_125526-lnzvk3fv/logs (TaskRunner pid=2146199) Exception ignored in atexit callback: .teardown_atexit at 0x7d98b249b9c0> (TaskRunner pid=2146199) Traceback (most recent call last): (TaskRunner pid=2146199) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2146199) conn.teardown(hooks.exit_code) (TaskRunner pid=2146199) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2146199) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2146199) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2146199) return future.result() (TaskRunner pid=2146199) ^^^^^^^^^^^^^^^ (TaskRunner pid=2146199) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2146199) return self.__get_result() (TaskRunner pid=2146199) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2146199) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2146199) raise self._exception (TaskRunner pid=2146199) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2146199) return await fn() (TaskRunner pid=2146199) ^^^^^^^^^^ (TaskRunner pid=2146199) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2146199) await self._client.close() (TaskRunner pid=2146199) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2146199) await self._writer.wait_closed() (TaskRunner pid=2146199) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2146199) await self._protocol._get_close_waiter(self) (TaskRunner pid=2146199) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 13:51:56] Finished material grpo [2026-07-02 13:51:56] Starting material rlsd_tr Experiment: qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: material Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet Ray tmp: /tmp/ray_q3g_material_rlsd_tr_Qwen3_4B 2026-07-02 13:51:58,605 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/material Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_material_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/material +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_material_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/material', 'EXPERIMENT': 'qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 13:52:09,359 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2167046) TaskRunner hostname: mole-workspace-rw, PID: 2167046 (TaskRunner pid=2167046) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2167046) 'calculate_entropy': False, (TaskRunner pid=2167046) 'calculate_sum_pi_squared': False, (TaskRunner pid=2167046) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2167046) 'async_save': False, (TaskRunner pid=2167046) 'load_contents': ['model', (TaskRunner pid=2167046) 'optimizer', (TaskRunner pid=2167046) 'extra'], (TaskRunner pid=2167046) 'save_contents': ['model', (TaskRunner pid=2167046) 'optimizer', (TaskRunner pid=2167046) 'extra']}, (TaskRunner pid=2167046) 'clip_ratio': 0.2, (TaskRunner pid=2167046) 'clip_ratio_c': 3.0, (TaskRunner pid=2167046) 'clip_ratio_high': 0.28, (TaskRunner pid=2167046) 'clip_ratio_low': 0.2, (TaskRunner pid=2167046) 'data_loader_seed': 42, (TaskRunner pid=2167046) 'entropy_checkpointing': False, (TaskRunner pid=2167046) 'entropy_coeff': 0, (TaskRunner pid=2167046) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2167046) 'freeze_vision_tower': False, (TaskRunner pid=2167046) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2167046) 'dtype': 'bfloat16', (TaskRunner pid=2167046) 'entropy_checkpointing': False, (TaskRunner pid=2167046) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2167046) 'forward_only': False, (TaskRunner pid=2167046) 'forward_prefetch': False, (TaskRunner pid=2167046) 'fsdp_size': -1, (TaskRunner pid=2167046) 'full_determinism': False, (TaskRunner pid=2167046) 'model_dtype': 'fp32', (TaskRunner pid=2167046) 'offload_policy': False, (TaskRunner pid=2167046) 'optimizer_offload': False, (TaskRunner pid=2167046) 'param_offload': False, (TaskRunner pid=2167046) 'reshard_after_forward': True, (TaskRunner pid=2167046) 'seed': 42, (TaskRunner pid=2167046) 'strategy': 'fsdp', (TaskRunner pid=2167046) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2167046) 'use_orig_params': False, (TaskRunner pid=2167046) 'use_torch_compile': True, (TaskRunner pid=2167046) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2167046) 'grad_clip': 1.0, (TaskRunner pid=2167046) 'kl_loss_coef': 0.001, (TaskRunner pid=2167046) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2167046) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2167046) 'loss_scale_factor': None, (TaskRunner pid=2167046) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2167046) 'betas': [0.9, 0.999], (TaskRunner pid=2167046) 'clip_grad': 1.0, (TaskRunner pid=2167046) 'lr': 1e-06, (TaskRunner pid=2167046) 'lr_scheduler_type': 'constant', (TaskRunner pid=2167046) 'lr_warmup_steps': 10, (TaskRunner pid=2167046) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2167046) 'min_lr_ratio': 0.0, (TaskRunner pid=2167046) 'num_cycles': 0.5, (TaskRunner pid=2167046) 'optimizer': 'AdamW', (TaskRunner pid=2167046) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2167046) 'override_optimizer_config': None, (TaskRunner pid=2167046) 'total_training_steps': -1, (TaskRunner pid=2167046) 'warmup_style': None, (TaskRunner pid=2167046) 'weight_decay': 0.01}, (TaskRunner pid=2167046) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2167046) 'clip_cov_lb': 1.0, (TaskRunner pid=2167046) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2167046) 'clip_cov_ub': 5.0, (TaskRunner pid=2167046) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2167046) 'loss_mode': 'rlsd', (TaskRunner pid=2167046) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2167046) 'ppo_epochs': 1, (TaskRunner pid=2167046) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2167046) 'ppo_micro_batch_size': None, (TaskRunner pid=2167046) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2167046) 'ppo_mini_batch_size': 8, (TaskRunner pid=2167046) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2167046) 'all_ranks': False, (TaskRunner pid=2167046) 'enable': False, (TaskRunner pid=2167046) 'ranks': [], (TaskRunner pid=2167046) 'save_path': 'outputs/profile', (TaskRunner pid=2167046) 'tool': None, (TaskRunner pid=2167046) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2167046) 'analysis': True, (TaskRunner pid=2167046) 'contents': [], (TaskRunner pid=2167046) 'discrete': False, (TaskRunner pid=2167046) 'level': 'level0'}, (TaskRunner pid=2167046) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2167046) 'discrete': False}, (TaskRunner pid=2167046) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2167046) 'step_end': None, (TaskRunner pid=2167046) 'step_start': 0}, (TaskRunner pid=2167046) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2167046) 'stack_depth': 32, (TaskRunner pid=2167046) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2167046) 'rollout_n': 8, (TaskRunner pid=2167046) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2167046) 'mode': 'disabled', (TaskRunner pid=2167046) 'record_file': None, (TaskRunner pid=2167046) 'replay_file': None}, (TaskRunner pid=2167046) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2167046) 'alpha': 0.5, (TaskRunner pid=2167046) 'distillation_add_tail': True, (TaskRunner pid=2167046) 'distillation_topk': 100, (TaskRunner pid=2167046) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2167046) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2167046) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2167046) 'enable': False, (TaskRunner pid=2167046) 'loss_weight': 0.0, (TaskRunner pid=2167046) 'mask': 'all'}, (TaskRunner pid=2167046) 'feedback_template': '\n' (TaskRunner pid=2167046) 'The ' (TaskRunner pid=2167046) 'following ' (TaskRunner pid=2167046) 'is ' (TaskRunner pid=2167046) 'feedback ' (TaskRunner pid=2167046) 'from ' (TaskRunner pid=2167046) 'your ' (TaskRunner pid=2167046) 'unsuccessful ' (TaskRunner pid=2167046) 'earlier ' (TaskRunner pid=2167046) 'attempt:\n' (TaskRunner pid=2167046) '\n' (TaskRunner pid=2167046) '{feedback_raw}', (TaskRunner pid=2167046) 'full_logit_distillation': True, (TaskRunner pid=2167046) 'include_environment_feedback': True, (TaskRunner pid=2167046) 'is_clip': 2, (TaskRunner pid=2167046) 'max_reprompt_len': 10240, (TaskRunner pid=2167046) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2167046) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2167046) '\n' (TaskRunner pid=2167046) 'Correctly ' (TaskRunner pid=2167046) 'solve ' (TaskRunner pid=2167046) 'the ' (TaskRunner pid=2167046) 'original ' (TaskRunner pid=2167046) 'question.', (TaskRunner pid=2167046) 'reprompt_truncation': 'right', (TaskRunner pid=2167046) 'solution_template': '\n' (TaskRunner pid=2167046) 'Correct ' (TaskRunner pid=2167046) 'solution:\n' (TaskRunner pid=2167046) '\n' (TaskRunner pid=2167046) '{successful_previous_attempt}', (TaskRunner pid=2167046) 'srpo_dynamic_weighting': False, (TaskRunner pid=2167046) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2167046) 'success_reward_threshold': 0.5, (TaskRunner pid=2167046) 'teacher_regularization': 'trust-region', (TaskRunner pid=2167046) 'teacher_update_rate': 0.1, (TaskRunner pid=2167046) 'token_reweight_decay_steps': 0, (TaskRunner pid=2167046) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2167046) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2167046) 'shuffle': False, (TaskRunner pid=2167046) 'strategy': 'fsdp', (TaskRunner pid=2167046) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2167046) 'tau_neg': 1.05, (TaskRunner pid=2167046) 'tau_pos': 1.0, (TaskRunner pid=2167046) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2167046) 'use_dynamic_bsz': False, (TaskRunner pid=2167046) 'use_fused_kernels': False, (TaskRunner pid=2167046) 'use_kl_loss': False, (TaskRunner pid=2167046) 'use_prefix_grouper': False, (TaskRunner pid=2167046) 'use_remove_padding': True, (TaskRunner pid=2167046) 'use_torch_compile': True}, (TaskRunner pid=2167046) 'hybrid_engine': True, (TaskRunner pid=2167046) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2167046) 'custom_chat_template': None, (TaskRunner pid=2167046) 'enable_activation_offload': False, (TaskRunner pid=2167046) 'enable_gradient_checkpointing': True, (TaskRunner pid=2167046) 'exclude_modules': None, (TaskRunner pid=2167046) 'external_lib': None, (TaskRunner pid=2167046) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2167046) 'hf_config_path': None, (TaskRunner pid=2167046) 'lora_adapter_path': None, (TaskRunner pid=2167046) 'lora_alpha': 16, (TaskRunner pid=2167046) 'lora_rank': 0, (TaskRunner pid=2167046) 'override_config': {}, (TaskRunner pid=2167046) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2167046) 'target_modules': 'all-linear', (TaskRunner pid=2167046) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2167046) 'num_shards': 4}, (TaskRunner pid=2167046) 'tokenizer_path': None, (TaskRunner pid=2167046) 'trust_remote_code': True, (TaskRunner pid=2167046) 'use_fused_kernels': False, (TaskRunner pid=2167046) 'use_liger': False, (TaskRunner pid=2167046) 'use_remove_padding': True, (TaskRunner pid=2167046) 'use_shm': False}, (TaskRunner pid=2167046) 'nccl_timeout': 600, (TaskRunner pid=2167046) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2167046) 'entropy_checkpointing': False, (TaskRunner pid=2167046) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2167046) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2167046) 'dtype': 'bfloat16', (TaskRunner pid=2167046) 'entropy_checkpointing': False, (TaskRunner pid=2167046) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2167046) 'forward_only': True, (TaskRunner pid=2167046) 'forward_prefetch': False, (TaskRunner pid=2167046) 'fsdp_size': -1, (TaskRunner pid=2167046) 'full_determinism': False, (TaskRunner pid=2167046) 'model_dtype': 'fp32', (TaskRunner pid=2167046) 'offload_policy': False, (TaskRunner pid=2167046) 'optimizer_offload': False, (TaskRunner pid=2167046) 'param_offload': False, (TaskRunner pid=2167046) 'reshard_after_forward': True, (TaskRunner pid=2167046) 'seed': 42, (TaskRunner pid=2167046) 'strategy': 'fsdp', (TaskRunner pid=2167046) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2167046) 'use_orig_params': False, (TaskRunner pid=2167046) 'use_torch_compile': True, (TaskRunner pid=2167046) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2167046) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2167046) 'log_prob_micro_batch_size': None, (TaskRunner pid=2167046) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2167046) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2167046) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2167046) 'all_ranks': False, (TaskRunner pid=2167046) 'enable': False, (TaskRunner pid=2167046) 'ranks': [], (TaskRunner pid=2167046) 'save_path': 'outputs/profile', (TaskRunner pid=2167046) 'tool': None, (TaskRunner pid=2167046) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2167046) 'analysis': True, (TaskRunner pid=2167046) 'contents': [], (TaskRunner pid=2167046) 'discrete': False, (TaskRunner pid=2167046) 'level': 'level0'}, (TaskRunner pid=2167046) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2167046) 'discrete': False}, (TaskRunner pid=2167046) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2167046) 'step_end': None, (TaskRunner pid=2167046) 'step_start': 0}, (TaskRunner pid=2167046) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2167046) 'stack_depth': 32, (TaskRunner pid=2167046) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2167046) 'rollout_n': 8, (TaskRunner pid=2167046) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2167046) 'mode': 'disabled', (TaskRunner pid=2167046) 'record_file': None, (TaskRunner pid=2167046) 'replay_file': None}, (TaskRunner pid=2167046) 'strategy': 'fsdp', (TaskRunner pid=2167046) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2167046) 'use_torch_compile': True}, (TaskRunner pid=2167046) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2167046) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2167046) 'agent_loop_config_path': None, (TaskRunner pid=2167046) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2167046) 'name': None, (TaskRunner pid=2167046) 'path': None}, (TaskRunner pid=2167046) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2167046) 'num_workers': 8}, (TaskRunner pid=2167046) 'calculate_log_probs': True, (TaskRunner pid=2167046) 'cudagraph_capture_sizes': None, (TaskRunner pid=2167046) 'data_parallel_size': 1, (TaskRunner pid=2167046) 'disable_log_stats': True, (TaskRunner pid=2167046) 'do_sample': True, (TaskRunner pid=2167046) 'dtype': 'bfloat16', (TaskRunner pid=2167046) 'enable_chunked_prefill': True, (TaskRunner pid=2167046) 'enable_prefix_caching': True, (TaskRunner pid=2167046) 'enable_rollout_routing_replay': False, (TaskRunner pid=2167046) 'enforce_eager': False, (TaskRunner pid=2167046) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2167046) 'expert_parallel_size': 1, (TaskRunner pid=2167046) 'free_cache_engine': True, (TaskRunner pid=2167046) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2167046) 'ignore_eos': False, (TaskRunner pid=2167046) 'layered_summon': False, (TaskRunner pid=2167046) 'load_format': 'dummy', (TaskRunner pid=2167046) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2167046) 'log_prob_micro_batch_size': None, (TaskRunner pid=2167046) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2167046) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2167046) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2167046) 'max_model_len': 10240, (TaskRunner pid=2167046) 'max_num_batched_tokens': 10240, (TaskRunner pid=2167046) 'max_num_seqs': 1024, (TaskRunner pid=2167046) 'mode': 'async', (TaskRunner pid=2167046) 'multi_stage_wake_up': False, (TaskRunner pid=2167046) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2167046) 'enable': False, (TaskRunner pid=2167046) 'format': 'hermes', (TaskRunner pid=2167046) 'interaction_config_path': None, (TaskRunner pid=2167046) 'max_assistant_turns': None, (TaskRunner pid=2167046) 'max_parallel_calls': 1, (TaskRunner pid=2167046) 'max_tool_response_length': 256, (TaskRunner pid=2167046) 'max_user_turns': None, (TaskRunner pid=2167046) 'num_repeat_rollouts': None, (TaskRunner pid=2167046) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2167046) 'tool_config_path': None, (TaskRunner pid=2167046) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2167046) 'use_inference_chat_template': False}, (TaskRunner pid=2167046) 'n': 8, (TaskRunner pid=2167046) 'name': 'vllm', (TaskRunner pid=2167046) 'over_sample_rate': 0, (TaskRunner pid=2167046) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2167046) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2167046) 'all_ranks': False, (TaskRunner pid=2167046) 'enable': False, (TaskRunner pid=2167046) 'ranks': [], (TaskRunner pid=2167046) 'save_path': 'outputs/profile', (TaskRunner pid=2167046) 'tool': None, (TaskRunner pid=2167046) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2167046) 'analysis': True, (TaskRunner pid=2167046) 'contents': [], (TaskRunner pid=2167046) 'discrete': False, (TaskRunner pid=2167046) 'level': 'level0'}, (TaskRunner pid=2167046) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2167046) 'discrete': False}, (TaskRunner pid=2167046) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2167046) 'step_end': None, (TaskRunner pid=2167046) 'step_start': 0}, (TaskRunner pid=2167046) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2167046) 'stack_depth': 32, (TaskRunner pid=2167046) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2167046) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2167046) 'enable': False, (TaskRunner pid=2167046) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2167046) 'port': 9090, (TaskRunner pid=2167046) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2167046) 'prompt_length': 2048, (TaskRunner pid=2167046) 'quantization': None, (TaskRunner pid=2167046) 'quantization_config_file': None, (TaskRunner pid=2167046) 'response_length': 8192, (TaskRunner pid=2167046) 'scheduling_policy': 'fcfs', (TaskRunner pid=2167046) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2167046) 'skip_rollout': False, (TaskRunner pid=2167046) 'skip_tokenizer_init': True, (TaskRunner pid=2167046) 'temperature': 1.0, (TaskRunner pid=2167046) 'tensor_model_parallel_size': 2, (TaskRunner pid=2167046) 'top_k': -1, (TaskRunner pid=2167046) 'top_p': 1.0, (TaskRunner pid=2167046) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2167046) 'backend': None, (TaskRunner pid=2167046) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2167046) 'token2text': False}, (TaskRunner pid=2167046) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2167046) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2167046) 'do_sample': True, (TaskRunner pid=2167046) 'n': 16, (TaskRunner pid=2167046) 'temperature': 0.6, (TaskRunner pid=2167046) 'top_k': -1, (TaskRunner pid=2167046) 'top_p': 0.95}}}, (TaskRunner pid=2167046) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2167046) 'adv_estimator': 'grpo', (TaskRunner pid=2167046) 'gamma': 1.0, (TaskRunner pid=2167046) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2167046) 'horizon': 10000, (TaskRunner pid=2167046) 'kl_coef': 0.001, (TaskRunner pid=2167046) 'target_kl': 0.1, (TaskRunner pid=2167046) 'type': 'fixed'}, (TaskRunner pid=2167046) 'kl_penalty': 'kl', (TaskRunner pid=2167046) 'lam': 1.0, (TaskRunner pid=2167046) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2167046) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2167046) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2167046) 'loss_type': 'ppo_clip', (TaskRunner pid=2167046) 'rollout_is': 'token', (TaskRunner pid=2167046) 'rollout_is_batch_normalize': False, (TaskRunner pid=2167046) 'rollout_is_threshold': 2.0, (TaskRunner pid=2167046) 'rollout_rs': None, (TaskRunner pid=2167046) 'rollout_rs_threshold': None}, (TaskRunner pid=2167046) 'use_kl_in_reward': False, (TaskRunner pid=2167046) 'use_pf_ppo': False}, (TaskRunner pid=2167046) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2167046) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2167046) 'async_save': False, (TaskRunner pid=2167046) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2167046) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2167046) 'cliprange_value': 0.5, (TaskRunner pid=2167046) 'data_loader_seed': 42, (TaskRunner pid=2167046) 'enable': None, (TaskRunner pid=2167046) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2167046) 'forward_micro_batch_size': None, (TaskRunner pid=2167046) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2167046) 'grad_clip': 1.0, (TaskRunner pid=2167046) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2167046) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2167046) 'enable_activation_offload': False, (TaskRunner pid=2167046) 'enable_gradient_checkpointing': True, (TaskRunner pid=2167046) 'external_lib': None, (TaskRunner pid=2167046) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2167046) 'dtype': 'bfloat16', (TaskRunner pid=2167046) 'entropy_checkpointing': False, (TaskRunner pid=2167046) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2167046) 'forward_only': False, (TaskRunner pid=2167046) 'forward_prefetch': False, (TaskRunner pid=2167046) 'fsdp_size': -1, (TaskRunner pid=2167046) 'full_determinism': False, (TaskRunner pid=2167046) 'model_dtype': 'fp32', (TaskRunner pid=2167046) 'offload_policy': False, (TaskRunner pid=2167046) 'optimizer_offload': False, (TaskRunner pid=2167046) 'param_offload': False, (TaskRunner pid=2167046) 'reshard_after_forward': True, (TaskRunner pid=2167046) 'seed': 42, (TaskRunner pid=2167046) 'strategy': 'fsdp', (TaskRunner pid=2167046) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2167046) 'use_orig_params': False, (TaskRunner pid=2167046) 'use_torch_compile': True, (TaskRunner pid=2167046) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2167046) 'lora_alpha': 16, (TaskRunner pid=2167046) 'lora_rank': 0, (TaskRunner pid=2167046) 'override_config': {}, (TaskRunner pid=2167046) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2167046) 'target_modules': 'all-linear', (TaskRunner pid=2167046) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2167046) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2167046) 'trust_remote_code': True, (TaskRunner pid=2167046) 'use_remove_padding': False, (TaskRunner pid=2167046) 'use_shm': False}, (TaskRunner pid=2167046) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2167046) 'betas': [0.9, 0.999], (TaskRunner pid=2167046) 'clip_grad': 1.0, (TaskRunner pid=2167046) 'lr': 1e-05, (TaskRunner pid=2167046) 'lr_scheduler_type': 'constant', (TaskRunner pid=2167046) 'lr_warmup_steps': -1, (TaskRunner pid=2167046) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2167046) 'min_lr_ratio': 0.0, (TaskRunner pid=2167046) 'num_cycles': 0.5, (TaskRunner pid=2167046) 'optimizer': 'AdamW', (TaskRunner pid=2167046) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2167046) 'override_optimizer_config': None, (TaskRunner pid=2167046) 'total_training_steps': -1, (TaskRunner pid=2167046) 'warmup_style': None, (TaskRunner pid=2167046) 'weight_decay': 0.01}, (TaskRunner pid=2167046) 'ppo_epochs': 1, (TaskRunner pid=2167046) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2167046) 'ppo_micro_batch_size': None, (TaskRunner pid=2167046) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2167046) 'ppo_mini_batch_size': 8, (TaskRunner pid=2167046) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2167046) 'all_ranks': False, (TaskRunner pid=2167046) 'enable': False, (TaskRunner pid=2167046) 'ranks': [], (TaskRunner pid=2167046) 'save_path': 'outputs/profile', (TaskRunner pid=2167046) 'tool': None, (TaskRunner pid=2167046) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2167046) 'analysis': True, (TaskRunner pid=2167046) 'contents': [], (TaskRunner pid=2167046) 'discrete': False, (TaskRunner pid=2167046) 'level': 'level0'}, (TaskRunner pid=2167046) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2167046) 'discrete': False}, (TaskRunner pid=2167046) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2167046) 'step_end': None, (TaskRunner pid=2167046) 'step_start': 0}, (TaskRunner pid=2167046) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2167046) 'stack_depth': 32, (TaskRunner pid=2167046) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2167046) 'rollout_n': 8, (TaskRunner pid=2167046) 'shuffle': False, (TaskRunner pid=2167046) 'strategy': 'fsdp', (TaskRunner pid=2167046) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2167046) 'use_dynamic_bsz': False}, (TaskRunner pid=2167046) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2167046) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2167046) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2167046) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2167046) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2167046) 'dataloader_num_workers': 8, (TaskRunner pid=2167046) 'filter_overlong_prompts': True, (TaskRunner pid=2167046) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2167046) 'image_key': 'images', (TaskRunner pid=2167046) 'image_patch_size': 14, (TaskRunner pid=2167046) 'max_prompt_length': 2048, (TaskRunner pid=2167046) 'max_response_length': 8192, (TaskRunner pid=2167046) 'prompt_key': 'prompt', (TaskRunner pid=2167046) 'return_full_prompt': False, (TaskRunner pid=2167046) 'return_multi_modal_inputs': True, (TaskRunner pid=2167046) 'return_raw_chat': True, (TaskRunner pid=2167046) 'return_raw_input_ids': False, (TaskRunner pid=2167046) 'reward_fn_key': 'data_source', (TaskRunner pid=2167046) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2167046) 'seed': None, (TaskRunner pid=2167046) 'shuffle': True, (TaskRunner pid=2167046) 'tokenizer': None, (TaskRunner pid=2167046) 'tool_config_path': None, (TaskRunner pid=2167046) 'train_batch_size': 32, (TaskRunner pid=2167046) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/train.parquet'], (TaskRunner pid=2167046) 'train_max_samples': 3200, (TaskRunner pid=2167046) 'truncation': 'error', (TaskRunner pid=2167046) 'trust_remote_code': True, (TaskRunner pid=2167046) 'use_shm': False, (TaskRunner pid=2167046) 'val_batch_size': None, (TaskRunner pid=2167046) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/material/test.parquet'], (TaskRunner pid=2167046) 'val_max_samples': -1, (TaskRunner pid=2167046) 'validation_shuffle': False, (TaskRunner pid=2167046) 'video_key': 'videos'}, (TaskRunner pid=2167046) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2167046) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2167046) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2167046) 'cuda-memory-usage': 'true', (TaskRunner pid=2167046) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2167046) 'discrete': False, (TaskRunner pid=2167046) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2167046) 'capture-range-end': None, (TaskRunner pid=2167046) 'cuda-graph-trace': 'graph', (TaskRunner pid=2167046) 'cuda-memory-usage': 'true', (TaskRunner pid=2167046) 'kill': 'none', (TaskRunner pid=2167046) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2167046) 'torch_memory': {'context': 'all', (TaskRunner pid=2167046) 'kw_args': {}, (TaskRunner pid=2167046) 'stack_depth': 32, (TaskRunner pid=2167046) 'stacks': 'all', (TaskRunner pid=2167046) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2167046) 'profile_continuous_steps': False, (TaskRunner pid=2167046) 'save_path': 'outputs/profile', (TaskRunner pid=2167046) 'steps': None, (TaskRunner pid=2167046) 'tool': None}, (TaskRunner pid=2167046) 'max_model_len': 10240, (TaskRunner pid=2167046) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_material_rlsd_tr_Qwen3_4B', (TaskRunner pid=2167046) 'include_dashboard': False, (TaskRunner pid=2167046) 'num_cpus': None, (TaskRunner pid=2167046) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2167046) 'TASK': 'datasets/sciknoweval/material', (TaskRunner pid=2167046) 'USER': 'root'}}}, (TaskRunner pid=2167046) 'timeline_json_file': None}, (TaskRunner pid=2167046) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2167046) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2167046) 'name': 'custom_reward_manager', (TaskRunner pid=2167046) 'path': None}, (TaskRunner pid=2167046) 'name': 'naive', (TaskRunner pid=2167046) 'source': 'register'}, (TaskRunner pid=2167046) 'reward_model': {'enable': False, (TaskRunner pid=2167046) 'enable_resource_pool': False, (TaskRunner pid=2167046) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2167046) 'launch_reward_fn_async': False, (TaskRunner pid=2167046) 'max_length': None, (TaskRunner pid=2167046) 'micro_batch_size': None, (TaskRunner pid=2167046) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2167046) 'model': {'external_lib': None, (TaskRunner pid=2167046) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2167046) 'forward_prefetch': False, (TaskRunner pid=2167046) 'fsdp_size': -1, (TaskRunner pid=2167046) 'param_offload': False, (TaskRunner pid=2167046) 'reshard_after_forward': True, (TaskRunner pid=2167046) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2167046) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2167046) 'override_config': {}, (TaskRunner pid=2167046) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2167046) 'trust_remote_code': False, (TaskRunner pid=2167046) 'use_fused_kernels': False, (TaskRunner pid=2167046) 'use_remove_padding': False, (TaskRunner pid=2167046) 'use_shm': False}, (TaskRunner pid=2167046) 'n_gpus_per_node': 8, (TaskRunner pid=2167046) 'nnodes': 0, (TaskRunner pid=2167046) 'num_workers': 1, (TaskRunner pid=2167046) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2167046) 'all_ranks': False, (TaskRunner pid=2167046) 'enable': False, (TaskRunner pid=2167046) 'ranks': [], (TaskRunner pid=2167046) 'save_path': 'outputs/profile', (TaskRunner pid=2167046) 'tool': None, (TaskRunner pid=2167046) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2167046) 'analysis': True, (TaskRunner pid=2167046) 'contents': [], (TaskRunner pid=2167046) 'discrete': False, (TaskRunner pid=2167046) 'level': 'level0'}, (TaskRunner pid=2167046) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2167046) 'discrete': False}, (TaskRunner pid=2167046) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2167046) 'step_end': None, (TaskRunner pid=2167046) 'step_start': 0}, (TaskRunner pid=2167046) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2167046) 'stack_depth': 32, (TaskRunner pid=2167046) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2167046) 'reward_loop_class_name': None, (TaskRunner pid=2167046) 'reward_loop_module_path': None, (TaskRunner pid=2167046) 'reward_loop_source': 'register', (TaskRunner pid=2167046) 'reward_manager': 'naive', (TaskRunner pid=2167046) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2167046) 'cudagraph_capture_sizes': None, (TaskRunner pid=2167046) 'data_parallel_size': 1, (TaskRunner pid=2167046) 'disable_log_stats': True, (TaskRunner pid=2167046) 'dtype': 'bfloat16', (TaskRunner pid=2167046) 'enable_chunked_prefill': True, (TaskRunner pid=2167046) 'enable_prefix_caching': True, (TaskRunner pid=2167046) 'enforce_eager': True, (TaskRunner pid=2167046) 'engine_kwargs': {}, (TaskRunner pid=2167046) 'expert_parallel_size': 1, (TaskRunner pid=2167046) 'free_cache_engine': True, (TaskRunner pid=2167046) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2167046) 'limit_images': None, (TaskRunner pid=2167046) 'load_format': 'auto', (TaskRunner pid=2167046) 'max_model_len': None, (TaskRunner pid=2167046) 'max_num_batched_tokens': 8192, (TaskRunner pid=2167046) 'max_num_seqs': 1024, (TaskRunner pid=2167046) 'name': '???', (TaskRunner pid=2167046) 'prompt_length': 2048, (TaskRunner pid=2167046) 'response_length': 2048, (TaskRunner pid=2167046) 'skip_tokenizer_init': False, (TaskRunner pid=2167046) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2167046) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2167046) 'memory_limit_mb': 1024, (TaskRunner pid=2167046) 'url': None}, (TaskRunner pid=2167046) 'strategy': 'fsdp', (TaskRunner pid=2167046) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2167046) 'use_dynamic_bsz': False, (TaskRunner pid=2167046) 'use_reward_loop': False}, (TaskRunner pid=2167046) 'trainer': {'balance_batch': True, (TaskRunner pid=2167046) 'critic_warmup': 0, (TaskRunner pid=2167046) 'default_hdfs_dir': None, (TaskRunner pid=2167046) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2167046) 'del_local_ckpt_after_load': False, (TaskRunner pid=2167046) 'device': 'cuda', (TaskRunner pid=2167046) 'esi_redundant_time': 0, (TaskRunner pid=2167046) 'experiment_name': 'qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2167046) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2167046) 'log_val_generations': 0, (TaskRunner pid=2167046) 'logger': ['console', 'wandb'], (TaskRunner pid=2167046) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2167046) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2167046) 'n_gpus_per_node': 8, (TaskRunner pid=2167046) 'nnodes': 1, (TaskRunner pid=2167046) 'project_name': 'SDPO-root', (TaskRunner pid=2167046) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2167046) 'resume_from_path': None, (TaskRunner pid=2167046) 'resume_mode': 'auto', (TaskRunner pid=2167046) 'rollout_data_dir': None, (TaskRunner pid=2167046) 'save_freq': 10, (TaskRunner pid=2167046) 'test_freq': 10, (TaskRunner pid=2167046) 'total_epochs': 30, (TaskRunner pid=2167046) 'total_training_steps': 100, (TaskRunner pid=2167046) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2167046) 'val_before_train': False, (TaskRunner pid=2167046) 'val_only': False, (TaskRunner pid=2167046) 'validation_data_dir': None}, (TaskRunner pid=2167046) 'transfer_queue': {'enable': False}, (TaskRunner pid=2167046) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/material', (TaskRunner pid=2167046) 'dir': '/users/root/SDPO', (TaskRunner pid=2167046) 'log_dir': '/users/root/output', (TaskRunner pid=2167046) 'task': 'datasets/sciknoweval/material'}} (TaskRunner pid=2167046) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2167046) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2167046) use_critic=need_critic(config), (TaskRunner pid=2167046) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2167046) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2167046) Using dataset class: RLHFDataset (TaskRunner pid=2167046) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2167046) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2167046) dataset len: 841 (TaskRunner pid=2167046) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2167046) WARNING:2026-07-02 13:52:18,686:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2167046) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/841 [00:00 (TaskRunner pid=2167046) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2167046) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2167046) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2167046) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2167046) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2167046) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 94/94 [00:00<00:00, 135.84 examples/s] (TaskRunner pid=2167046) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2167046) self.use_critic = need_critic(self.config) (WorkerDict pid=2167440) [W702 13:52:26.332882838 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:44967 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2167440) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2167440) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2167440) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2167444) [W702 13:52:28.689030292 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:44967 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2167440) Model config after override: Qwen3Config { (WorkerDict pid=2167440) "architectures": [ (WorkerDict pid=2167440) "Qwen3ForCausalLM" (WorkerDict pid=2167440) ], (WorkerDict pid=2167440) "attention_bias": false, (WorkerDict pid=2167440) "attention_dropout": 0.0, (WorkerDict pid=2167440) "dtype": "bfloat16", (WorkerDict pid=2167440) "eos_token_id": 151645, (WorkerDict pid=2167440) "head_dim": 128, (WorkerDict pid=2167440) "hidden_act": "silu", (WorkerDict pid=2167440) "hidden_size": 2560, (WorkerDict pid=2167440) "initializer_range": 0.02, (WorkerDict pid=2167440) "intermediate_size": 9728, (WorkerDict pid=2167440) "layer_types": [ (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention" (WorkerDict pid=2167440) ], (WorkerDict pid=2167440) "max_position_embeddings": 40960, (WorkerDict pid=2167440) "max_window_layers": 36, (WorkerDict pid=2167440) "model_type": "qwen3", (WorkerDict pid=2167440) "num_attention_heads": 32, (WorkerDict pid=2167440) "num_hidden_layers": 36, (WorkerDict pid=2167440) "num_key_value_heads": 8, (WorkerDict pid=2167440) "pad_token_id": 151643, (WorkerDict pid=2167440) "rms_norm_eps": 1e-06, (WorkerDict pid=2167440) "rope_scaling": null, (WorkerDict pid=2167440) "rope_theta": 1000000, (WorkerDict pid=2167440) "sliding_window": null, (WorkerDict pid=2167440) "tie_word_embeddings": true, (WorkerDict pid=2167440) "transformers_version": "4.57.1", (WorkerDict pid=2167440) "use_cache": true, (WorkerDict pid=2167440) "use_sliding_window": false, (WorkerDict pid=2167440) "vocab_size": 151936 (WorkerDict pid=2167440) } (WorkerDict pid=2167440) (WorkerDict pid=2167441) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2167441) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2167441) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2167441) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2167440) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2167440) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2167440) Actor use_remove_padding=True (WorkerDict pid=2167440) Actor use_fused_kernels=False (WorkerDict pid=2167440) Actor use_prefix_grouper=False (WorkerDict pid=2167440) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2167440) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2167440) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2167440) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2167440) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:06<00:03, 3.06s/it] [repeated 14x across cluster] (WorkerDict pid=2167440) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.05s/it] [repeated 6x across cluster] (WorkerDict pid=2167443) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2167443) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2167440) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2167440) reference model: Qwen/Qwen3-4B (WorkerDict pid=2167440) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2167440) warnings.warn( (WorkerDict pid=2167444) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2167444) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2167440) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2167440) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2167440) Model config after override: Qwen3Config { (WorkerDict pid=2167440) "architectures": [ (WorkerDict pid=2167440) "Qwen3ForCausalLM" (WorkerDict pid=2167440) ], (WorkerDict pid=2167440) "attention_bias": false, (WorkerDict pid=2167440) "attention_dropout": 0.0, (WorkerDict pid=2167440) "dtype": "bfloat16", (WorkerDict pid=2167440) "eos_token_id": 151645, (WorkerDict pid=2167440) "head_dim": 128, (WorkerDict pid=2167440) "hidden_act": "silu", (WorkerDict pid=2167440) "hidden_size": 2560, (WorkerDict pid=2167440) "initializer_range": 0.02, (WorkerDict pid=2167440) "intermediate_size": 9728, (WorkerDict pid=2167440) "layer_types": [ (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention", (WorkerDict pid=2167440) "full_attention" (WorkerDict pid=2167440) ], (WorkerDict pid=2167440) "max_position_embeddings": 40960, (WorkerDict pid=2167440) "max_window_layers": 36, (WorkerDict pid=2167440) "model_type": "qwen3", (WorkerDict pid=2167440) "num_attention_heads": 32, (WorkerDict pid=2167440) "num_hidden_layers": 36, (WorkerDict pid=2167440) "num_key_value_heads": 8, (WorkerDict pid=2167440) "pad_token_id": 151643, (WorkerDict pid=2167440) "rms_norm_eps": 1e-06, (WorkerDict pid=2167440) "rope_scaling": null, (WorkerDict pid=2167440) "rope_theta": 1000000, (WorkerDict pid=2167440) "sliding_window": null, (WorkerDict pid=2167440) "tie_word_embeddings": true, (WorkerDict pid=2167440) "transformers_version": "4.57.1", (WorkerDict pid=2167440) "use_cache": true, (WorkerDict pid=2167440) "use_sliding_window": false, (WorkerDict pid=2167440) "vocab_size": 151936 (WorkerDict pid=2167440) } (WorkerDict pid=2167440) (WorkerDict pid=2167440) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2167440) Ref use_remove_padding=True (WorkerDict pid=2167440) Ref use_fused_kernels=False (WorkerDict pid=2167440) Ref use_prefix_grouper=False (TaskRunner pid=2167046) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2167046) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2167441) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.95s/it] [repeated 14x across cluster] (WorkerDict pid=2167441) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.98s/it] [repeated 6x across cluster] (vLLMHttpServer pid=2168339) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2168339) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2168339) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2168339) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2168337) ['serve', (vLLMHttpServer pid=2168337) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2168337) '--dtype', (vLLMHttpServer pid=2168337) 'bfloat16', (vLLMHttpServer pid=2168337) '--load_format', (vLLMHttpServer pid=2168337) 'dummy', (vLLMHttpServer pid=2168337) '--trust_remote_code', (vLLMHttpServer pid=2168337) '--max_model_len', (vLLMHttpServer pid=2168337) '40960', (vLLMHttpServer pid=2168337) '--max_num_seqs', (vLLMHttpServer pid=2168337) '1024', (vLLMHttpServer pid=2168337) '--enable_chunked_prefill', (vLLMHttpServer pid=2168337) '--max_num_batched_tokens', (vLLMHttpServer pid=2168337) '10240', (vLLMHttpServer pid=2168337) '--enable_prefix_caching', (vLLMHttpServer pid=2168337) '--enable_sleep_mode', (vLLMHttpServer pid=2168337) '--logprobs_mode', (vLLMHttpServer pid=2168337) 'processed_logprobs', (vLLMHttpServer pid=2168337) '--disable_custom_all_reduce', (vLLMHttpServer pid=2168337) '--gpu_memory_utilization', (vLLMHttpServer pid=2168337) '0.8', (vLLMHttpServer pid=2168337) '--disable_log_stats', (vLLMHttpServer pid=2168337) '--tensor_parallel_size', (vLLMHttpServer pid=2168337) '2', (vLLMHttpServer pid=2168337) '--seed', (vLLMHttpServer pid=2168337) '0', (vLLMHttpServer pid=2168337) '--override_generation_config', (vLLMHttpServer pid=2168337) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2168337) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2168337) '--hf_overrides', (vLLMHttpServer pid=2168337) '{}', (vLLMHttpServer pid=2168337) '--scheduling_policy', (vLLMHttpServer pid=2168337) 'fcfs'] (WorkerDict pid=2167441) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2167441) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (vLLMHttpServer pid=2168339) WARNING 07-02 13:53:26 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2167445) WARNING 07-02 13:53:34 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2168340) WARNING 07-02 13:53:27 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2167442) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2167444) [rank4]:W0702 13:53:44.996000 2167444 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2167444) [rank4]:W0702 13:53:44.996000 2167444 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2167444) [rank4]:W0702 13:53:44.996000 2167444 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2167444) [rank4]:W0702 13:53:44.996000 2167444 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2167444) [rank4]:W0702 13:53:44.996000 2167444 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2167444) [rank4]:W0702 13:53:44.996000 2167444 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2167440.127442857162432.tmp' (vLLMHttpServer pid=2168340) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2168340) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2168340) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2168340) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2167440) 2026-07-02 13:53:48,700 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2167440) 2026-07-02 13:53:48,716 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2167440) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ V = a \times b \times c \times \sin(\alpha) \times \sin(\beta) \times \sin(\gamma) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) However, since all angles (α, β, γ) are 90.00º, their sines are all 1. Therefore, the formula simplifies to: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ V = a \times b \times c $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given: (TaskRunner pid=2167046) - $ a = 7.82 $ Å (TaskRunner pid=2167046) - $ b = 7.82 $ Å (TaskRunner pid=2167046) - $ c = 17.88 $ Å (TaskRunner pid=2167046) (TaskRunner pid=2167046) Substituting the values: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then multiply by 17.88: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ 61.1524 \times 17.88 \approx 1094.31 $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2167046) step:10 - global_seqlen/min:20683 - global_seqlen/max:25751 - global_seqlen/minmax_diff:5068 - global_seqlen/balanced_min:23251 - global_seqlen/balanced_max:23253 - global_seqlen/mean:23252.5 - actor/entropy:0.04874306544661522 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5441901683807373 - training/rollout_probs_diff_mean:0.0026878858916461468 - training/rollout_probs_diff_std:0.014487761072814465 - training/rollout_actor_probs_pearson_corr:0.9926190972328186 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:0.9999591708183289 - rollout_corr/rollout_is_ratio_fraction_high:0.00016809951921459287 - rollout_corr/rollout_is_ratio_fraction_low:0.0003722203546203673 - rollout_corr/rollout_is_max:5.771607875823975 - rollout_corr/rollout_is_min:0.23010946810245514 - rollout_corr/rollout_is_std:0.03833707422018051 - rollout_corr/rollout_is_eff_sample_size:0.9985320690303704 - rollout_corr/rollout_is_seq_mean:1.0000007152557373 - rollout_corr/rollout_is_seq_std:0.003206924069672823 - rollout_corr/rollout_is_seq_max:1.0279439687728882 - rollout_corr/rollout_is_seq_min:0.9925442934036255 - rollout_corr/rollout_is_seq_max_deviation:0.027943968772888184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.048803678713739) - rollout_corr/training_log_ppl:np.float64(0.04719654364271264) - rollout_corr/kl:np.float64(0.0009922661836050395) - rollout_corr/k3_kl:np.float64(0.0009911187928040022) - rollout_corr/rollout_ppl:np.float64(1.0477598579600453) - rollout_corr/rollout_log_ppl:np.float64(0.0462042771250708) - rollout_corr/log_ppl_diff:np.float64(0.0009922665176418377) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020914054821332684) - rollout_corr/log_ppl_diff_max:np.float64(0.013107702136039734) - rollout_corr/log_ppl_diff_min:np.float64(-0.0085487961769104) - rollout_corr/ppl_ratio:np.float64(1.0009965328499675) - rollout_corr/chi2_token:np.float64(0.002523774513974786) - rollout_corr/chi2_seq:np.float64(0.6901092522311956) - actor/pg_loss:np.float64(0.001417467137798667) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004598129285113828) - actor/ppo_kl:np.float64(8.220813472037403e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(172479.47193897865) - self_distillation/token_reweight_w_std:np.float64(2368412.845991519) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999711983837187) - self_distillation/token_reweight_w_clip_frac:np.float64(0.018695179103815462) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.4023687466979027) - perf/mfu/actor:np.float64(0.04252209567923851) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.220703125) - perf/cpu_memory_used_gb:np.float64(141.84356307983398) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6529255319148937) - val-aux/sciknoweval/reward/std@16:np.float64(0.12541629540699634) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7040851063829787) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.09844906330661858) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6004680851063829) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.11257283375619549) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6526063829787234) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.12594630096934759) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7411914893617022) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06750000467366356) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5533297872340426) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.08969014630553164) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6605106382978723) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.09894316736071386) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7673297872340427) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.044519346897205984) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5161702127659574) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06201496457679105) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6653191489361702) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.06961725285691082) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7854255319148936) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02629508396493057) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.49265957446808506) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0343499741330905) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6698404255319149) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04941944187100074) - val-aux/sciknoweval/score/mean@16:np.float64(0.6529255319148937) - val-aux/sciknoweval/score/std@16:np.float64(0.12541629540699634) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7040851063829787) - val-aux/sciknoweval/score/best@2/std:np.float64(0.09844906330661858) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6004680851063829) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.11257283375619549) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6526063829787234) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.12594630096934759) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7411914893617022) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06750000467366356) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5533297872340426) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.08969014630553164) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6605106382978723) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.09894316736071386) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7673297872340427) - val-aux/sciknoweval/score/best@8/std:np.float64(0.044519346897205984) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5161702127659574) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06201496457679105) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6653191489361702) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.06961725285691082) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7854255319148936) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02629508396493057) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.49265957446808506) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0343499741330905) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6698404255319149) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04941944187100074) - val-core/sciknoweval/acc/mean@16:np.float64(0.6529255319148937) - val-aux/sciknoweval/acc/std@16:np.float64(0.12541629540699634) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7040851063829787) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.09844906330661858) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6004680851063829) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.11257283375619549) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6526063829787234) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.12594630096934759) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7411914893617022) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06750000467366356) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5533297872340426) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.08969014630553164) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6605106382978723) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.09894316736071386) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7673297872340427) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.044519346897205984) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5161702127659574) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06201496457679105) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6653191489361702) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.06961725285691082) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7854255319148936) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02629508396493057) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.49265957446808506) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0343499741330905) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6698404255319149) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04941944187100074) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9993351063829787) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0025751219057230166) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999468085106383) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0007503582967907386) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9988404255319148) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.003315310792311214) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.999404255319149) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0024459758065021133) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9977553191489362) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.004340621084455887) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9998617021276596) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0012050425965809124) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9958510638297873) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.005188829105641361) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9931489361702127) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.005093778770605872) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013102756347507238 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0013102756347507238 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:325.328125 - response_length/max:768.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:325.328125 - response_length_non_aborted/max:768.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.3125 - prompt_length/max:483.0 - prompt_length/min:267.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.082576096057892e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6101906280964613) - timing_s/agent_loop/generate_sequences/max:np.float64(6.086158093065023) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.493253128573997) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.086158093065023) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:768 - timing_s/gen:12.023752644658089 - timing_s/reward:0.05113005079329014 - timing_s/old_log_prob:2.298787211999297 - timing_s/adv:0.5260595064610243 - timing_s/update_actor:15.555556382983923 - timing_s/step:30.489348156377673 - timing_s/testing:82.0245217550546 - timing_s/save_checkpoint:6.857228877022862 - timing_s/stop_profile:9.977072477340698e-05 - timing_per_token_ms/gen:0.1443704990713473 - timing_per_token_ms/adv:0.002827972833356759 - timing_per_token_ms/update_actor:0.08362303184057587 - perf/total_num_tokens:186020 - perf/time_per_step:30.489348156377673 - perf/throughput:762.6433953503893 (TaskRunner pid=2167046) Training Progress: 10%|█ | 10/100 [07:01<1:27:30, 58.34s/it] (TaskRunner pid=2167046) step:11 - global_seqlen/min:21959 - global_seqlen/max:24923 - global_seqlen/minmax_diff:2964 - global_seqlen/balanced_min:23564 - global_seqlen/balanced_max:23566 - global_seqlen/mean:23565.25 - actor/entropy:0.04310470446944237 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.725770115852356 - training/rollout_probs_diff_mean:0.0022230474278330803 - training/rollout_probs_diff_std:0.013385867699980736 - training/rollout_actor_probs_pearson_corr:0.9929795861244202 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.0000650882720947 - rollout_corr/rollout_is_ratio_fraction_high:0.00010156867210753262 - rollout_corr/rollout_is_ratio_fraction_low:0.00024827898596413434 - rollout_corr/rollout_is_max:2.600924015045166 - rollout_corr/rollout_is_min:0.21203798055648804 - rollout_corr/rollout_is_std:0.032979775220155716 - rollout_corr/rollout_is_eff_sample_size:0.9989137540740176 - rollout_corr/rollout_is_seq_mean:1.0000079870224 - rollout_corr/rollout_is_seq_std:0.0018282465171068907 - rollout_corr/rollout_is_seq_max:1.0058741569519043 - rollout_corr/rollout_is_seq_min:0.99213707447052 - rollout_corr/rollout_is_seq_max_deviation:0.00786292552947998 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.04387602256611) - rollout_corr/training_log_ppl:np.float64(0.04242584969688323) - rollout_corr/kl:np.float64(0.0006134525538925573) - rollout_corr/k3_kl:np.float64(0.0007203111015670061) - rollout_corr/rollout_ppl:np.float64(1.043228325434029) - rollout_corr/rollout_log_ppl:np.float64(0.04181239714671392) - rollout_corr/log_ppl_diff:np.float64(0.0006134525501693133) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016334005340468138) - rollout_corr/log_ppl_diff_max:np.float64(0.010004661977291107) - rollout_corr/log_ppl_diff_min:np.float64(-0.004983656108379364) - rollout_corr/ppl_ratio:np.float64(1.0006159681361169) - rollout_corr/chi2_token:np.float64(0.0017383343074470758) - rollout_corr/chi2_seq:np.float64(0.5499970901291817) - actor/pg_loss:np.float64(0.0008666964713484049) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003046969727620308) - actor/ppo_kl:np.float64(3.5258178954222785e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(184026.21509693912) - self_distillation/token_reweight_w_std:np.float64(2792648.148677664) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008155833929777) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014882214221870527) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.21856637299060822) - perf/mfu/actor:np.float64(0.04311457286710895) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.220703125) - perf/cpu_memory_used_gb:np.float64(144.24859619140625) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006627355702221394 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.006627355702221394 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:346.1328125 - response_length/max:644.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:346.1328125 - response_length_non_aborted/max:644.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.28125 - prompt_length/max:481.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.9384310841560364e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7554168365895748) - timing_s/agent_loop/generate_sequences/max:np.float64(5.804619351401925) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8684470643129316) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.804619351401925) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:644 - timing_s/gen:11.557333318516612 - timing_s/reward:0.06140909530222416 - timing_s/old_log_prob:2.5711327120661736 - timing_s/adv:0.5249299556016922 - timing_s/update_actor:15.628794183954597 - timing_s/step:30.43499557301402 - timing_s/stop_profile:0.00010865740478038788 - timing_per_token_ms/gen:0.13042922151581776 - timing_per_token_ms/adv:0.0027844493247562207 - timing_per_token_ms/update_actor:0.08290169945128206 - perf/total_num_tokens:188522 - perf/time_per_step:30.43499557301402 - perf/throughput:774.2813677585924 (TaskRunner pid=2167046) Training Progress: 11%|█ | 11/100 [07:31<1:13:53, 49.81s/it] (TaskRunner pid=2167046) step:12 - global_seqlen/min:20706 - global_seqlen/max:24659 - global_seqlen/minmax_diff:3953 - global_seqlen/balanced_min:23407 - global_seqlen/balanced_max:23571 - global_seqlen/mean:23429.625 - actor/entropy:0.048118073493242264 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5477700233459473 - training/rollout_probs_diff_mean:0.0025818103458732367 - training/rollout_probs_diff_std:0.014044265262782574 - training/rollout_actor_probs_pearson_corr:0.9929377436637878 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999388456344604 - rollout_corr/rollout_is_ratio_fraction_high:9.061562013812363e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0004077702760696411 - rollout_corr/rollout_is_max:2.6784048080444336 - rollout_corr/rollout_is_min:0.13162191212177277 - rollout_corr/rollout_is_std:0.03624316677451134 - rollout_corr/rollout_is_eff_sample_size:0.9986879182401984 - rollout_corr/rollout_is_seq_mean:0.9999709129333496 - rollout_corr/rollout_is_seq_std:0.0021563658956438303 - rollout_corr/rollout_is_seq_max:1.0135271549224854 - rollout_corr/rollout_is_seq_min:0.9925267696380615 - rollout_corr/rollout_is_seq_max_deviation:0.013527154922485352 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0476039084605873) - rollout_corr/training_log_ppl:np.float64(0.04610199074159027) - rollout_corr/kl:np.float64(0.0009057767044708953) - rollout_corr/k3_kl:np.float64(0.0008639268171179992) - rollout_corr/rollout_ppl:np.float64(1.0466419314034283) - rollout_corr/rollout_log_ppl:np.float64(0.0451962135030044) - rollout_corr/log_ppl_diff:np.float64(0.0009057772385858698) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020111543035454815) - rollout_corr/log_ppl_diff_max:np.float64(0.009585428982973099) - rollout_corr/log_ppl_diff_min:np.float64(-0.008911248296499252) - rollout_corr/ppl_ratio:np.float64(1.000909530557692) - rollout_corr/chi2_token:np.float64(0.0016000857576727867) - rollout_corr/chi2_seq:np.float64(0.8010658347047865) - actor/pg_loss:np.float64(0.0020140354754403234) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008085124986791925) - actor/ppo_kl:np.float64(0.00010460224394570616) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(478855.19562057476) - self_distillation/token_reweight_w_std:np.float64(5985952.214456502) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002396942581981) - self_distillation/token_reweight_w_clip_frac:np.float64(0.030733069819689263) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.40229838341474533) - perf/mfu/actor:np.float64(0.042521912948479346) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.494140625) - perf/cpu_memory_used_gb:np.float64(144.2531280517578) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012176473392173648 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012176473392173648 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.86328125 - response_length/max:1112.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.86328125 - response_length_non_aborted/max:1112.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:387.3125 - prompt_length/max:482.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001245979219675064 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5674352757632732) - timing_s/agent_loop/generate_sequences/max:np.float64(7.914562286809087) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.760728602836025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.914562286809087) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:1112 - timing_s/gen:13.942733470350504 - timing_s/reward:0.05999826639890671 - timing_s/old_log_prob:2.3878198470920324 - timing_s/adv:0.5685920994728804 - timing_s/update_actor:15.464109597727656 - timing_s/step:32.510761769488454 - timing_s/stop_profile:0.00010918453335762024 - timing_per_token_ms/gen:0.1579286795078496 - timing_per_token_ms/adv:0.003033510456702147 - timing_per_token_ms/update_actor:0.0825029721865355 - perf/total_num_tokens:187437 - perf/time_per_step:32.510761769488454 - perf/throughput:720.6729010572999 (TaskRunner pid=2167046) Training Progress: 12%|█▏ | 12/100 [08:04<1:05:21, 44.56s/it] (TaskRunner pid=2167046) step:13 - global_seqlen/min:22261 - global_seqlen/max:24593 - global_seqlen/minmax_diff:2332 - global_seqlen/balanced_min:23464 - global_seqlen/balanced_max:23466 - global_seqlen/mean:23465.25 - actor/entropy:0.040193114429712296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39976027607917786 - training/rollout_probs_diff_mean:0.002327559981495142 - training/rollout_probs_diff_std:0.013716135174036026 - training/rollout_actor_probs_pearson_corr:0.9918900728225708 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.51953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0001914501190186 - rollout_corr/rollout_is_ratio_fraction_high:0.0001476632314734161 - rollout_corr/rollout_is_ratio_fraction_low:0.0002707159146666527 - rollout_corr/rollout_is_max:2.9459996223449707 - rollout_corr/rollout_is_min:0.25255417823791504 - rollout_corr/rollout_is_std:0.03525106608867645 - rollout_corr/rollout_is_eff_sample_size:0.9987594991657962 - rollout_corr/rollout_is_seq_mean:1.0001559257507324 - rollout_corr/rollout_is_seq_std:0.002297038910910487 - rollout_corr/rollout_is_seq_max:1.0091631412506104 - rollout_corr/rollout_is_seq_min:0.9913782477378845 - rollout_corr/rollout_is_seq_max_deviation:0.009163141250610352 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0394970341585577) - rollout_corr/training_log_ppl:np.float64(0.03850088814215269) - rollout_corr/kl:np.float64(0.0007251129889169761) - rollout_corr/k3_kl:np.float64(0.0006720473286687678) - rollout_corr/rollout_ppl:np.float64(1.0387373333796859) - rollout_corr/rollout_log_ppl:np.float64(0.037775775064801564) - rollout_corr/log_ppl_diff:np.float64(0.000725113077351125) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018534923983679619) - rollout_corr/log_ppl_diff_max:np.float64(0.012402225285768509) - rollout_corr/log_ppl_diff_min:np.float64(-0.005973838269710541) - rollout_corr/ppl_ratio:np.float64(1.0007284900639206) - rollout_corr/chi2_token:np.float64(0.0012406953610479832) - rollout_corr/chi2_seq:np.float64(1.1534826629795134) - actor/pg_loss:np.float64(0.0009124996140599251) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002558234486969013) - actor/ppo_kl:np.float64(0.00024511930615922495) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51953125) - self_distillation/token_reweight_w_mean:np.float64(293645.92851710087) - self_distillation/token_reweight_w_std:np.float64(2976888.573851983) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000577387399971) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014476351476332638) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.33747463673353195) - perf/mfu/actor:np.float64(0.04315192332532921) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.494140625) - perf/cpu_memory_used_gb:np.float64(144.07625579833984) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.51953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010813255794346333 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010813255794346333 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:317.4453125 - response_length/max:600.0 - response_length/min:158.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:317.4453125 - response_length_non_aborted/max:600.0 - response_length_non_aborted/min:158.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:415.84375 - prompt_length/max:493.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.387863636016846e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8269872777163982) - timing_s/agent_loop/generate_sequences/max:np.float64(5.382273999974132) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.526476628081582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.382273999974132) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:600 - timing_s/gen:11.363034663721919 - timing_s/reward:0.060489051043987274 - timing_s/old_log_prob:2.4501777328550816 - timing_s/adv:0.5203315634280443 - timing_s/update_actor:15.524447416886687 - timing_s/step:30.005078058689833 - timing_s/stop_profile:0.00011765211820602417 - timing_per_token_ms/gen:0.1398251995142116 - timing_per_token_ms/adv:0.002771819836929312 - timing_per_token_ms/update_actor:0.08269913711172205 - perf/total_num_tokens:187722 - perf/time_per_step:30.005078058689833 - perf/throughput:782.0426247217904 (TaskRunner pid=2167046) Training Progress: 13%|█▎ | 13/100 [08:34<58:14, 40.17s/it] (TaskRunner pid=2167046) step:14 - global_seqlen/min:18580 - global_seqlen/max:25429 - global_seqlen/minmax_diff:6849 - global_seqlen/balanced_min:22314 - global_seqlen/balanced_max:22321 - global_seqlen/mean:22318.875 - actor/entropy:0.046363964676856995 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7826330661773682 - training/rollout_probs_diff_mean:0.0022935778833925724 - training/rollout_probs_diff_std:0.013024184852838516 - training/rollout_actor_probs_pearson_corr:0.9938106536865234 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0002796649932861 - rollout_corr/rollout_is_ratio_fraction_high:0.0001367223885608837 - rollout_corr/rollout_is_ratio_fraction_low:0.000198868932784535 - rollout_corr/rollout_is_max:3.9407498836517334 - rollout_corr/rollout_is_min:0.12595464289188385 - rollout_corr/rollout_is_std:0.034748680889606476 - rollout_corr/rollout_is_eff_sample_size:0.9987944610216803 - rollout_corr/rollout_is_seq_mean:1.0002996921539307 - rollout_corr/rollout_is_seq_std:0.0024711701553314924 - rollout_corr/rollout_is_seq_max:1.015850305557251 - rollout_corr/rollout_is_seq_min:0.9948000311851501 - rollout_corr/rollout_is_seq_max_deviation:0.015850305557250977 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0458477013744414) - rollout_corr/training_log_ppl:np.float64(0.0445621470498736) - rollout_corr/kl:np.float64(0.0007553838675917746) - rollout_corr/k3_kl:np.float64(0.0007893300585628538) - rollout_corr/rollout_ppl:np.float64(1.0450514145195484) - rollout_corr/rollout_log_ppl:np.float64(0.04380676286928065) - rollout_corr/log_ppl_diff:np.float64(0.0007553841805929551) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019627007295639487) - rollout_corr/log_ppl_diff_max:np.float64(0.007131712511181831) - rollout_corr/log_ppl_diff_min:np.float64(-0.006304092705249786) - rollout_corr/ppl_ratio:np.float64(1.0007586732972413) - rollout_corr/chi2_token:np.float64(0.001714975107461214) - rollout_corr/chi2_seq:np.float64(0.7236321275122464) - actor/pg_loss:np.float64(0.0008898881496861577) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003190960019310296) - actor/ppo_kl:np.float64(0.0003813289751271043) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(175243.05724862264) - self_distillation/token_reweight_w_std:np.float64(1823834.6288890508) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001764104235917) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011729434947483242) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.306204192340374) - perf/mfu/actor:np.float64(0.04095481831414559) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.494140625) - perf/cpu_memory_used_gb:np.float64(144.13796997070312) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.009107575751841068 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.009107575751841068 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:314.27734375 - response_length/max:838.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.27734375 - response_length_non_aborted/max:838.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.1875 - prompt_length/max:519.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.609200060367584e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.740596003830433) - timing_s/agent_loop/generate_sequences/max:np.float64(6.187471503391862) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4451819638779853) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.187471503391862) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:271 - timing_s/agent_loop/slowest/response_length:838 - timing_s/gen:12.253078617155552 - timing_s/reward:0.08061863295733929 - timing_s/old_log_prob:2.397874131798744 - timing_s/adv:0.5357093457132578 - timing_s/update_actor:15.62681182846427 - timing_s/step:30.94428632222116 - timing_s/stop_profile:0.00011233426630496979 - timing_per_token_ms/gen:0.15229729186695112 - timing_per_token_ms/adv:0.0030003155720956915 - timing_per_token_ms/update_actor:0.08752015854553752 - perf/total_num_tokens:178551 - perf/time_per_step:30.94428632222116 - perf/throughput:721.2599692103018 (TaskRunner pid=2167046) Training Progress: 14%|█▍ | 14/100 [09:05<53:36, 37.40s/it] (TaskRunner pid=2167046) step:15 - global_seqlen/min:20380 - global_seqlen/max:25231 - global_seqlen/minmax_diff:4851 - global_seqlen/balanced_min:23489 - global_seqlen/balanced_max:23499 - global_seqlen/mean:23496.25 - actor/entropy:0.04732014238834381 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.583829402923584 - training/rollout_probs_diff_mean:0.0023648759815841913 - training/rollout_probs_diff_std:0.01362634263932705 - training/rollout_actor_probs_pearson_corr:0.9935072660446167 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.000017762184143 - rollout_corr/rollout_is_ratio_fraction_high:7.937048212625086e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00026078871451318264 - rollout_corr/rollout_is_max:3.049744129180908 - rollout_corr/rollout_is_min:0.09667123854160309 - rollout_corr/rollout_is_std:0.035375967621803284 - rollout_corr/rollout_is_eff_sample_size:0.9987501050707064 - rollout_corr/rollout_is_seq_mean:1.000028133392334 - rollout_corr/rollout_is_seq_std:0.002376183634623885 - rollout_corr/rollout_is_seq_max:1.0091273784637451 - rollout_corr/rollout_is_seq_min:0.9939660429954529 - rollout_corr/rollout_is_seq_max_deviation:0.009127378463745117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0488063767552376) - rollout_corr/training_log_ppl:np.float64(0.047080223926968756) - rollout_corr/kl:np.float64(0.0009711041951301347) - rollout_corr/k3_kl:np.float64(0.0009545547938358823) - rollout_corr/rollout_ppl:np.float64(1.0477678030729294) - rollout_corr/rollout_log_ppl:np.float64(0.04610911908275739) - rollout_corr/log_ppl_diff:np.float64(0.0009711048442113679) - rollout_corr/log_ppl_abs_diff:np.float64(0.002235111896879971) - rollout_corr/log_ppl_diff_max:np.float64(0.014504663646221161) - rollout_corr/log_ppl_diff_min:np.float64(-0.0077231526374816895) - rollout_corr/ppl_ratio:np.float64(1.0009760304819793) - rollout_corr/chi2_token:np.float64(0.002052654279395938) - rollout_corr/chi2_seq:np.float64(1.0901849407237023) - actor/pg_loss:np.float64(0.0008747054962441325) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000255831333106471) - actor/ppo_kl:np.float64(0.000288242356635493) - actor/pg_clipfrac_lower:np.float64(5.612428140011616e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(126971.70744604967) - self_distillation/token_reweight_w_std:np.float64(1583958.2053749266) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001212873030454) - self_distillation/token_reweight_w_clip_frac:np.float64(0.013588594767497852) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.2117880955338478) - perf/mfu/actor:np.float64(0.04322908957344593) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.1304931640625) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024647368118166924 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024647368118166924 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:344.5078125 - response_length/max:1091.0 - response_length/min:158.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.5078125 - response_length_non_aborted/max:1091.0 - response_length_non_aborted/min:158.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:389.75 - prompt_length/max:512.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001931600272655487 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8088673129677773) - timing_s/agent_loop/generate_sequences/max:np.float64(7.8003514017909765) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8534513168633566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.8003514017909765) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:1091 - timing_s/gen:13.576189883053303 - timing_s/reward:0.06108715012669563 - timing_s/old_log_prob:2.514427835121751 - timing_s/adv:0.5808148588985205 - timing_s/update_actor:15.497010707855225 - timing_s/step:32.31879555620253 - timing_s/stop_profile:0.00010493770241737366 - timing_per_token_ms/gen:0.1539355271679854 - timing_per_token_ms/adv:0.0030899338133666037 - timing_per_token_ms/update_actor:0.0824440639881642 - perf/total_num_tokens:187970 - perf/time_per_step:32.31879555620253 - perf/throughput:727.0150262604903 (TaskRunner pid=2167046) Training Progress: 15%|█▌ | 15/100 [09:37<50:50, 35.88s/it] (TaskRunner pid=2167046) step:16 - global_seqlen/min:20902 - global_seqlen/max:26195 - global_seqlen/minmax_diff:5293 - global_seqlen/balanced_min:23678 - global_seqlen/balanced_max:23682 - global_seqlen/mean:23680.125 - actor/entropy:0.052205078303813934 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5838291645050049 - training/rollout_probs_diff_mean:0.0023996636737138033 - training/rollout_probs_diff_std:0.013024316169321537 - training/rollout_actor_probs_pearson_corr:0.9943579435348511 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.0000131130218506 - rollout_corr/rollout_is_ratio_fraction_high:9.346558363176882e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017524797294754535 - rollout_corr/rollout_is_max:3.5453145503997803 - rollout_corr/rollout_is_min:0.32816171646118164 - rollout_corr/rollout_is_std:0.03295627236366272 - rollout_corr/rollout_is_eff_sample_size:0.9989150625306705 - rollout_corr/rollout_is_seq_mean:1.000024676322937 - rollout_corr/rollout_is_seq_std:0.00202034879475832 - rollout_corr/rollout_is_seq_max:1.0080302953720093 - rollout_corr/rollout_is_seq_min:0.9921471476554871 - rollout_corr/rollout_is_seq_max_deviation:0.008030295372009277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0531891910359263) - rollout_corr/training_log_ppl:np.float64(0.0511211731009098) - rollout_corr/kl:np.float64(0.0006379728563885578) - rollout_corr/k3_kl:np.float64(0.0006680498469080476) - rollout_corr/rollout_ppl:np.float64(1.0525041660293937) - rollout_corr/rollout_log_ppl:np.float64(0.050483200158851105) - rollout_corr/log_ppl_diff:np.float64(0.0006379729420586955) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017212007733178325) - rollout_corr/log_ppl_diff_max:np.float64(0.012218199670314789) - rollout_corr/log_ppl_diff_min:np.float64(-0.008264167234301567) - rollout_corr/ppl_ratio:np.float64(1.000640834448859) - rollout_corr/chi2_token:np.float64(0.0014848981518298388) - rollout_corr/chi2_seq:np.float64(0.45312996907159686) - actor/pg_loss:np.float64(0.0012663383968174458) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036590168474504026) - actor/ppo_kl:np.float64(8.058341451189222e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(145555.51019261987) - self_distillation/token_reweight_w_std:np.float64(1861382.0687996645) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992707001511008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.022003042249707505) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.2867910601198673) - perf/mfu/actor:np.float64(0.04326631512717655) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.0705108642578) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011858446523547173 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011858446523547173 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.34765625 - response_length/max:646.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.34765625 - response_length_non_aborted/max:646.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:405.65625 - prompt_length/max:495.0 - prompt_length/min:268.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.343718945980072e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8071682453155518) - timing_s/agent_loop/generate_sequences/max:np.float64(5.703692192211747) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.66322832651349) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.703692192211747) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:446 - timing_s/agent_loop/slowest/response_length:646 - timing_s/gen:12.136053482070565 - timing_s/reward:0.05776057951152325 - timing_s/old_log_prob:2.4096784368157387 - timing_s/adv:0.5424105860292912 - timing_s/update_actor:15.675222979858518 - timing_s/step:30.901057101786137 - timing_s/stop_profile:3.7867575883865356e-05 - timing_per_token_ms/gen:0.14178792053170897 - timing_per_token_ms/adv:0.002863216442213096 - timing_per_token_ms/update_actor:0.08274461695123293 - perf/total_num_tokens:189441 - perf/time_per_step:30.901057101786137 - perf/throughput:766.3208712245396 (TaskRunner pid=2167046) Training Progress: 16%|█▌ | 16/100 [10:08<48:08, 34.39s/it] (TaskRunner pid=2167046) step:17 - global_seqlen/min:19849 - global_seqlen/max:25601 - global_seqlen/minmax_diff:5752 - global_seqlen/balanced_min:22440 - global_seqlen/balanced_max:22447 - global_seqlen/mean:22445.25 - actor/entropy:0.057849083095788956 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6970136761665344 - training/rollout_probs_diff_mean:0.002556848106905818 - training/rollout_probs_diff_std:0.013555520214140415 - training/rollout_actor_probs_pearson_corr:0.9945673942565918 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:1.0001786947250366 - rollout_corr/rollout_is_ratio_fraction_high:0.00012861074355896562 - rollout_corr/rollout_is_ratio_fraction_low:0.00023149934713728726 - rollout_corr/rollout_is_max:2.902588129043579 - rollout_corr/rollout_is_min:0.17272382974624634 - rollout_corr/rollout_is_std:0.03630724921822548 - rollout_corr/rollout_is_eff_sample_size:0.9986839946654996 - rollout_corr/rollout_is_seq_mean:1.0002074241638184 - rollout_corr/rollout_is_seq_std:0.002327009802684188 - rollout_corr/rollout_is_seq_max:1.0122935771942139 - rollout_corr/rollout_is_seq_min:0.993233323097229 - rollout_corr/rollout_is_seq_max_deviation:0.012293577194213867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0583903319202363) - rollout_corr/training_log_ppl:np.float64(0.055446361937356414) - rollout_corr/kl:np.float64(0.0006691735518495889) - rollout_corr/k3_kl:np.float64(0.0008160383540030125) - rollout_corr/rollout_ppl:np.float64(1.0576571812853217) - rollout_corr/rollout_log_ppl:np.float64(0.05477718745532911) - rollout_corr/log_ppl_diff:np.float64(0.0006691744820273016) - rollout_corr/log_ppl_abs_diff:np.float64(0.002002931298193289) - rollout_corr/log_ppl_diff_max:np.float64(0.016655150800943375) - rollout_corr/log_ppl_diff_min:np.float64(-0.008791331201791763) - rollout_corr/ppl_ratio:np.float64(1.0006732817273587) - rollout_corr/chi2_token:np.float64(0.0020205182954669) - rollout_corr/chi2_seq:np.float64(1.0497447636444122) - actor/pg_loss:np.float64(0.0015107695944607258) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045274926287675044) - actor/ppo_kl:np.float64(0.00018547102608934551) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(183102.81868288037) - self_distillation/token_reweight_w_std:np.float64(1977929.5838652337) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002353701274842) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020067830417247023) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3122555688023567) - perf/mfu/actor:np.float64(0.04100007467048859) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.06307220458984) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001212156261317432 - critic/advantages/max:0.75 - critic/advantages/min:-0.75 - critic/returns/mean:0.001212156261317432 - critic/returns/max:0.75 - critic/returns/min:-0.75 - response_length/mean:303.7265625 - response_length/max:878.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:303.7265625 - response_length_non_aborted/max:878.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.6875 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.551481783390045e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8471311274915934) - timing_s/agent_loop/generate_sequences/max:np.float64(6.516193592920899) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.42985059296916) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.516193592920899) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:878 - timing_s/gen:12.268045771867037 - timing_s/reward:0.05959229916334152 - timing_s/old_log_prob:2.319757817313075 - timing_s/adv:0.5240046866238117 - timing_s/update_actor:15.775454144924879 - timing_s/step:31.036159802228212 - timing_s/stop_profile:0.00011171400547027588 - timing_per_token_ms/gen:0.1577802527441294 - timing_per_token_ms/adv:0.002918238194182576 - timing_per_token_ms/update_actor:0.0878551928856043 - perf/total_num_tokens:179562 - perf/time_per_step:31.036159802228212 - perf/throughput:723.1967531752612 (TaskRunner pid=2167046) Training Progress: 17%|█▋ | 17/100 [10:39<46:12, 33.40s/it] (TaskRunner pid=2167046) step:18 - global_seqlen/min:20824 - global_seqlen/max:23587 - global_seqlen/minmax_diff:2763 - global_seqlen/balanced_min:22174 - global_seqlen/balanced_max:22178 - global_seqlen/mean:22176.375 - actor/entropy:0.04556930065155029 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5623742341995239 - training/rollout_probs_diff_mean:0.0023348708637058735 - training/rollout_probs_diff_std:0.013050748035311699 - training/rollout_actor_probs_pearson_corr:0.9937483072280884 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.000004768371582 - rollout_corr/rollout_is_ratio_fraction_high:9.006343316286802e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015439445269294083 - rollout_corr/rollout_is_max:2.489572763442993 - rollout_corr/rollout_is_min:0.08543625473976135 - rollout_corr/rollout_is_std:0.034692030400037766 - rollout_corr/rollout_is_eff_sample_size:0.9987979097727459 - rollout_corr/rollout_is_seq_mean:0.9999890327453613 - rollout_corr/rollout_is_seq_std:0.0021977166179567575 - rollout_corr/rollout_is_seq_max:1.0082364082336426 - rollout_corr/rollout_is_seq_min:0.9947015643119812 - rollout_corr/rollout_is_seq_max_deviation:0.008236408233642578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0492625422775745) - rollout_corr/training_log_ppl:np.float64(0.04721992137547204) - rollout_corr/kl:np.float64(0.0008239883364355993) - rollout_corr/k3_kl:np.float64(0.0007315587119585132) - rollout_corr/rollout_ppl:np.float64(1.0484029022045434) - rollout_corr/rollout_log_ppl:np.float64(0.046395932438372256) - rollout_corr/log_ppl_diff:np.float64(0.0008239889370997844) - rollout_corr/log_ppl_abs_diff:np.float64(0.002035233240803791) - rollout_corr/log_ppl_diff_max:np.float64(0.009614132344722748) - rollout_corr/log_ppl_diff_min:np.float64(-0.006796486675739288) - rollout_corr/ppl_ratio:np.float64(1.0008276372682303) - rollout_corr/chi2_token:np.float64(0.0012551543768495321) - rollout_corr/chi2_seq:np.float64(0.4554172637872398) - actor/pg_loss:np.float64(0.0013457525055855513) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005007227500755107) - actor/ppo_kl:np.float64(0.00018828859354158567) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(568884.8094898432) - self_distillation/token_reweight_w_std:np.float64(5111404.572583297) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010124356485903) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02188170356384944) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.3384881727397442) - perf/mfu/actor:np.float64(0.04055189458228217) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.11115264892578) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0028788133058696985 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0028788133058696985 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:303.60546875 - response_length/max:558.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:303.60546875 - response_length_non_aborted/max:558.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:389.40625 - prompt_length/max:492.0 - prompt_length/min:206.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001553371548652649 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5853841304779053) - timing_s/agent_loop/generate_sequences/max:np.float64(5.02026298828423) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3856407619750826) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.02026298828423) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:452 - timing_s/agent_loop/slowest/response_length:558 - timing_s/gen:11.470710515975952 - timing_s/reward:0.058458348736166954 - timing_s/old_log_prob:2.323413386940956 - timing_s/adv:0.5060285422950983 - timing_s/update_actor:15.650896208360791 - timing_s/step:30.107804691419005 - timing_s/stop_profile:0.00011293962597846985 - timing_per_token_ms/gen:0.14758450543566193 - timing_per_token_ms/adv:0.002852295191927774 - timing_per_token_ms/update_actor:0.08821829654508903 - perf/total_num_tokens:177411 - perf/time_per_step:30.107804691419005 - perf/throughput:736.5656588811494 (TaskRunner pid=2167046) Training Progress: 18%|█▊ | 18/100 [11:09<44:19, 32.43s/it] (TaskRunner pid=2167046) step:19 - global_seqlen/min:20557 - global_seqlen/max:27231 - global_seqlen/minmax_diff:6674 - global_seqlen/balanced_min:23834 - global_seqlen/balanced_max:23840 - global_seqlen/mean:23837.375 - actor/entropy:0.05351593345403671 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6858960390090942 - training/rollout_probs_diff_mean:0.0023528658784925938 - training/rollout_probs_diff_std:0.012376116588711739 - training/rollout_actor_probs_pearson_corr:0.9950661659240723 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.65625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999329447746277 - rollout_corr/rollout_is_ratio_fraction_high:5.4138919949764386e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00028152239974588156 - rollout_corr/rollout_is_max:2.5501341819763184 - rollout_corr/rollout_is_min:0.03525986894965172 - rollout_corr/rollout_is_std:0.033763863146305084 - rollout_corr/rollout_is_eff_sample_size:0.9988611807426664 - rollout_corr/rollout_is_seq_mean:0.9999514818191528 - rollout_corr/rollout_is_seq_std:0.002110761357471347 - rollout_corr/rollout_is_seq_max:1.0080809593200684 - rollout_corr/rollout_is_seq_min:0.991990327835083 - rollout_corr/rollout_is_seq_max_deviation:0.00808095932006836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.053273911587894) - rollout_corr/training_log_ppl:np.float64(0.051362776504902286) - rollout_corr/kl:np.float64(0.0007405043419939084) - rollout_corr/k3_kl:np.float64(0.0007356684092343357) - rollout_corr/rollout_ppl:np.float64(1.0524831032380462) - rollout_corr/rollout_log_ppl:np.float64(0.05062227204325609) - rollout_corr/log_ppl_diff:np.float64(0.0007405044616461964) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016553804071008926) - rollout_corr/log_ppl_diff_max:np.float64(0.011551260948181152) - rollout_corr/log_ppl_diff_min:np.float64(-0.005798947066068649) - rollout_corr/ppl_ratio:np.float64(1.000743430107832) - rollout_corr/chi2_token:np.float64(0.0014110791962593794) - rollout_corr/chi2_seq:np.float64(0.35151986847631633) - actor/pg_loss:np.float64(0.0021186874946579337) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006218356556928484) - actor/ppo_kl:np.float64(4.354648220283863e-05) - actor/pg_clipfrac_lower:np.float64(1.5024038475530688e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_w_mean:np.float64(256372.4387400134) - self_distillation/token_reweight_w_std:np.float64(3430843.094189947) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000731071922928) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02942131183226593) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.3311096951365471) - perf/mfu/actor:np.float64(0.043486210929261346) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.10540008544922) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0031359968706965446 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0031359968706965446 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:360.76171875 - response_length/max:891.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.76171875 - response_length_non_aborted/max:891.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.15625 - prompt_length/max:494.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.698793292045593e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7831313852220774) - timing_s/agent_loop/generate_sequences/max:np.float64(7.0466447789222) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.911957973483368) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.0466447789222) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:891 - timing_s/gen:12.909162299707532 - timing_s/reward:0.06130508333444595 - timing_s/old_log_prob:2.3158216159790754 - timing_s/adv:0.5459947045892477 - timing_s/update_actor:15.681017262861133 - timing_s/step:31.60114638134837 - timing_s/stop_profile:0.00011025555431842804 - timing_per_token_ms/gen:0.13977762221544618 - timing_per_token_ms/adv:0.002863123060893071 - timing_per_token_ms/update_actor:0.08222915307820772 - perf/total_num_tokens:190699 - perf/time_per_step:31.60114638134837 - perf/throughput:754.3199449900113 (TaskRunner pid=2167046) Training Progress: 19%|█▉ | 19/100 [11:41<43:27, 32.20s/it] (TaskRunner pid=2167046) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2167046) validation generation end (TaskRunner pid=2167046) [prompt] system (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume $ V $ of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \times b \times c \times \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) However, since all the angles $ \alpha, \beta, \gamma $ are 90.00º, the formula simplifies because $ \cos(90^\circ) = 0 $. This makes the formula reduce to: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \times b \times c (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given: (TaskRunner pid=2167046) - $ a = 7.82 $ Å (TaskRunner pid=2167046) - $ b = 7.82 $ Å (TaskRunner pid=2167046) - $ c = 17.88 $ Å (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = 7.82 \times 7.82 \times 17.88 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then multiply by 17.88: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 61.1524 \times 17.88 \approx 1094.31 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2167440) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2167046) step:20 - global_seqlen/min:20705 - global_seqlen/max:27047 - global_seqlen/minmax_diff:6342 - global_seqlen/balanced_min:23525 - global_seqlen/balanced_max:23528 - global_seqlen/mean:23526.875 - actor/entropy:0.05020154267549515 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9398549199104309 - training/rollout_probs_diff_mean:0.002406689804047346 - training/rollout_probs_diff_std:0.012952032499015331 - training/rollout_actor_probs_pearson_corr:0.9944028854370117 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9998804330825806 - rollout_corr/rollout_is_ratio_fraction_high:8.183019235730171e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016366038471460342 - rollout_corr/rollout_is_max:3.036044120788574 - rollout_corr/rollout_is_min:6.201778160175309e-05 - rollout_corr/rollout_is_std:0.03356378525495529 - rollout_corr/rollout_is_eff_sample_size:0.9988745019683117 - rollout_corr/rollout_is_seq_mean:0.9997984170913696 - rollout_corr/rollout_is_seq_std:0.002129780361428857 - rollout_corr/rollout_is_seq_max:1.006217122077942 - rollout_corr/rollout_is_seq_min:0.9912189245223999 - rollout_corr/rollout_is_seq_max_deviation:0.008781075477600098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0514160860329866) - rollout_corr/training_log_ppl:np.float64(0.04974506429698522) - rollout_corr/kl:np.float64(0.000963009055914732) - rollout_corr/k3_kl:np.float64(0.0007886385717021938) - rollout_corr/rollout_ppl:np.float64(1.0503967669792473) - rollout_corr/rollout_log_ppl:np.float64(0.04878205476416042) - rollout_corr/log_ppl_diff:np.float64(0.000963009532824799) - rollout_corr/log_ppl_abs_diff:np.float64(0.001833038001677778) - rollout_corr/log_ppl_diff_max:np.float64(0.01848556473851204) - rollout_corr/log_ppl_diff_min:np.float64(-0.004827976226806641) - rollout_corr/ppl_ratio:np.float64(1.0009666522964835) - rollout_corr/chi2_token:np.float64(0.001090053003281355) - rollout_corr/chi2_seq:np.float64(0.2828070535324514) - actor/pg_loss:np.float64(0.0013467443641275167) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040249306493933545) - actor/ppo_kl:np.float64(6.146384180816256e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(141580.48590263491) - self_distillation/token_reweight_w_std:np.float64(2119907.649028) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000380342360586) - self_distillation/token_reweight_w_clip_frac:np.float64(0.017205427793669514) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.27972015365958214) - perf/mfu/actor:np.float64(0.042532409561833504) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.1010398864746) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7014627659574468) - val-aux/sciknoweval/reward/std@16:np.float64(0.10150057696241616) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7414893617021275) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.0709393703512258) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6616702127659574) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1000502036630689) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7016276595744682) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.10117805601327312) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7668829787234043) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.04292488363975787) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6220851063829788) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.08987312912439538) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7107765957446809) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.07709394668544002) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7828936170212766) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.024478038431684158) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5851489361702128) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07344831803022588) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7162234042553192) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.05402567777533359) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7917659574468086) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.013242035795835636) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.552968085106383) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.053209786570752704) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.719372340425532) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.03940992971234313) - val-aux/sciknoweval/score/mean@16:np.float64(0.7014627659574468) - val-aux/sciknoweval/score/std@16:np.float64(0.10150057696241616) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7414893617021275) - val-aux/sciknoweval/score/best@2/std:np.float64(0.0709393703512258) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6616702127659574) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1000502036630689) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7016276595744682) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.10117805601327312) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7668829787234043) - val-aux/sciknoweval/score/best@4/std:np.float64(0.04292488363975787) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6220851063829788) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.08987312912439538) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7107765957446809) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.07709394668544002) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7828936170212766) - val-aux/sciknoweval/score/best@8/std:np.float64(0.024478038431684158) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5851489361702128) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07344831803022588) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7162234042553192) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.05402567777533359) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7917659574468086) - val-aux/sciknoweval/score/best@16/std:np.float64(0.013242035795835636) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.552968085106383) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.053209786570752704) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.719372340425532) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.03940992971234313) - val-core/sciknoweval/acc/mean@16:np.float64(0.7014627659574468) - val-aux/sciknoweval/acc/std@16:np.float64(0.10150057696241616) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7414893617021275) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.0709393703512258) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6616702127659574) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1000502036630689) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7016276595744682) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.10117805601327312) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7668829787234043) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.04292488363975787) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6220851063829788) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.08987312912439538) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7107765957446809) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.07709394668544002) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7828936170212766) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.024478038431684158) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5851489361702128) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07344831803022588) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7162234042553192) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.05402567777533359) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7917659574468086) - val-core/sciknoweval/acc/best@16/std:np.float64(0.013242035795835636) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.552968085106383) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.053209786570752704) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.719372340425532) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.03940992971234313) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007262429571710527 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007262429571710527 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.15234375 - response_length/max:692.0 - response_length/min:160.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.15234375 - response_length_non_aborted/max:692.0 - response_length_non_aborted/min:160.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.0625 - prompt_length/max:492.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010342895984649658 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9291263706982136) - timing_s/agent_loop/generate_sequences/max:np.float64(5.923729814589024) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6822924212174257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.923729814589024) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:396 - timing_s/agent_loop/slowest/response_length:680 - timing_s/gen:12.48599156178534 - timing_s/reward:0.05641772411763668 - timing_s/old_log_prob:2.356994668021798 - timing_s/adv:0.5368302222341299 - timing_s/update_actor:15.727698093280196 - timing_s/step:31.241562753915787 - timing_s/testing:38.38132827542722 - timing_s/save_checkpoint:6.6703202817589045 - timing_s/stop_profile:0.00014017336070537567 - timing_per_token_ms/gen:0.1459615814477554 - timing_per_token_ms/adv:0.0028522180603784498 - timing_per_token_ms/update_actor:0.08356240519236084 - perf/total_num_tokens:188215 - perf/time_per_step:31.241562753915787 - perf/throughput:753.0633209777947 (TaskRunner pid=2167046) Training Progress: 20%|██ | 20/100 [12:57<1:00:36, 45.46s/it] (TaskRunner pid=2167046) step:21 - global_seqlen/min:20775 - global_seqlen/max:26559 - global_seqlen/minmax_diff:5784 - global_seqlen/balanced_min:23627 - global_seqlen/balanced_max:23631 - global_seqlen/mean:23629.0 - actor/entropy:0.05921781063079834 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8452574610710144 - training/rollout_probs_diff_mean:0.002616164507344365 - training/rollout_probs_diff_std:0.013296348042786121 - training/rollout_actor_probs_pearson_corr:0.9946975708007812 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9999632239341736 - rollout_corr/rollout_is_ratio_fraction_high:4.538852590485476e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015885983884800225 - rollout_corr/rollout_is_max:2.8393192291259766 - rollout_corr/rollout_is_min:0.007855840027332306 - rollout_corr/rollout_is_std:0.03435363247990608 - rollout_corr/rollout_is_eff_sample_size:0.9988209813407378 - rollout_corr/rollout_is_seq_mean:0.9998990297317505 - rollout_corr/rollout_is_seq_std:0.0020701477769762278 - rollout_corr/rollout_is_seq_max:1.0062670707702637 - rollout_corr/rollout_is_seq_min:0.9937349557876587 - rollout_corr/rollout_is_seq_max_deviation:0.006267070770263672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0587520101107657) - rollout_corr/training_log_ppl:np.float64(0.05634416791508556) - rollout_corr/kl:np.float64(0.0007776328535911148) - rollout_corr/k3_kl:np.float64(0.0008263086628659266) - rollout_corr/rollout_ppl:np.float64(1.0579165108501911) - rollout_corr/rollout_log_ppl:np.float64(0.055566534494573716) - rollout_corr/log_ppl_diff:np.float64(0.0007776334205118474) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018763783118629362) - rollout_corr/log_ppl_diff_max:np.float64(0.009136740118265152) - rollout_corr/log_ppl_diff_min:np.float64(-0.007562510669231415) - rollout_corr/ppl_ratio:np.float64(1.0007807007059455) - rollout_corr/chi2_token:np.float64(0.002017973456531763) - rollout_corr/chi2_seq:np.float64(0.6339192821178585) - actor/pg_loss:np.float64(0.0016546305269002914) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005765741029790661) - actor/ppo_kl:np.float64(-1.5955860524385912e-05) - actor/pg_clipfrac_lower:np.float64(1.859824169514468e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(251456.31727892137) - self_distillation/token_reweight_w_std:np.float64(3611087.4808393884) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008972657378763) - self_distillation/token_reweight_w_clip_frac:np.float64(0.023536412860266864) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.30859530717134476) - perf/mfu/actor:np.float64(0.043484721780607125) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.56780242919922) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006455099210143089 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.006455099210143089 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:344.25 - response_length/max:852.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:344.25 - response_length_non_aborted/max:852.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.15625 - prompt_length/max:524.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.492039680480957e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8489494323730469) - timing_s/agent_loop/generate_sequences/max:np.float64(6.843407042324543) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7803078965080203) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.843407042324543) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:852 - timing_s/gen:12.684395214542747 - timing_s/reward:0.05293784849345684 - timing_s/old_log_prob:2.5294907856732607 - timing_s/adv:0.5785386748611927 - timing_s/update_actor:15.463469890877604 - timing_s/step:31.37583982385695 - timing_s/stop_profile:0.00012954510748386383 - timing_per_token_ms/gen:0.14393149980191025 - timing_per_token_ms/adv:0.003060533004259558 - timing_per_token_ms/update_actor:0.08180345069024082 - perf/total_num_tokens:189032 - perf/time_per_step:31.37583982385695 - perf/throughput:753.0953795229871 (TaskRunner pid=2167046) Training Progress: 21%|██ | 21/100 [13:29<54:18, 41.25s/it] (TaskRunner pid=2167046) step:22 - global_seqlen/min:18203 - global_seqlen/max:26237 - global_seqlen/minmax_diff:8034 - global_seqlen/balanced_min:22751 - global_seqlen/balanced_max:22753 - global_seqlen/mean:22751.875 - actor/entropy:0.04705189913511276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4668269157409668 - training/rollout_probs_diff_mean:0.002335731405764818 - training/rollout_probs_diff_std:0.012947346083819866 - training/rollout_actor_probs_pearson_corr:0.993709921836853 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0000593662261963 - rollout_corr/rollout_is_ratio_fraction_high:9.395074448548257e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002113891823682934 - rollout_corr/rollout_is_max:5.089398384094238 - rollout_corr/rollout_is_min:0.002671187976375222 - rollout_corr/rollout_is_std:0.033378586173057556 - rollout_corr/rollout_is_eff_sample_size:0.9988869909400563 - rollout_corr/rollout_is_seq_mean:1.0000618696212769 - rollout_corr/rollout_is_seq_std:0.002444080077111721 - rollout_corr/rollout_is_seq_max:1.0121523141860962 - rollout_corr/rollout_is_seq_min:0.9936273694038391 - rollout_corr/rollout_is_seq_max_deviation:0.012152314186096191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0460349675267935) - rollout_corr/training_log_ppl:np.float64(0.04464656937489053) - rollout_corr/kl:np.float64(0.00055560777209962) - rollout_corr/k3_kl:np.float64(0.0007500760087446601) - rollout_corr/rollout_ppl:np.float64(1.0454549426212907) - rollout_corr/rollout_log_ppl:np.float64(0.04409096138442692) - rollout_corr/log_ppl_diff:np.float64(0.0005556079904636135) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019402096240810351) - rollout_corr/log_ppl_diff_max:np.float64(0.009045779705047607) - rollout_corr/log_ppl_diff_min:np.float64(-0.007189542055130005) - rollout_corr/ppl_ratio:np.float64(1.0005588978528976) - rollout_corr/chi2_token:np.float64(0.0021106130443513393) - rollout_corr/chi2_seq:np.float64(0.889191702241078) - actor/pg_loss:np.float64(0.0017684989143162966) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005356281330932688) - actor/ppo_kl:np.float64(-9.231486468763705e-05) - actor/pg_clipfrac_lower:np.float64(1.1421783710829914e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(110542.58722938504) - self_distillation/token_reweight_w_std:np.float64(1534410.273381344) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997402860317379) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020450474316021428) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.31053709238767624) - perf/mfu/actor:np.float64(0.040778937791466366) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.4330825805664) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004377517383545637 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004377517383545637 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:332.62109375 - response_length/max:628.0 - response_length/min:153.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.62109375 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:153.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:378.375 - prompt_length/max:509.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.261816740036011e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8655258305370808) - timing_s/agent_loop/generate_sequences/max:np.float64(5.580149905756116) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.612725309583766) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.580149905756116) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:577 - timing_s/gen:11.961912382394075 - timing_s/reward:0.06006980314850807 - timing_s/old_log_prob:2.27942262776196 - timing_s/adv:0.5185586716979742 - timing_s/update_actor:16.07531176134944 - timing_s/step:30.982574217021465 - timing_s/stop_profile:3.722868859767914e-05 - timing_per_token_ms/gen:0.1404788244694023 - timing_per_token_ms/adv:0.002848988664109959 - timing_per_token_ms/update_actor:0.08831860979232173 - perf/total_num_tokens:182015 - perf/time_per_step:30.982574217021465 - perf/throughput:734.3442426904729 (TaskRunner pid=2167046) Training Progress: 22%|██▏ | 22/100 [14:00<49:37, 38.17s/it] (TaskRunner pid=2167046) step:23 - global_seqlen/min:16765 - global_seqlen/max:25461 - global_seqlen/minmax_diff:8696 - global_seqlen/balanced_min:22231 - global_seqlen/balanced_max:22234 - global_seqlen/mean:22233.0 - actor/entropy:0.051857516169548035 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8924874663352966 - training/rollout_probs_diff_mean:0.002473828848451376 - training/rollout_probs_diff_std:0.014042152091860771 - training/rollout_actor_probs_pearson_corr:0.9934764504432678 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.78125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:0.999901294708252 - rollout_corr/rollout_is_ratio_fraction_high:8.789553248789161e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002762430813163519 - rollout_corr/rollout_is_max:3.1934919357299805 - rollout_corr/rollout_is_min:0.05045768618583679 - rollout_corr/rollout_is_std:0.03492661938071251 - rollout_corr/rollout_is_eff_sample_size:0.9987814986875572 - rollout_corr/rollout_is_seq_mean:0.9999404549598694 - rollout_corr/rollout_is_seq_std:0.002376072108745575 - rollout_corr/rollout_is_seq_max:1.0088611841201782 - rollout_corr/rollout_is_seq_min:0.9945573806762695 - rollout_corr/rollout_is_seq_max_deviation:0.008861184120178223 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0516580799594522) - rollout_corr/training_log_ppl:np.float64(0.0500233011553064) - rollout_corr/kl:np.float64(0.0007878192633476999) - rollout_corr/k3_kl:np.float64(0.0008335432382278185) - rollout_corr/rollout_ppl:np.float64(1.0508313993923366) - rollout_corr/rollout_log_ppl:np.float64(0.04923548138322076) - rollout_corr/log_ppl_diff:np.float64(0.0007878197720856406) - rollout_corr/log_ppl_abs_diff:np.float64(0.001956198444531765) - rollout_corr/log_ppl_diff_max:np.float64(0.012837372720241547) - rollout_corr/log_ppl_diff_min:np.float64(-0.006586827337741852) - rollout_corr/ppl_ratio:np.float64(1.0007912560831755) - rollout_corr/chi2_token:np.float64(0.0018308169674128294) - rollout_corr/chi2_seq:np.float64(0.6063053519465029) - actor/pg_loss:np.float64(0.001039000111632049) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003526491568663914) - actor/ppo_kl:np.float64(-5.0933335728586826e-05) - actor/pg_clipfrac_lower:np.float64(8.510348379786592e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_w_mean:np.float64(190123.5627077485) - self_distillation/token_reweight_w_std:np.float64(2663452.2716900078) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003106228541583) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015045241816551425) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.28235869854688644) - perf/mfu/actor:np.float64(0.04055485686561945) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.30136489868164) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0057085007429122925 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0057085007429122925 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:311.09375 - response_length/max:662.0 - response_length/min:164.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:311.09375 - response_length_non_aborted/max:662.0 - response_length_non_aborted/min:164.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.6875 - prompt_length/max:493.0 - prompt_length/min:257.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.78582626581192e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9778354968875647) - timing_s/agent_loop/generate_sequences/max:np.float64(5.440660933032632) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4218751417065505) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.440660933032632) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:662 - timing_s/gen:11.383110078051686 - timing_s/reward:0.060048116371035576 - timing_s/old_log_prob:2.3218838050961494 - timing_s/adv:0.5430666841566563 - timing_s/update_actor:15.781536480411887 - timing_s/step:30.183814767748117 - timing_s/stop_profile:0.00010877661406993866 - timing_per_token_ms/gen:0.1429320702919599 - timing_per_token_ms/adv:0.003053269262788739 - timing_per_token_ms/update_actor:0.08872810956917582 - perf/total_num_tokens:177864 - perf/time_per_step:30.183814767748117 - perf/throughput:736.5868155192998 (TaskRunner pid=2167046) Training Progress: 23%|██▎ | 23/100 [14:30<45:55, 35.79s/it] (TaskRunner pid=2167046) step:24 - global_seqlen/min:20488 - global_seqlen/max:26007 - global_seqlen/minmax_diff:5519 - global_seqlen/balanced_min:23418 - global_seqlen/balanced_max:23421 - global_seqlen/mean:23419.625 - actor/entropy:0.048930879682302475 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8673235774040222 - training/rollout_probs_diff_mean:0.002186768688261509 - training/rollout_probs_diff_std:0.012573663145303726 - training/rollout_actor_probs_pearson_corr:0.9943985939025879 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:0.9998534321784973 - rollout_corr/rollout_is_ratio_fraction_high:3.3702941436786205e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001797490258468315 - rollout_corr/rollout_is_max:2.5502538681030273 - rollout_corr/rollout_is_min:0.007685937453061342 - rollout_corr/rollout_is_std:0.030430540442466736 - rollout_corr/rollout_is_eff_sample_size:0.9990746009767545 - rollout_corr/rollout_is_seq_mean:0.9998515248298645 - rollout_corr/rollout_is_seq_std:0.0018696229672059417 - rollout_corr/rollout_is_seq_max:1.0090330839157104 - rollout_corr/rollout_is_seq_min:0.9939467906951904 - rollout_corr/rollout_is_seq_max_deviation:0.00903308391571045 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0505288592539728) - rollout_corr/training_log_ppl:np.float64(0.0487635411645897) - rollout_corr/kl:np.float64(0.0006307011686601527) - rollout_corr/k3_kl:np.float64(0.0005982627429759191) - rollout_corr/rollout_ppl:np.float64(1.049854587763548) - rollout_corr/rollout_log_ppl:np.float64(0.04813283952717029) - rollout_corr/log_ppl_diff:np.float64(0.0006307016374194063) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015282359381671995) - rollout_corr/log_ppl_diff_max:np.float64(0.009290523827075958) - rollout_corr/log_ppl_diff_min:np.float64(-0.005929280072450638) - rollout_corr/ppl_ratio:np.float64(1.000633022049442) - rollout_corr/chi2_token:np.float64(0.001259797951206565) - rollout_corr/chi2_seq:np.float64(0.3109738379716873) - actor/pg_loss:np.float64(0.0009422041475772858) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003166221840729122) - actor/ppo_kl:np.float64(-0.00011196534618562737) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_w_mean:np.float64(158053.5074800374) - self_distillation/token_reweight_w_std:np.float64(2085045.1837178322) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999543417012319) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01681827876745956) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.25178469717502594) - perf/mfu/actor:np.float64(0.042570392907962495) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.3254165649414) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007204003632068634 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007204003632068634 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:347.70703125 - response_length/max:768.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:347.70703125 - response_length_non_aborted/max:768.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.15625 - prompt_length/max:477.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.500926196575165e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.854590266942978) - timing_s/agent_loop/generate_sequences/max:np.float64(6.362085048109293) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8548405513865873) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.362085048109293) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:453 - timing_s/agent_loop/slowest/response_length:768 - timing_s/gen:12.194638228043914 - timing_s/reward:0.06035494804382324 - timing_s/old_log_prob:2.3020692132413387 - timing_s/adv:0.5678048375993967 - timing_s/update_actor:15.75032832287252 - timing_s/step:30.96565161831677 - timing_s/stop_profile:3.7107616662979126e-05 - timing_per_token_ms/gen:0.13699839605500225 - timing_per_token_ms/adv:0.003030603807700789 - timing_per_token_ms/update_actor:0.08406586528858019 - perf/total_num_tokens:187357 - perf/time_per_step:30.96565161831677 - perf/throughput:756.3097747359158 (TaskRunner pid=2167046) Training Progress: 24%|██▍ | 24/100 [15:01<43:30, 34.35s/it] (TaskRunner pid=2167046) step:25 - global_seqlen/min:17336 - global_seqlen/max:25737 - global_seqlen/minmax_diff:8401 - global_seqlen/balanced_min:22469 - global_seqlen/balanced_max:22472 - global_seqlen/mean:22471.25 - actor/entropy:0.052130524069070816 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9925975203514099 - training/rollout_probs_diff_mean:0.0023709130473434925 - training/rollout_probs_diff_std:0.013253876008093357 - training/rollout_actor_probs_pearson_corr:0.9941772222518921 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:0.9997007846832275 - rollout_corr/rollout_is_ratio_fraction_high:1.2528502338682301e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016287053585983813 - rollout_corr/rollout_is_max:2.049886465072632 - rollout_corr/rollout_is_min:0.0007144523551687598 - rollout_corr/rollout_is_std:0.03154897689819336 - rollout_corr/rollout_is_eff_sample_size:0.9990051163742315 - rollout_corr/rollout_is_seq_mean:0.999676525592804 - rollout_corr/rollout_is_seq_std:0.002087613334879279 - rollout_corr/rollout_is_seq_max:1.0080496072769165 - rollout_corr/rollout_is_seq_min:0.9920728206634521 - rollout_corr/rollout_is_seq_max_deviation:0.008049607276916504 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0590065233409405) - rollout_corr/training_log_ppl:np.float64(0.05567418575810734) - rollout_corr/kl:np.float64(0.0014148961926281345) - rollout_corr/k3_kl:np.float64(0.0010043128627899023) - rollout_corr/rollout_ppl:np.float64(1.057496966328472) - rollout_corr/rollout_log_ppl:np.float64(0.054259289096080465) - rollout_corr/log_ppl_diff:np.float64(0.0014148966620268766) - rollout_corr/log_ppl_abs_diff:np.float64(0.002337703506782418) - rollout_corr/log_ppl_diff_max:np.float64(0.025214478373527527) - rollout_corr/log_ppl_diff_min:np.float64(-0.007557865232229233) - rollout_corr/ppl_ratio:np.float64(1.0014220480807126) - rollout_corr/chi2_token:np.float64(0.0008356994949281216) - rollout_corr/chi2_seq:np.float64(0.20744728250429034) - actor/pg_loss:np.float64(0.0009773996425792575) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005361891062420909) - actor/ppo_kl:np.float64(0.0004331388923333179) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(198215.93650202313) - self_distillation/token_reweight_w_std:np.float64(2314051.2728974866) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000367683591321) - self_distillation/token_reweight_w_clip_frac:np.float64(0.017195056650962215) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.28173942118883133) - perf/mfu/actor:np.float64(0.04129532551700841) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.3085479736328) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005011400789953768 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005011400789953768 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:311.7890625 - response_length/max:647.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:311.7890625 - response_length_non_aborted/max:647.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.4375 - prompt_length/max:481.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.693523705005646e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.714713603258133) - timing_s/agent_loop/generate_sequences/max:np.float64(5.528594700619578) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4725499730047886) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.528594700619578) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:429 - timing_s/agent_loop/slowest/response_length:647 - timing_s/gen:11.64700966142118 - timing_s/reward:0.0598090086132288 - timing_s/old_log_prob:2.29461557418108 - timing_s/adv:0.5115159582346678 - timing_s/update_actor:15.511012205854058 - timing_s/step:30.108242020010948 - timing_s/stop_profile:3.7707388401031494e-05 - timing_per_token_ms/gen:0.14591958783007816 - timing_per_token_ms/adv:0.002845391101043933 - timing_per_token_ms/update_actor:0.08628253994467408 - perf/total_num_tokens:179770 - perf/time_per_step:30.108242020010948 - perf/throughput:746.3487899780018 (TaskRunner pid=2167046) Training Progress: 25%|██▌ | 25/100 [15:31<41:21, 33.08s/it] (TaskRunner pid=2167046) step:26 - global_seqlen/min:21815 - global_seqlen/max:25977 - global_seqlen/minmax_diff:4162 - global_seqlen/balanced_min:23256 - global_seqlen/balanced_max:23259 - global_seqlen/mean:23257.375 - actor/entropy:0.05010559782385826 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9880870580673218 - training/rollout_probs_diff_mean:0.002416659612208605 - training/rollout_probs_diff_std:0.013399409130215645 - training/rollout_actor_probs_pearson_corr:0.9939301013946533 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9997743964195251 - rollout_corr/rollout_is_ratio_fraction_high:3.553029091563076e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00023686859640292823 - rollout_corr/rollout_is_max:2.335609197616577 - rollout_corr/rollout_is_min:0.006727981381118298 - rollout_corr/rollout_is_std:0.03265010192990303 - rollout_corr/rollout_is_eff_sample_size:0.9989349277027401 - rollout_corr/rollout_is_seq_mean:0.9997029304504395 - rollout_corr/rollout_is_seq_std:0.001995098777115345 - rollout_corr/rollout_is_seq_max:1.0078387260437012 - rollout_corr/rollout_is_seq_min:0.9932174682617188 - rollout_corr/rollout_is_seq_max_deviation:0.007838726043701172 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0507985781878233) - rollout_corr/training_log_ppl:np.float64(0.04918104125681566) - rollout_corr/kl:np.float64(0.0010938695645918983) - rollout_corr/k3_kl:np.float64(0.00082757380827303) - rollout_corr/rollout_ppl:np.float64(1.04963733907789) - rollout_corr/rollout_log_ppl:np.float64(0.04808717123887618) - rollout_corr/log_ppl_diff:np.float64(0.00109387001793948) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019342835767020006) - rollout_corr/log_ppl_diff_max:np.float64(0.013077668845653534) - rollout_corr/log_ppl_diff_min:np.float64(-0.004524499177932739) - rollout_corr/ppl_ratio:np.float64(1.0010978260543197) - rollout_corr/chi2_token:np.float64(0.0010612753685563803) - rollout_corr/chi2_seq:np.float64(0.26944553572684526) - actor/pg_loss:np.float64(0.0011050777975469828) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036502554121398134) - actor/ppo_kl:np.float64(0.00015411552791277927) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(148665.91016933904) - self_distillation/token_reweight_w_std:np.float64(2118637.270728631) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002983210142702) - self_distillation/token_reweight_w_clip_frac:np.float64(0.018292905369889922) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.3034274894744158) - perf/mfu/actor:np.float64(0.041829506814014934) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(144.37565231323242) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00418517179787159 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00418517179787159 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:329.82421875 - response_length/max:591.0 - response_length/min:159.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:329.82421875 - response_length_non_aborted/max:591.0 - response_length_non_aborted/min:159.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.96875 - prompt_length/max:491.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.7172030210494995e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8563682362437248) - timing_s/agent_loop/generate_sequences/max:np.float64(5.39569622464478) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.636151098224218) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.39569622464478) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:271 - timing_s/agent_loop/slowest/response_length:591 - timing_s/gen:11.275522960349917 - timing_s/reward:0.059716515243053436 - timing_s/old_log_prob:2.3524911049753428 - timing_s/adv:0.5290274638682604 - timing_s/update_actor:16.032338209450245 - timing_s/step:30.3343845885247 - timing_s/stop_profile:3.7476420402526855e-05 - timing_per_token_ms/gen:0.1335408652851296 - timing_per_token_ms/adv:0.0028433317596475334 - timing_per_token_ms/update_actor:0.08616803384652312 - perf/total_num_tokens:186059 - perf/time_per_step:30.3343845885247 - perf/throughput:766.7000770075987 (TaskRunner pid=2167046) Training Progress: 26%|██▌ | 26/100 [16:01<39:47, 32.26s/it] (TaskRunner pid=2167046) step:27 - global_seqlen/min:19833 - global_seqlen/max:25283 - global_seqlen/minmax_diff:5450 - global_seqlen/balanced_min:22907 - global_seqlen/balanced_max:22913 - global_seqlen/mean:22912.0 - actor/entropy:0.042662691324949265 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8065875172615051 - training/rollout_probs_diff_mean:0.0022665185388177633 - training/rollout_probs_diff_std:0.01338112261146307 - training/rollout_actor_probs_pearson_corr:0.9929261803627014 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:0.9998301267623901 - rollout_corr/rollout_is_ratio_fraction_high:3.683331669890322e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002455554495099932 - rollout_corr/rollout_is_max:3.015857458114624 - rollout_corr/rollout_is_min:0.013438464142382145 - rollout_corr/rollout_is_std:0.0322062112390995 - rollout_corr/rollout_is_eff_sample_size:0.99896335887444 - rollout_corr/rollout_is_seq_mean:0.9998033046722412 - rollout_corr/rollout_is_seq_std:0.002107642823830247 - rollout_corr/rollout_is_seq_max:1.0079267024993896 - rollout_corr/rollout_is_seq_min:0.9947221279144287 - rollout_corr/rollout_is_seq_max_deviation:0.007926702499389648 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.044278597459197) - rollout_corr/training_log_ppl:np.float64(0.04298725330590969) - rollout_corr/kl:np.float64(0.0008925169439546465) - rollout_corr/k3_kl:np.float64(0.0006722692353946513) - rollout_corr/rollout_ppl:np.float64(1.0433417870663106) - rollout_corr/rollout_log_ppl:np.float64(0.04209473588707624) - rollout_corr/log_ppl_diff:np.float64(0.0008925174188334495) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019630308088380843) - rollout_corr/log_ppl_diff_max:np.float64(0.011757582426071167) - rollout_corr/log_ppl_diff_min:np.float64(-0.006372295320034027) - rollout_corr/ppl_ratio:np.float64(1.000895933015272) - rollout_corr/chi2_token:np.float64(0.0008611760567873716) - rollout_corr/chi2_seq:np.float64(0.5474486453458667) - actor/pg_loss:np.float64(0.0006962757324799895) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002548012948864198) - actor/ppo_kl:np.float64(5.430080918955582e-05) - actor/pg_clipfrac_lower:np.float64(1.073145631380612e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(212501.34884866723) - self_distillation/token_reweight_w_std:np.float64(2760884.2314605396) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005110597703606) - self_distillation/token_reweight_w_clip_frac:np.float64(0.012378365972836036) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.23239875957369804) - perf/mfu/actor:np.float64(0.04186334349800554) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.43539905548096) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-2.1486101104528643e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-2.1486101104528643e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:318.15625 - response_length/max:662.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:318.15625 - response_length_non_aborted/max:662.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.84375 - prompt_length/max:494.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003025401383638382 - timing_s/agent_loop/generate_sequences/min:np.float64(1.900025250390172) - timing_s/agent_loop/generate_sequences/max:np.float64(5.731828100979328) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6442061743873637) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.731828100979328) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:662 - timing_s/gen:11.743904303759336 - timing_s/reward:0.060592472553253174 - timing_s/old_log_prob:2.529304649680853 - timing_s/adv:0.5707656517624855 - timing_s/update_actor:15.67794787697494 - timing_s/step:30.6938898563385 - timing_s/stop_profile:0.0001104213297367096 - timing_per_token_ms/gen:0.14418898320105264 - timing_per_token_ms/adv:0.003113901294968169 - timing_per_token_ms/update_actor:0.08553349705926447 - perf/total_num_tokens:183296 - perf/time_per_step:30.6938898563385 - perf/throughput:746.4677858439801 (TaskRunner pid=2167046) Training Progress: 27%|██▋ | 27/100 [16:32<38:48, 31.90s/it] (TaskRunner pid=2167046) step:28 - global_seqlen/min:18564 - global_seqlen/max:25860 - global_seqlen/minmax_diff:7296 - global_seqlen/balanced_min:22067 - global_seqlen/balanced_max:22074 - global_seqlen/mean:22070.75 - actor/entropy:0.050707269459962845 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6709722280502319 - training/rollout_probs_diff_mean:0.0023314820136874914 - training/rollout_probs_diff_std:0.013388410210609436 - training/rollout_actor_probs_pearson_corr:0.99385005235672 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999811053276062 - rollout_corr/rollout_is_ratio_fraction_high:7.481483044102788e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00023691363458056003 - rollout_corr/rollout_is_max:3.443509578704834 - rollout_corr/rollout_is_min:0.28613391518592834 - rollout_corr/rollout_is_std:0.03239627555012703 - rollout_corr/rollout_is_eff_sample_size:0.9989513438303744 - rollout_corr/rollout_is_seq_mean:1.000024676322937 - rollout_corr/rollout_is_seq_std:0.0022317811381071806 - rollout_corr/rollout_is_seq_max:1.0114712715148926 - rollout_corr/rollout_is_seq_min:0.9920425415039062 - rollout_corr/rollout_is_seq_max_deviation:0.011471271514892578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0535097629763186) - rollout_corr/training_log_ppl:np.float64(0.051123634291798226) - rollout_corr/kl:np.float64(0.0006726607275737706) - rollout_corr/k3_kl:np.float64(0.0008511580656858087) - rollout_corr/rollout_ppl:np.float64(1.0527968504466116) - rollout_corr/rollout_log_ppl:np.float64(0.05045097289439582) - rollout_corr/log_ppl_diff:np.float64(0.0006726613974024076) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018435009587847162) - rollout_corr/log_ppl_diff_max:np.float64(0.012318253517150879) - rollout_corr/log_ppl_diff_min:np.float64(-0.0074674468487501144) - rollout_corr/ppl_ratio:np.float64(1.0006759993266314) - rollout_corr/chi2_token:np.float64(0.0040700845420360565) - rollout_corr/chi2_seq:np.float64(0.6581041102763265) - actor/pg_loss:np.float64(0.0009383222786709666) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029797435490763746) - actor/ppo_kl:np.float64(9.442604779663633e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(150508.01074047922) - self_distillation/token_reweight_w_std:np.float64(2154139.784913294) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003592369612306) - self_distillation/token_reweight_w_clip_frac:np.float64(0.013951479126262711) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.1988047994673252) - perf/mfu/actor:np.float64(0.03959577009416985) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.41423416137695) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0020215590484440327 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0020215590484440327 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:313.2734375 - response_length/max:689.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:313.2734375 - response_length_non_aborted/max:689.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:376.4375 - prompt_length/max:478.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018197670578956604 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1266491133719683) - timing_s/agent_loop/generate_sequences/max:np.float64(5.846070012077689) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.404199506265286) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.846070012077689) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:658 - timing_s/gen:11.615958483889699 - timing_s/reward:0.05892422795295715 - timing_s/old_log_prob:2.3650641944259405 - timing_s/adv:0.5204076077789068 - timing_s/update_actor:15.855927553027868 - timing_s/step:30.50866850093007 - timing_s/stop_profile:2.9498711228370667e-05 - timing_per_token_ms/gen:0.14484099957467392 - timing_per_token_ms/adv:0.0029473828923966493 - timing_per_token_ms/update_actor:0.0898017033462154 - perf/total_num_tokens:176566 - perf/time_per_step:30.50866850093007 - perf/throughput:723.4255404927673 (TaskRunner pid=2167046) Training Progress: 28%|██▊ | 28/100 [17:03<37:47, 31.49s/it] (TaskRunner pid=2167046) step:29 - global_seqlen/min:18921 - global_seqlen/max:27487 - global_seqlen/minmax_diff:8566 - global_seqlen/balanced_min:22991 - global_seqlen/balanced_max:22998 - global_seqlen/mean:22995.25 - actor/entropy:0.039533860981464386 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7222251296043396 - training/rollout_probs_diff_mean:0.0019475192530080676 - training/rollout_probs_diff_std:0.011736846528947353 - training/rollout_actor_probs_pearson_corr:0.9941072463989258 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999589920043945 - rollout_corr/rollout_is_ratio_fraction_high:3.7319932744139805e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016171969764400274 - rollout_corr/rollout_is_max:2.6647143363952637 - rollout_corr/rollout_is_min:0.116624616086483 - rollout_corr/rollout_is_std:0.02993287518620491 - rollout_corr/rollout_is_eff_sample_size:0.9991047060399887 - rollout_corr/rollout_is_seq_mean:0.9999057054519653 - rollout_corr/rollout_is_seq_std:0.0018940429436042905 - rollout_corr/rollout_is_seq_max:1.0079894065856934 - rollout_corr/rollout_is_seq_min:0.9945310950279236 - rollout_corr/rollout_is_seq_max_deviation:0.00798940658569336 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.040629532188177) - rollout_corr/training_log_ppl:np.float64(0.03935588929562073) - rollout_corr/kl:np.float64(0.0008194791879141405) - rollout_corr/k3_kl:np.float64(0.0005604395872609302) - rollout_corr/rollout_ppl:np.float64(1.0397686720825732) - rollout_corr/rollout_log_ppl:np.float64(0.03853640963188809) - rollout_corr/log_ppl_diff:np.float64(0.0008194796637326363) - rollout_corr/log_ppl_abs_diff:np.float64(0.001599156311385741) - rollout_corr/log_ppl_diff_max:np.float64(0.009142909198999405) - rollout_corr/log_ppl_diff_min:np.float64(-0.004533983767032623) - rollout_corr/ppl_ratio:np.float64(1.000821965513751) - rollout_corr/chi2_token:np.float64(0.0006333421915769577) - rollout_corr/chi2_seq:np.float64(0.12789275567047298) - actor/pg_loss:np.float64(0.0002092909999191761) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.36119579616934e-05) - actor/ppo_kl:np.float64(0.0002650999025854972) - actor/pg_clipfrac_lower:np.float64(9.717039574752562e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(155151.15066527855) - self_distillation/token_reweight_w_std:np.float64(1819004.7734375) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00056552561) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0050811002001864836) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.12285954970866442) - perf/mfu/actor:np.float64(0.042161963586262195) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.6112823486328) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005940088885836303 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:-0.0005940088885836303 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:314.0078125 - response_length/max:665.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.0078125 - response_length_non_aborted/max:665.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:404.59375 - prompt_length/max:512.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010629184544086456 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8923852071166039) - timing_s/agent_loop/generate_sequences/max:np.float64(5.652709232643247) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.552825273196504) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.652709232643247) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:512 - timing_s/agent_loop/slowest/response_length:665 - timing_s/gen:11.469049690291286 - timing_s/reward:0.04871102795004845 - timing_s/old_log_prob:2.3286929316818714 - timing_s/adv:0.5320415683090687 - timing_s/update_actor:15.673507444560528 - timing_s/step:30.08731354214251 - timing_s/stop_profile:0.00011089444160461426 - timing_per_token_ms/gen:0.14267471562574685 - timing_per_token_ms/adv:0.0028921275497606497 - timing_per_token_ms/update_actor:0.08519970126743853 - perf/total_num_tokens:183962 - perf/time_per_step:30.08731354214251 - perf/throughput:764.2839221185752 (TaskRunner pid=2167046) Training Progress: 29%|██▉ | 29/100 [17:33<36:46, 31.08s/it] (TaskRunner pid=2167046) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2167046) validation generation end (TaskRunner pid=2167046) [prompt] system (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume of a unit cell in a crystal with orthogonal lattice parameters (α = β = γ = 90.00º) is calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) V = a × b × c (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given: (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) (TaskRunner pid=2167046) Substituting the values: (TaskRunner pid=2167046) (TaskRunner pid=2167046) V = 7.82 × 7.82 × 17.88 (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate 7.82 × 7.82: (TaskRunner pid=2167046) 7.82 × 7.82 ≈ 61.1524 (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then, multiply by 17.88: (TaskRunner pid=2167046) 61.1524 × 17.88 ≈ 1094.31 (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2167440) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2167046) step:30 - global_seqlen/min:19442 - global_seqlen/max:27088 - global_seqlen/minmax_diff:7646 - global_seqlen/balanced_min:23922 - global_seqlen/balanced_max:23932 - global_seqlen/mean:23928.875 - actor/entropy:0.04492096230387688 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7854477167129517 - training/rollout_probs_diff_mean:0.0021507125347852707 - training/rollout_probs_diff_std:0.013460204005241394 - training/rollout_actor_probs_pearson_corr:0.9931148886680603 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.55078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:1.0000746250152588 - rollout_corr/rollout_is_ratio_fraction_high:9.3432841822505e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002335821045562625 - rollout_corr/rollout_is_max:2.5501413345336914 - rollout_corr/rollout_is_min:0.10825343430042267 - rollout_corr/rollout_is_std:0.03233734518289566 - rollout_corr/rollout_is_eff_sample_size:0.9989556263871889 - rollout_corr/rollout_is_seq_mean:1.0000686645507812 - rollout_corr/rollout_is_seq_std:0.0021643403451889753 - rollout_corr/rollout_is_seq_max:1.0080591440200806 - rollout_corr/rollout_is_seq_min:0.9934688806533813 - rollout_corr/rollout_is_seq_max_deviation:0.008059144020080566 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0462839058600366) - rollout_corr/training_log_ppl:np.float64(0.044393875357854995) - rollout_corr/kl:np.float64(0.0005853226012781931) - rollout_corr/k3_kl:np.float64(0.00065308902737371) - rollout_corr/rollout_ppl:np.float64(1.045660697389394) - rollout_corr/rollout_log_ppl:np.float64(0.04380855248609805) - rollout_corr/log_ppl_diff:np.float64(0.0005853228717569436) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016660557507748308) - rollout_corr/log_ppl_diff_max:np.float64(0.022737577557563782) - rollout_corr/log_ppl_diff_min:np.float64(-0.0062180086970329285) - rollout_corr/ppl_ratio:np.float64(1.0005887241568416) - rollout_corr/chi2_token:np.float64(0.0014040102250874043) - rollout_corr/chi2_seq:np.float64(0.52653125859797) - actor/pg_loss:np.float64(0.0013053459115326405) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032061494812296587) - actor/ppo_kl:np.float64(7.5638094044983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55078125) - self_distillation/token_reweight_w_mean:np.float64(151288.02862975863) - self_distillation/token_reweight_w_std:np.float64(1988820.3295037607) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011276304721832) - self_distillation/token_reweight_w_clip_frac:np.float64(0.017992489927564748) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.24706468358635902) - perf/mfu/actor:np.float64(0.0430726144645511) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.65000915527344) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7253989361702128) - val-aux/sciknoweval/reward/std@16:np.float64(0.08655772986262227) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7613191489361701) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.05961052684599058) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6877127659574468) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.08852506830517469) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7250638297872342) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.08731454078539842) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7811808510638297) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.030545979305562004) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6498936170212767) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.07794589909969647) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7372234042553192) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.06937401821560567) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7911170212765959) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.014284231202351703) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.614563829787234) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.055794230866803046) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7457234042553191) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.04655866320804533) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7960106382978723) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.007062805808376465) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5936489361702127) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.02670029318730673) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7498510638297872) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.026313533043186964) - val-aux/sciknoweval/score/mean@16:np.float64(0.7253989361702128) - val-aux/sciknoweval/score/std@16:np.float64(0.08655772986262227) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7613191489361701) - val-aux/sciknoweval/score/best@2/std:np.float64(0.05961052684599058) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6877127659574468) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.08852506830517469) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7250638297872342) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.08731454078539842) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7811808510638297) - val-aux/sciknoweval/score/best@4/std:np.float64(0.030545979305562004) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6498936170212767) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.07794589909969647) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7372234042553192) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.06937401821560567) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7911170212765959) - val-aux/sciknoweval/score/best@8/std:np.float64(0.014284231202351703) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.614563829787234) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.055794230866803046) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7457234042553191) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.04655866320804533) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7960106382978723) - val-aux/sciknoweval/score/best@16/std:np.float64(0.007062805808376465) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5936489361702127) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.02670029318730673) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7498510638297872) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.026313533043186964) - val-core/sciknoweval/acc/mean@16:np.float64(0.7253989361702128) - val-aux/sciknoweval/acc/std@16:np.float64(0.08655772986262227) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7613191489361701) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.05961052684599058) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6877127659574468) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.08852506830517469) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7250638297872342) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.08731454078539842) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7811808510638297) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.030545979305562004) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6498936170212767) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.07794589909969647) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7372234042553192) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.06937401821560567) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7911170212765959) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.014284231202351703) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.614563829787234) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.055794230866803046) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7457234042553191) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.04655866320804533) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7960106382978723) - val-core/sciknoweval/acc/best@16/std:np.float64(0.007062805808376465) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5936489361702127) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.02670029318730673) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7498510638297872) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.026313533043186964) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.55078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003116861218586564 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:0.003116861218586564 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:334.46484375 - response_length/max:717.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.46484375 - response_length_non_aborted/max:717.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:413.3125 - prompt_length/max:482.0 - prompt_length/min:206.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014251284301280975 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1249499879777431) - timing_s/agent_loop/generate_sequences/max:np.float64(6.132532795891166) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7483569873002125) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.132532795891166) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:446 - timing_s/agent_loop/slowest/response_length:717 - timing_s/gen:11.998515015468001 - timing_s/reward:0.06351801194250584 - timing_s/old_log_prob:2.330166148021817 - timing_s/adv:0.5252022966742516 - timing_s/update_actor:15.934441544115543 - timing_s/step:30.939870880916715 - timing_s/testing:37.920565428212285 - timing_s/save_checkpoint:6.606392407789826 - timing_s/stop_profile:0.0001103077083826065 - timing_per_token_ms/gen:0.14013191567064925 - timing_per_token_ms/adv:0.002743559280755215 - timing_per_token_ms/update_actor:0.08323856399494096 - perf/total_num_tokens:191431 - perf/time_per_step:30.939870880916715 - perf/throughput:773.3993167618227 (TaskRunner pid=2167046) Training Progress: 30%|███ | 30/100 [18:49<51:49, 44.42s/it] (TaskRunner pid=2167046) step:31 - global_seqlen/min:21305 - global_seqlen/max:25291 - global_seqlen/minmax_diff:3986 - global_seqlen/balanced_min:23036 - global_seqlen/balanced_max:23041 - global_seqlen/mean:23039.625 - actor/entropy:0.04938299581408501 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9892370104789734 - training/rollout_probs_diff_mean:0.002258756896480918 - training/rollout_probs_diff_std:0.013091050088405609 - training/rollout_actor_probs_pearson_corr:0.9941160082817078 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.0000802278518677 - rollout_corr/rollout_is_ratio_fraction_high:1.2028917808493134e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002165205223718658 - rollout_corr/rollout_is_max:2.550828695297241 - rollout_corr/rollout_is_min:0.0067202127538621426 - rollout_corr/rollout_is_std:0.03224320337176323 - rollout_corr/rollout_is_eff_sample_size:0.9989616934055325 - rollout_corr/rollout_is_seq_mean:1.0001165866851807 - rollout_corr/rollout_is_seq_std:0.00201058154925704 - rollout_corr/rollout_is_seq_max:1.0065182447433472 - rollout_corr/rollout_is_seq_min:0.9951715469360352 - rollout_corr/rollout_is_seq_max_deviation:0.006518244743347168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0496189133264124) - rollout_corr/training_log_ppl:np.float64(0.047763853480319085) - rollout_corr/kl:np.float64(0.0005609981700303024) - rollout_corr/k3_kl:np.float64(0.0007823906714463646) - rollout_corr/rollout_ppl:np.float64(1.0490215150639415) - rollout_corr/rollout_log_ppl:np.float64(0.047202855099840235) - rollout_corr/log_ppl_diff:np.float64(0.0005609983804788499) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018915072992058413) - rollout_corr/log_ppl_diff_max:np.float64(0.018269948661327362) - rollout_corr/log_ppl_diff_min:np.float64(-0.009728636592626572) - rollout_corr/ppl_ratio:np.float64(1.0005653724074364) - rollout_corr/chi2_token:np.float64(0.001830502413213253) - rollout_corr/chi2_seq:np.float64(1.5425163942854851) - actor/pg_loss:np.float64(0.0014576166868209839) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036924146979799843) - actor/ppo_kl:np.float64(6.366286959824663e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(209419.71391360858) - self_distillation/token_reweight_w_std:np.float64(2615016.670161984) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000283203087747) - self_distillation/token_reweight_w_clip_frac:np.float64(0.018031955318292603) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.28416722267866135) - perf/mfu/actor:np.float64(0.04234140442013515) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.89453125) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001890043611638248 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001890043611638248 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:324.73828125 - response_length/max:677.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.73828125 - response_length_non_aborted/max:677.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:395.25 - prompt_length/max:492.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011043809354305267 - timing_s/agent_loop/generate_sequences/min:np.float64(1.864338455721736) - timing_s/agent_loop/generate_sequences/max:np.float64(5.793053427711129) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6212092554196715) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.793053427711129) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:434 - timing_s/agent_loop/slowest/response_length:677 - timing_s/gen:11.567019982263446 - timing_s/reward:0.05429721064865589 - timing_s/old_log_prob:2.397083157673478 - timing_s/adv:0.5327631570398808 - timing_s/update_actor:15.5287045519799 - timing_s/step:30.18702084571123 - timing_s/stop_profile:0.00011414475739002228 - timing_per_token_ms/gen:0.13913872929238022 - timing_per_token_ms/adv:0.0028904721595939646 - timing_per_token_ms/update_actor:0.08424998536206589 - perf/total_num_tokens:184317 - perf/time_per_step:30.18702084571123 - perf/throughput:763.2295057454573 (TaskRunner pid=2167046) Training Progress: 31%|███ | 31/100 [19:19<46:11, 40.17s/it] (TaskRunner pid=2167046) step:32 - global_seqlen/min:18196 - global_seqlen/max:26796 - global_seqlen/minmax_diff:8600 - global_seqlen/balanced_min:22588 - global_seqlen/balanced_max:22596 - global_seqlen/mean:22594.125 - actor/entropy:0.04554832726716995 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.507658839225769 - training/rollout_probs_diff_mean:0.002093696966767311 - training/rollout_probs_diff_std:0.011700347065925598 - training/rollout_actor_probs_pearson_corr:0.9949268102645874 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000239610671997 - rollout_corr/rollout_is_ratio_fraction_high:6.305408896878362e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012610817793756723 - rollout_corr/rollout_is_max:3.8349814414978027 - rollout_corr/rollout_is_min:0.2868506908416748 - rollout_corr/rollout_is_std:0.03249547630548477 - rollout_corr/rollout_is_eff_sample_size:0.9989452769377425 - rollout_corr/rollout_is_seq_mean:1.0000345706939697 - rollout_corr/rollout_is_seq_std:0.0022742771543562412 - rollout_corr/rollout_is_seq_max:1.0197118520736694 - rollout_corr/rollout_is_seq_min:0.9935188293457031 - rollout_corr/rollout_is_seq_max_deviation:0.019711852073669434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0453667039982975) - rollout_corr/training_log_ppl:np.float64(0.04353243351852143) - rollout_corr/kl:np.float64(0.0006469867601166523) - rollout_corr/k3_kl:np.float64(0.0006021329918155516) - rollout_corr/rollout_ppl:np.float64(1.0446764542721212) - rollout_corr/rollout_log_ppl:np.float64(0.04288544589826415) - rollout_corr/log_ppl_diff:np.float64(0.0006469876202572777) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016619250986877887) - rollout_corr/log_ppl_diff_max:np.float64(0.00940629094839096) - rollout_corr/log_ppl_diff_min:np.float64(-0.009224757552146912) - rollout_corr/ppl_ratio:np.float64(1.0006496729329228) - rollout_corr/chi2_token:np.float64(0.0011666147038340569) - rollout_corr/chi2_seq:np.float64(0.5588384328875691) - actor/pg_loss:np.float64(0.0007467931136488914) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025311750505352393) - actor/ppo_kl:np.float64(0.00014509259286654697) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(25800.459359957604) - self_distillation/token_reweight_w_std:np.float64(407525.20962225855) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998260058928281) - self_distillation/token_reweight_w_clip_frac:np.float64(0.00925165212538559) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.14854632876813412) - perf/mfu/actor:np.float64(0.041973087574399694) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.82848739624023) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0028468919917941093 - critic/advantages/max:0.5 - critic/advantages/min:-0.75 - critic/returns/mean:0.0028468919917941093 - critic/returns/max:0.5 - critic/returns/min:-0.75 - response_length/mean:309.75390625 - response_length/max:731.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:309.75390625 - response_length_non_aborted/max:731.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001710951328277588 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8522760216146708) - timing_s/agent_loop/generate_sequences/max:np.float64(5.938589612022042) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.537644969408575) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.938589612022042) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:427 - timing_s/agent_loop/slowest/response_length:731 - timing_s/gen:11.74093264900148 - timing_s/reward:0.05617862567305565 - timing_s/old_log_prob:2.2790592797100544 - timing_s/adv:0.528248555958271 - timing_s/update_actor:15.360722405835986 - timing_s/step:30.06082314439118 - timing_s/stop_profile:0.00011457502841949463 - timing_per_token_ms/gen:0.14806275961261434 - timing_per_token_ms/adv:0.0029224884563922647 - timing_per_token_ms/update_actor:0.08498183933785877 - perf/total_num_tokens:180753 - perf/time_per_step:30.06082314439118 - perf/throughput:751.6136498150306 (TaskRunner pid=2167046) Training Progress: 32%|███▏ | 32/100 [19:49<42:06, 37.15s/it] (TaskRunner pid=2167046) step:33 - global_seqlen/min:18392 - global_seqlen/max:24737 - global_seqlen/minmax_diff:6345 - global_seqlen/balanced_min:22379 - global_seqlen/balanced_max:22384 - global_seqlen/mean:22382.5 - actor/entropy:0.048572197556495667 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6976189017295837 - training/rollout_probs_diff_mean:0.002187278587371111 - training/rollout_probs_diff_std:0.013085621409118176 - training/rollout_actor_probs_pearson_corr:0.9939641952514648 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:1.0001271963119507 - rollout_corr/rollout_is_ratio_fraction_high:9.94480651570484e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00023618915292900056 - rollout_corr/rollout_is_max:3.2328717708587646 - rollout_corr/rollout_is_min:0.02593221887946129 - rollout_corr/rollout_is_std:0.03340000659227371 - rollout_corr/rollout_is_eff_sample_size:0.9988860393883612 - rollout_corr/rollout_is_seq_mean:1.000138759613037 - rollout_corr/rollout_is_seq_std:0.0020758023019880056 - rollout_corr/rollout_is_seq_max:1.0095303058624268 - rollout_corr/rollout_is_seq_min:0.9938346147537231 - rollout_corr/rollout_is_seq_max_deviation:0.009530305862426758 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0498127928003669) - rollout_corr/training_log_ppl:np.float64(0.04764294929736934) - rollout_corr/kl:np.float64(0.000776656330423009) - rollout_corr/k3_kl:np.float64(0.0008650090900808038) - rollout_corr/rollout_ppl:np.float64(1.048987964168191) - rollout_corr/rollout_log_ppl:np.float64(0.04686629328352865) - rollout_corr/log_ppl_diff:np.float64(0.000776656013840693) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018832581390597625) - rollout_corr/log_ppl_diff_max:np.float64(0.01899798959493637) - rollout_corr/log_ppl_diff_min:np.float64(-0.010233291424810886) - rollout_corr/ppl_ratio:np.float64(1.00078120874241) - rollout_corr/chi2_token:np.float64(0.0018827361054718494) - rollout_corr/chi2_seq:np.float64(0.43429425405338407) - actor/pg_loss:np.float64(0.0005226503126323223) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025913522904374986) - actor/ppo_kl:np.float64(0.0002559889147786265) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(285482.3281259318) - self_distillation/token_reweight_w_std:np.float64(3867517.0114729353) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010869828984141) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01669350147858495) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.27803053706884384) - perf/mfu/actor:np.float64(0.0417407421312785) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.80722427368164) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0030891054775565863 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0030891054775565863 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:314.234375 - response_length/max:673.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.234375 - response_length_non_aborted/max:673.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.21875 - prompt_length/max:480.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015045702457427979 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9350745882838964) - timing_s/agent_loop/generate_sequences/max:np.float64(5.753504239022732) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5311595983803272) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.753504239022732) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:419 - timing_s/agent_loop/slowest/response_length:673 - timing_s/gen:11.515646539628506 - timing_s/reward:0.060460347682237625 - timing_s/old_log_prob:2.275907948613167 - timing_s/adv:0.5435283668339252 - timing_s/update_actor:15.336240395903587 - timing_s/step:29.82015441171825 - timing_s/stop_profile:0.00013785436749458313 - timing_per_token_ms/gen:0.14315109317821723 - timing_per_token_ms/adv:0.0030354538525294607 - timing_per_token_ms/update_actor:0.08564861161567959 - perf/total_num_tokens:179060 - perf/time_per_step:29.82015441171825 - perf/throughput:750.5829678468896 (TaskRunner pid=2167046) Training Progress: 33%|███▎ | 33/100 [20:19<39:03, 34.97s/it] (TaskRunner pid=2167046) step:34 - global_seqlen/min:21779 - global_seqlen/max:25692 - global_seqlen/minmax_diff:3913 - global_seqlen/balanced_min:23445 - global_seqlen/balanced_max:23465 - global_seqlen/mean:23448.125 - actor/entropy:0.04247506335377693 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3394765853881836 - training/rollout_probs_diff_mean:0.0020800086203962564 - training/rollout_probs_diff_std:0.011497821658849716 - training/rollout_actor_probs_pearson_corr:0.9946447610855103 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.8125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.000171422958374 - rollout_corr/rollout_is_ratio_fraction_high:7.357179129030555e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001471435825806111 - rollout_corr/rollout_is_max:9.17504596710205 - rollout_corr/rollout_is_min:0.22716903686523438 - rollout_corr/rollout_is_std:0.031074028462171555 - rollout_corr/rollout_is_eff_sample_size:0.9990353362293889 - rollout_corr/rollout_is_seq_mean:1.0002288818359375 - rollout_corr/rollout_is_seq_std:0.002648404100909829 - rollout_corr/rollout_is_seq_max:1.029280662536621 - rollout_corr/rollout_is_seq_min:0.9955718517303467 - rollout_corr/rollout_is_seq_max_deviation:0.029280662536621094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0434476546943188) - rollout_corr/training_log_ppl:np.float64(0.04219085097838615) - rollout_corr/kl:np.float64(0.0005611525840052423) - rollout_corr/k3_kl:np.float64(0.000653065330164182) - rollout_corr/rollout_ppl:np.float64(1.0428576031699777) - rollout_corr/rollout_log_ppl:np.float64(0.041629698092947365) - rollout_corr/log_ppl_diff:np.float64(0.000561152885438787) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015396711655739637) - rollout_corr/log_ppl_diff_max:np.float64(0.009032197296619415) - rollout_corr/log_ppl_diff_min:np.float64(-0.006443668156862259) - rollout_corr/ppl_ratio:np.float64(1.000563275301829) - rollout_corr/chi2_token:np.float64(0.001628566300496459) - rollout_corr/chi2_seq:np.float64(0.45253721135668457) - actor/pg_loss:np.float64(0.0004543724935501814) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000334290034516016) - actor/ppo_kl:np.float64(0.00019341592874866365) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_w_mean:np.float64(151244.62870255834) - self_distillation/token_reweight_w_std:np.float64(1700770.1650854703) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998153406195343) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011545672423380893) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.24764136131852865) - perf/mfu/actor:np.float64(0.043330077101173106) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.78245162963867) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.8125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014346498064696789 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014346498064696789 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:318.56640625 - response_length/max:811.0 - response_length/min:155.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:318.56640625 - response_length_non_aborted/max:811.0 - response_length_non_aborted/min:155.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:414.1875 - prompt_length/max:482.0 - prompt_length/min:271.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.432807564735413e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8409772999584675) - timing_s/agent_loop/generate_sequences/max:np.float64(6.334641754627228) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6130348054721253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.334641754627228) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:428 - timing_s/agent_loop/slowest/response_length:811 - timing_s/gen:12.351888993754983 - timing_s/reward:0.07414030469954014 - timing_s/old_log_prob:2.325253529474139 - timing_s/adv:0.5831031519919634 - timing_s/update_actor:15.37999014928937 - timing_s/step:30.796508744359016 - timing_s/stop_profile:0.00013626553118228912 - timing_per_token_ms/gen:0.15145842573240695 - timing_per_token_ms/adv:0.0031084743022734406 - timing_per_token_ms/update_actor:0.08198944558087998 - perf/total_num_tokens:187585 - perf/time_per_step:30.796508744359016 - perf/throughput:761.3890650606616 (TaskRunner pid=2167046) Training Progress: 34%|███▍ | 34/100 [20:50<37:06, 33.73s/it] (TaskRunner pid=2167046) step:35 - global_seqlen/min:20095 - global_seqlen/max:24523 - global_seqlen/minmax_diff:4428 - global_seqlen/balanced_min:21778 - global_seqlen/balanced_max:21780 - global_seqlen/mean:21779.25 - actor/entropy:0.04991421476006508 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48201340436935425 - training/rollout_probs_diff_mean:0.002308846917003393 - training/rollout_probs_diff_std:0.012316430918872356 - training/rollout_actor_probs_pearson_corr:0.9947492480278015 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0002334117889404 - rollout_corr/rollout_is_ratio_fraction_high:1.3196442523621954e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013196442159824073 - rollout_corr/rollout_is_max:3.959545135498047 - rollout_corr/rollout_is_min:0.29720374941825867 - rollout_corr/rollout_is_std:0.0317457914352417 - rollout_corr/rollout_is_eff_sample_size:0.9989936951727617 - rollout_corr/rollout_is_seq_mean:1.0001981258392334 - rollout_corr/rollout_is_seq_std:0.0018588469829410315 - rollout_corr/rollout_is_seq_max:1.0078638792037964 - rollout_corr/rollout_is_seq_min:0.9947773814201355 - rollout_corr/rollout_is_seq_max_deviation:0.007863879203796387 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0496695381589234) - rollout_corr/training_log_ppl:np.float64(0.048008743393438635) - rollout_corr/kl:np.float64(0.0024177409742709155) - rollout_corr/k3_kl:np.float64(0.002347025056231189) - rollout_corr/rollout_ppl:np.float64(1.0471166698262095) - rollout_corr/rollout_log_ppl:np.float64(0.045591002022774774) - rollout_corr/log_ppl_diff:np.float64(0.002417741381577798) - rollout_corr/log_ppl_abs_diff:np.float64(0.003363729747434263) - rollout_corr/log_ppl_diff_max:np.float64(0.10936760157346725) - rollout_corr/log_ppl_diff_min:np.float64(-0.0114052165299654) - rollout_corr/ppl_ratio:np.float64(1.0024650054983795) - rollout_corr/chi2_token:np.float64(0.004743414698168635) - rollout_corr/chi2_seq:np.float64(0.5235199017915875) - actor/pg_loss:np.float64(0.0007961411029100418) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005753537584496371) - actor/ppo_kl:np.float64(0.002120418961538917) - actor/pg_clipfrac_lower:np.float64(2.271075572934933e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(210541.37167732976) - self_distillation/token_reweight_w_std:np.float64(1888247.1145525659) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001017071772367) - self_distillation/token_reweight_w_clip_frac:np.float64(0.008670316565257963) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.49630653485655785) - perf/mfu/actor:np.float64(0.039790707574217724) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.80047988891602) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018722452223300934 - critic/advantages/max:0.5 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0018722452223300934 - critic/returns/max:0.5 - critic/returns/min:-0.75 - response_length/mean:296.0078125 - response_length/max:574.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.0078125 - response_length_non_aborted/max:574.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:384.59375 - prompt_length/max:459.0 - prompt_length/min:274.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011700764298439026 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1263145357370377) - timing_s/agent_loop/generate_sequences/max:np.float64(5.01327658444643) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0205636951941415) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.01327658444643) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:416 - timing_s/agent_loop/slowest/response_length:574 - timing_s/gen:10.814715780317783 - timing_s/reward:0.05929145961999893 - timing_s/old_log_prob:2.238602187484503 - timing_s/adv:0.529649157077074 - timing_s/update_actor:15.761729579418898 - timing_s/step:29.49290233105421 - timing_s/stop_profile:0.0001153554767370224 - timing_per_token_ms/gen:0.14271577212802902 - timing_per_token_ms/adv:0.003039872568368252 - timing_per_token_ms/update_actor:0.09046299562323598 - perf/total_num_tokens:174234 - perf/time_per_step:29.49290233105421 - perf/throughput:738.4573330739237 (TaskRunner pid=2167046) Training Progress: 35%|███▌ | 35/100 [21:19<35:10, 32.48s/it] (TaskRunner pid=2167046) step:36 - global_seqlen/min:20820 - global_seqlen/max:26212 - global_seqlen/minmax_diff:5392 - global_seqlen/balanced_min:23515 - global_seqlen/balanced_max:23517 - global_seqlen/mean:23516.25 - actor/entropy:0.05420038849115372 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.002247673226520419 - training/rollout_probs_diff_std:0.011697757057845592 - training/rollout_actor_probs_pearson_corr:0.9956129789352417 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0001267194747925 - rollout_corr/rollout_is_ratio_fraction_high:6.95265262038447e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015064080071169883 - rollout_corr/rollout_is_max:3.5453145503997803 - rollout_corr/rollout_is_min:0.3200256824493408 - rollout_corr/rollout_is_std:0.031670600175857544 - rollout_corr/rollout_is_eff_sample_size:0.9989980970632323 - rollout_corr/rollout_is_seq_mean:1.0001379251480103 - rollout_corr/rollout_is_seq_std:0.0017661351012066007 - rollout_corr/rollout_is_seq_max:1.0061100721359253 - rollout_corr/rollout_is_seq_min:0.9940493106842041 - rollout_corr/rollout_is_seq_max_deviation:0.006110072135925293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.053559670690447) - rollout_corr/training_log_ppl:np.float64(0.051443011833725905) - rollout_corr/kl:np.float64(0.0007316862326405271) - rollout_corr/k3_kl:np.float64(0.0007606058412104044) - rollout_corr/rollout_ppl:np.float64(1.052789440844208) - rollout_corr/rollout_log_ppl:np.float64(0.05071132522425614) - rollout_corr/log_ppl_diff:np.float64(0.0007316866058317828) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018277808030688902) - rollout_corr/log_ppl_diff_max:np.float64(0.021980972960591316) - rollout_corr/log_ppl_diff_min:np.float64(-0.004337247461080551) - rollout_corr/ppl_ratio:np.float64(1.0007357993163168) - rollout_corr/chi2_token:np.float64(0.0014476962387561798) - rollout_corr/chi2_seq:np.float64(0.6039434908889234) - actor/pg_loss:np.float64(0.00037011189851909876) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023756584459988517) - actor/ppo_kl:np.float64(0.0003662115519347253) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(44200.58313611476) - self_distillation/token_reweight_w_std:np.float64(806335.131862045) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007616139482707) - self_distillation/token_reweight_w_clip_frac:np.float64(0.00920598856464494) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.11965901032090187) - perf/mfu/actor:np.float64(0.04255113735653425) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.83895874023438) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-4.3454077967908233e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-4.3454077967908233e-05 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:337.1015625 - response_length/max:587.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:337.1015625 - response_length_non_aborted/max:587.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.78125 - prompt_length/max:509.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001510176807641983 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6326084788888693) - timing_s/agent_loop/generate_sequences/max:np.float64(5.461965553462505) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6772474792960566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.461965553462505) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:583 - timing_s/gen:11.392364293336868 - timing_s/reward:0.060090767219662666 - timing_s/old_log_prob:2.2642620000988245 - timing_s/adv:0.5239562820643187 - timing_s/update_actor:15.78369846753776 - timing_s/step:30.11077237688005 - timing_s/stop_profile:2.750009298324585e-05 - timing_per_token_ms/gen:0.13201191561029071 - timing_per_token_ms/adv:0.0027850756501585 - timing_per_token_ms/update_actor:0.08389782845658726 - perf/total_num_tokens:188130 - perf/time_per_step:30.11077237688005 - perf/throughput:780.9912580673779 (TaskRunner pid=2167046) Training Progress: 36%|███▌ | 36/100 [21:49<33:54, 31.78s/it] (TaskRunner pid=2167046) step:37 - global_seqlen/min:18243 - global_seqlen/max:24615 - global_seqlen/minmax_diff:6372 - global_seqlen/balanced_min:21724 - global_seqlen/balanced_max:21729 - global_seqlen/mean:21726.625 - actor/entropy:0.05331616848707199 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3394768238067627 - training/rollout_probs_diff_mean:0.0020236121490597725 - training/rollout_probs_diff_std:0.010713296942412853 - training/rollout_actor_probs_pearson_corr:0.9962490200996399 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.8359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:1.0001013278961182 - rollout_corr/rollout_is_ratio_fraction_high:3.556651427061297e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015412156062666327 - rollout_corr/rollout_is_max:3.9899837970733643 - rollout_corr/rollout_is_min:0.09489698708057404 - rollout_corr/rollout_is_std:0.02917863428592682 - rollout_corr/rollout_is_eff_sample_size:0.9991495695201904 - rollout_corr/rollout_is_seq_mean:1.0000089406967163 - rollout_corr/rollout_is_seq_std:0.002048109658062458 - rollout_corr/rollout_is_seq_max:1.0107276439666748 - rollout_corr/rollout_is_seq_min:0.9943196177482605 - rollout_corr/rollout_is_seq_max_deviation:0.010727643966674805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0528704207390547) - rollout_corr/training_log_ppl:np.float64(0.05075585774739011) - rollout_corr/kl:np.float64(0.0006263373954951135) - rollout_corr/k3_kl:np.float64(0.0006165275314771179) - rollout_corr/rollout_ppl:np.float64(1.052202373277396) - rollout_corr/rollout_log_ppl:np.float64(0.05012952038759977) - rollout_corr/log_ppl_diff:np.float64(0.000626337359790341) - rollout_corr/log_ppl_abs_diff:np.float64(0.001629049289476825) - rollout_corr/log_ppl_diff_max:np.float64(0.012944355607032776) - rollout_corr/log_ppl_diff_min:np.float64(-0.012586958706378937) - rollout_corr/ppl_ratio:np.float64(1.0006291172467172) - rollout_corr/chi2_token:np.float64(0.0011920023243874311) - rollout_corr/chi2_seq:np.float64(0.7796085118316114) - actor/pg_loss:np.float64(0.0013440322363749146) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001347399347650935) - actor/ppo_kl:np.float64(7.59294577008518e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_w_mean:np.float64(109273.2849380807) - self_distillation/token_reweight_w_std:np.float64(1662628.403000201) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000219531590119) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015951965760905296) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.24890022352337837) - perf/mfu/actor:np.float64(0.03961164662388422) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.77448272705078) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.8359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0019176279893144965 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0019176279893144965 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:329.48828125 - response_length/max:807.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:329.48828125 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:349.46875 - prompt_length/max:491.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.014589548110962e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7062291856855154) - timing_s/agent_loop/generate_sequences/max:np.float64(6.44104958884418) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5039435348735424) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.44104958884418) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:796 - timing_s/gen:12.299582397565246 - timing_s/reward:0.05889640003442764 - timing_s/old_log_prob:2.3164712488651276 - timing_s/adv:0.5373764801770449 - timing_s/update_actor:15.747590590268373 - timing_s/step:31.048775907605886 - timing_s/stop_profile:0.00012016668915748596 - timing_per_token_ms/gen:0.14581776188888126 - timing_per_token_ms/adv:0.003091693257564422 - timing_per_token_ms/update_actor:0.09060076398352467 - perf/total_num_tokens:173813 - perf/time_per_step:31.048775907605886 - perf/throughput:699.7578604919404 (TaskRunner pid=2167046) Training Progress: 37%|███▋ | 37/100 [22:21<33:09, 31.58s/it] (TaskRunner pid=2167046) (TaskRunner pid=2167046) step:38 - global_seqlen/min:19549 - global_seqlen/max:25117 - global_seqlen/minmax_diff:5568 - global_seqlen/balanced_min:23218 - global_seqlen/balanced_max:23221 - global_seqlen/mean:23219.875 - actor/entropy:0.05836797133088112 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6118535995483398 - training/rollout_probs_diff_mean:0.00232762866653502 - training/rollout_probs_diff_std:0.011830761097371578 - training/rollout_actor_probs_pearson_corr:0.9958423972129822 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9998059272766113 - rollout_corr/rollout_is_ratio_fraction_high:4.672078648582101e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016352275270037353 - rollout_corr/rollout_is_max:2.6164438724517822 - rollout_corr/rollout_is_min:0.0921701192855835 - rollout_corr/rollout_is_std:0.031520623713731766 - rollout_corr/rollout_is_eff_sample_size:0.9990072578785806 - rollout_corr/rollout_is_seq_mean:0.9997893571853638 - rollout_corr/rollout_is_seq_std:0.0019738837145268917 - rollout_corr/rollout_is_seq_max:1.0098562240600586 - rollout_corr/rollout_is_seq_min:0.9930856823921204 - rollout_corr/rollout_is_seq_max_deviation:0.009856224060058594 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0563341127708554) - rollout_corr/training_log_ppl:np.float64(0.054111441248096526) - rollout_corr/kl:np.float64(0.0006969592394456825) - rollout_corr/k3_kl:np.float64(0.0005701098419429451) - rollout_corr/rollout_ppl:np.float64(1.0555989309214056) - rollout_corr/rollout_log_ppl:np.float64(0.053414481904837885) - rollout_corr/log_ppl_diff:np.float64(0.0006969593432586407) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015633855382475303) - rollout_corr/log_ppl_diff_max:np.float64(0.008656919002532959) - rollout_corr/log_ppl_diff_min:np.float64(-0.005964074283838272) - rollout_corr/ppl_ratio:np.float64(1.000699280994013) - rollout_corr/chi2_token:np.float64(0.000926322303712368) - rollout_corr/chi2_seq:np.float64(0.32595846615731716) - actor/pg_loss:np.float64(0.0011332343565300107) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003224353213227005) - actor/ppo_kl:np.float64(-6.34805843731101e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(184414.60750435363) - self_distillation/token_reweight_w_std:np.float64(2728702.5535712354) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006198440678418) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01655629460583441) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.27920282259583473) - perf/mfu/actor:np.float64(0.042506767077513594) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.75471115112305) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024061205331236124 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0024061205331236124 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:334.43359375 - response_length/max:699.0 - response_length/min:156.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.43359375 - response_length_non_aborted/max:699.0 - response_length_non_aborted/min:156.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.1875 - prompt_length/max:541.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013744644820690155 - timing_s/agent_loop/generate_sequences/min:np.float64(1.898912001401186) - timing_s/agent_loop/generate_sequences/max:np.float64(5.927438881248236) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.701512742743944) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.927438881248236) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:699 - timing_s/gen:11.834541860967875 - timing_s/reward:0.06558134220540524 - timing_s/old_log_prob:2.280883513391018 - timing_s/adv:0.5150403250008821 - timing_s/update_actor:15.6824850179255 - timing_s/step:30.473918160423636 - timing_s/stop_profile:0.0001121722161769867 - timing_per_token_ms/gen:0.13822977119626087 - timing_per_token_ms/adv:0.0027726264945487546 - timing_per_token_ms/update_actor:0.08442382343749429 - perf/total_num_tokens:185759 - perf/time_per_step:30.473918160423636 - perf/throughput:761.9589603727284 (TaskRunner pid=2167046) Training Progress: 38%|███▊ | 38/100 [22:51<32:18, 31.26s/it] (TaskRunner pid=2167046) step:39 - global_seqlen/min:20942 - global_seqlen/max:27210 - global_seqlen/minmax_diff:6268 - global_seqlen/balanced_min:23015 - global_seqlen/balanced_max:23017 - global_seqlen/mean:23015.625 - actor/entropy:0.047281794250011444 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083141922950745 - training/rollout_probs_diff_mean:0.0020674800034612417 - training/rollout_probs_diff_std:0.011524595320224762 - training/rollout_actor_probs_pearson_corr:0.9951632022857666 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.75390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0000718832015991 - rollout_corr/rollout_is_ratio_fraction_high:3.527876106090844e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.231710671680048e-05 - rollout_corr/rollout_is_max:2.6066808700561523 - rollout_corr/rollout_is_min:0.29022958874702454 - rollout_corr/rollout_is_std:0.030541982501745224 - rollout_corr/rollout_is_eff_sample_size:0.9990680566378604 - rollout_corr/rollout_is_seq_mean:1.0000585317611694 - rollout_corr/rollout_is_seq_std:0.001705444767139852 - rollout_corr/rollout_is_seq_max:1.0067297220230103 - rollout_corr/rollout_is_seq_min:0.9937018752098083 - rollout_corr/rollout_is_seq_max_deviation:0.006729722023010254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.04436213709414) - rollout_corr/training_log_ppl:np.float64(0.04305056084012904) - rollout_corr/kl:np.float64(0.0005853749285122944) - rollout_corr/k3_kl:np.float64(0.0005738799877832435) - rollout_corr/rollout_ppl:np.float64(1.043735504616052) - rollout_corr/rollout_log_ppl:np.float64(0.04246518543368438) - rollout_corr/log_ppl_diff:np.float64(0.0005853754064446548) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016567986331210705) - rollout_corr/log_ppl_diff_max:np.float64(0.014873620122671127) - rollout_corr/log_ppl_diff_min:np.float64(-0.0048823282122612) - rollout_corr/ppl_ratio:np.float64(1.0005881253164262) - rollout_corr/chi2_token:np.float64(0.0010363354813307524) - rollout_corr/chi2_seq:np.float64(0.46266718837432563) - actor/pg_loss:np.float64(0.000453876331448555) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011933857240364887) - actor/ppo_kl:np.float64(0.0001878124741701015) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75390625) - self_distillation/token_reweight_w_mean:np.float64(77515.16053502122) - self_distillation/token_reweight_w_std:np.float64(965262.9837699123) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998419873882085) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0069773956056451425) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.11174572631716728) - perf/mfu/actor:np.float64(0.042077119502450376) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.9169692993164) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008996084216050804 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008996084216050804 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:332.17578125 - response_length/max:778.0 - response_length/min:144.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:332.17578125 - response_length_non_aborted/max:778.0 - response_length_non_aborted/min:144.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:387.0625 - prompt_length/max:505.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.673647582530975e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.59378319978714) - timing_s/agent_loop/generate_sequences/max:np.float64(6.173383612185717) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.569423519220436) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.173383612185717) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:778 - timing_s/gen:11.949954964220524 - timing_s/reward:0.05974843166768551 - timing_s/old_log_prob:2.3209459595382214 - timing_s/adv:0.5298623107373714 - timing_s/update_actor:15.767534146085382 - timing_s/step:30.713386733084917 - timing_s/stop_profile:5.6201592087745667e-05 - timing_per_token_ms/gen:0.1405265350873211 - timing_per_token_ms/adv:0.0028777314907664437 - timing_per_token_ms/update_actor:0.08563494444581334 - perf/total_num_tokens:184125 - perf/time_per_step:30.713386733084917 - perf/throughput:749.3678635969907 (TaskRunner pid=2167046) Training Progress: 39%|███▉ | 39/100 [23:22<31:37, 31.10s/it] (TaskRunner pid=2167046) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2167046) validation generation end (TaskRunner pid=2167046) [prompt] system (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ V = a \times b \times c \times \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) However, since the given angles α, β, and γ are all 90.00º, which are right angles, the crystal structure is a rectangular prism (orthorhombic). In this case, the formula simplifies to: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ V = a \times b \times c $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given: (TaskRunner pid=2167046) - $ a = 7.82 $ Å (TaskRunner pid=2167046) - $ b = 7.82 $ Å (TaskRunner pid=2167046) - $ c = 17.88 $ Å (TaskRunner pid=2167046) (TaskRunner pid=2167046) Substituting these values into the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ V = 7.82 \times 7.82 \times 17.88 $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ 7.82 \times 7.82 = 61.1524 $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then multiply by 17.88: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ 61.1524 \times 17.88 \approx 1094.31 $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2167440) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2167046) step:40 - global_seqlen/min:18634 - global_seqlen/max:24321 - global_seqlen/minmax_diff:5687 - global_seqlen/balanced_min:22272 - global_seqlen/balanced_max:22276 - global_seqlen/mean:22274.625 - actor/entropy:0.0525180958211422 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9980729222297668 - training/rollout_probs_diff_mean:0.0023294829297810793 - training/rollout_probs_diff_std:0.014080265536904335 - training/rollout_actor_probs_pearson_corr:0.9934572577476501 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.8046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.99977707862854 - rollout_corr/rollout_is_ratio_fraction_high:4.945292675984092e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00021017494145780802 - rollout_corr/rollout_is_max:3.0684378147125244 - rollout_corr/rollout_is_min:0.00033825141144916415 - rollout_corr/rollout_is_std:0.03151306137442589 - rollout_corr/rollout_is_eff_sample_size:0.9990076147968647 - rollout_corr/rollout_is_seq_mean:0.9997336268424988 - rollout_corr/rollout_is_seq_std:0.002147440332919359 - rollout_corr/rollout_is_seq_max:1.0140759944915771 - rollout_corr/rollout_is_seq_min:0.9931351542472839 - rollout_corr/rollout_is_seq_max_deviation:0.014075994491577148 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0521052996627986) - rollout_corr/training_log_ppl:np.float64(0.05022077127978264) - rollout_corr/kl:np.float64(0.001029036797591143) - rollout_corr/k3_kl:np.float64(0.0009319911036609341) - rollout_corr/rollout_ppl:np.float64(1.050995112862438) - rollout_corr/rollout_log_ppl:np.float64(0.04919173403959576) - rollout_corr/log_ppl_diff:np.float64(0.0010290372401868808) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019625763334261137) - rollout_corr/log_ppl_diff_max:np.float64(0.025801215320825577) - rollout_corr/log_ppl_diff_min:np.float64(-0.005766719579696655) - rollout_corr/ppl_ratio:np.float64(1.001035234425217) - rollout_corr/chi2_token:np.float64(0.0011496609076857567) - rollout_corr/chi2_seq:np.float64(0.39592411555349827) - actor/pg_loss:np.float64(0.0008833230240270495) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00013724410791837727) - actor/ppo_kl:np.float64(-2.584329278576547e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_w_mean:np.float64(89189.93140461063) - self_distillation/token_reweight_w_std:np.float64(1213222.806374929) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000462993048131) - self_distillation/token_reweight_w_clip_frac:np.float64(0.008561011323763523) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.2542172442190349) - perf/mfu/actor:np.float64(0.04111878608146088) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.78087615966797) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7420212765957447) - val-aux/sciknoweval/reward/std@16:np.float64(0.04708054278965243) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7608510638297873) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.03635224696233118) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7230425531914892) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.04401129297371908) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7417340425531915) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.047178616354697986) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7729893617021276) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.023913360956787883) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7053510638297872) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.037173190452069925) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7458723404255319) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.03605128712188153) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7813191489361704) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0169222680326505) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.689468085106383) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.027399915554920624) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7496063829787233) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.024915362991156435) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.789159574468085) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.01353371420859435) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6784042553191489) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.016587731035100663) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7521914893617022) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.018473796207044105) - val-aux/sciknoweval/score/mean@16:np.float64(0.7420212765957447) - val-aux/sciknoweval/score/std@16:np.float64(0.04708054278965243) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7608510638297873) - val-aux/sciknoweval/score/best@2/std:np.float64(0.03635224696233118) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7230425531914892) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.04401129297371908) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7417340425531915) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.047178616354697986) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7729893617021276) - val-aux/sciknoweval/score/best@4/std:np.float64(0.023913360956787883) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7053510638297872) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.037173190452069925) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7458723404255319) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.03605128712188153) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7813191489361704) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0169222680326505) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.689468085106383) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.027399915554920624) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7496063829787233) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.024915362991156435) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.789159574468085) - val-aux/sciknoweval/score/best@16/std:np.float64(0.01353371420859435) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6784042553191489) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.016587731035100663) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7521914893617022) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.018473796207044105) - val-core/sciknoweval/acc/mean@16:np.float64(0.7420212765957447) - val-aux/sciknoweval/acc/std@16:np.float64(0.04708054278965243) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7608510638297873) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.03635224696233118) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7230425531914892) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.04401129297371908) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7417340425531915) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.047178616354697986) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7729893617021276) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.023913360956787883) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7053510638297872) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.037173190452069925) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7458723404255319) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.03605128712188153) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7813191489361704) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0169222680326505) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.689468085106383) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.027399915554920624) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7496063829787233) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.024915362991156435) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.789159574468085) - val-core/sciknoweval/acc/best@16/std:np.float64(0.01353371420859435) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6784042553191489) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.016587731035100663) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7521914893617022) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.018473796207044105) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0028729059267789125 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0028729059267789125 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:315.95703125 - response_length/max:735.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:315.95703125 - response_length_non_aborted/max:735.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:380.125 - prompt_length/max:468.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.007721483707428e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.688046870753169) - timing_s/agent_loop/generate_sequences/max:np.float64(5.831362264230847) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3310174102371093) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.831362264230847) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:735 - timing_s/gen:11.758395558223128 - timing_s/reward:0.05974454991519451 - timing_s/old_log_prob:2.2759518064558506 - timing_s/adv:0.543281452730298 - timing_s/update_actor:15.447217021137476 - timing_s/step:30.17333797737956 - timing_s/testing:37.50606122426689 - timing_s/save_checkpoint:6.6206671092659235 - timing_s/stop_profile:4.922039806842804e-05 - timing_per_token_ms/gen:0.14537176928012768 - timing_per_token_ms/adv:0.0030487687936962913 - timing_per_token_ms/update_actor:0.08668617889828378 - perf/total_num_tokens:178197 - perf/time_per_step:30.17333797737956 - perf/throughput:738.2221024634035 (TaskRunner pid=2167046) Training Progress: 40%|████ | 40/100 [24:36<44:04, 44.07s/it] (TaskRunner pid=2167046) step:41 - global_seqlen/min:21642 - global_seqlen/max:25611 - global_seqlen/minmax_diff:3969 - global_seqlen/balanced_min:23700 - global_seqlen/balanced_max:23701 - global_seqlen/mean:23700.75 - actor/entropy:0.04867272451519966 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6760060787200928 - training/rollout_probs_diff_mean:0.0020650848746299744 - training/rollout_probs_diff_std:0.011649777181446552 - training/rollout_actor_probs_pearson_corr:0.9952284097671509 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:1.0000721216201782 - rollout_corr/rollout_is_ratio_fraction_high:9.189698175759986e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.189698175759986e-05 - rollout_corr/rollout_is_max:3.1071109771728516 - rollout_corr/rollout_is_min:0.3156760036945343 - rollout_corr/rollout_is_std:0.03104524314403534 - rollout_corr/rollout_is_eff_sample_size:0.9990371209236779 - rollout_corr/rollout_is_seq_mean:1.0000920295715332 - rollout_corr/rollout_is_seq_std:0.00191143446136266 - rollout_corr/rollout_is_seq_max:1.0089445114135742 - rollout_corr/rollout_is_seq_min:0.9925116300582886 - rollout_corr/rollout_is_seq_max_deviation:0.008944511413574219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.048692300915718) - rollout_corr/training_log_ppl:np.float64(0.0470506060291882) - rollout_corr/kl:np.float64(0.0007030818600597399) - rollout_corr/k3_kl:np.float64(0.0006832498430933143) - rollout_corr/rollout_ppl:np.float64(1.0479420255869627) - rollout_corr/rollout_log_ppl:np.float64(0.04634752364108863) - rollout_corr/log_ppl_diff:np.float64(0.000703082388099574) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017172047000713064) - rollout_corr/log_ppl_diff_max:np.float64(0.012457795441150665) - rollout_corr/log_ppl_diff_min:np.float64(-0.007961910218000412) - rollout_corr/ppl_ratio:np.float64(1.000706325750798) - rollout_corr/chi2_token:np.float64(0.0012485028710216284) - rollout_corr/chi2_seq:np.float64(0.4721558114979416) - actor/pg_loss:np.float64(0.0008119867416098714) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003208621001249412) - actor/ppo_kl:np.float64(0.00029550659711219396) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(269301.0433200388) - self_distillation/token_reweight_w_std:np.float64(3263202.0098572485) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000196203123778) - self_distillation/token_reweight_w_clip_frac:np.float64(0.012059328131726943) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.1768742771819234) - perf/mfu/actor:np.float64(0.04364625047600226) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.9280548095703) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00124348106328398 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00124348106328398 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:340.0546875 - response_length/max:766.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:340.0546875 - response_length_non_aborted/max:766.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:400.59375 - prompt_length/max:499.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.827867865562439e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6161924973130226) - timing_s/agent_loop/generate_sequences/max:np.float64(6.236892959102988) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6952779031853424) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.236892959102988) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:279 - timing_s/agent_loop/slowest/response_length:766 - timing_s/gen:12.05727044120431 - timing_s/reward:0.05516492575407028 - timing_s/old_log_prob:2.4536157324910164 - timing_s/adv:0.545894231647253 - timing_s/update_actor:15.558406751602888 - timing_s/step:30.782631278038025 - timing_s/stop_profile:0.00012810342013835907 - timing_per_token_ms/gen:0.13850334782094229 - timing_per_token_ms/adv:0.002879097874789052 - timing_per_token_ms/update_actor:0.08205651061465823 - perf/total_num_tokens:189606 - perf/time_per_step:30.782631278038025 - perf/throughput:769.9390538101719 (TaskRunner pid=2167046) Training Progress: 41%|████ | 41/100 [25:07<39:26, 40.10s/it] (TaskRunner pid=2167046) step:42 - global_seqlen/min:22778 - global_seqlen/max:27169 - global_seqlen/minmax_diff:4391 - global_seqlen/balanced_min:24454 - global_seqlen/balanced_max:24460 - global_seqlen/mean:24458.25 - actor/entropy:0.05291866511106491 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6709703207015991 - training/rollout_probs_diff_mean:0.0021318153012543917 - training/rollout_probs_diff_std:0.011878360994160175 - training/rollout_actor_probs_pearson_corr:0.9954165816307068 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0000886917114258 - rollout_corr/rollout_is_ratio_fraction_high:5.514259828487411e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011028519656974822 - rollout_corr/rollout_is_max:3.494861125946045 - rollout_corr/rollout_is_min:0.09653104096651077 - rollout_corr/rollout_is_std:0.030159026384353638 - rollout_corr/rollout_is_eff_sample_size:0.9990914976663352 - rollout_corr/rollout_is_seq_mean:1.0000706911087036 - rollout_corr/rollout_is_seq_std:0.0021617193706333637 - rollout_corr/rollout_is_seq_max:1.01577627658844 - rollout_corr/rollout_is_seq_min:0.9939882159233093 - rollout_corr/rollout_is_seq_max_deviation:0.01577627658843994 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0560827553272247) - rollout_corr/training_log_ppl:np.float64(0.053460831095435424) - rollout_corr/kl:np.float64(0.0005555153836205307) - rollout_corr/k3_kl:np.float64(0.0007181561104125933) - rollout_corr/rollout_ppl:np.float64(1.0554866567254066) - rollout_corr/rollout_log_ppl:np.float64(0.05290531515947805) - rollout_corr/log_ppl_diff:np.float64(0.0005555159359573736) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016266171451206901) - rollout_corr/log_ppl_diff_max:np.float64(0.00821683555841446) - rollout_corr/log_ppl_diff_min:np.float64(-0.008203966543078423) - rollout_corr/ppl_ratio:np.float64(1.00055792927742) - rollout_corr/chi2_token:np.float64(0.0020039721857756376) - rollout_corr/chi2_seq:np.float64(0.6317513328976929) - actor/pg_loss:np.float64(0.0017515807412564754) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029606416092065047) - actor/ppo_kl:np.float64(9.767170322305674e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(114590.53960387851) - self_distillation/token_reweight_w_std:np.float64(1339969.1054521985) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998462598305196) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01849362153734546) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2964472509920597) - perf/mfu/actor:np.float64(0.04507077937465367) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.83689880371094) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009236385230906308 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0009236385230906308 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:354.1953125 - response_length/max:643.0 - response_length/min:158.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:354.1953125 - response_length_non_aborted/max:643.0 - response_length_non_aborted/min:158.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:410.125 - prompt_length/max:510.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010282173752784729 - timing_s/agent_loop/generate_sequences/min:np.float64(1.766824560239911) - timing_s/agent_loop/generate_sequences/max:np.float64(5.934482607990503) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9740968370097107) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.934482607990503) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:643 - timing_s/gen:11.804343018680811 - timing_s/reward:0.06125390715897083 - timing_s/old_log_prob:2.269976668059826 - timing_s/adv:0.5357288736850023 - timing_s/update_actor:15.710900589823723 - timing_s/step:30.46932632289827 - timing_s/stop_profile:4.24850732088089e-05 - timing_per_token_ms/gen:0.1301844301418357 - timing_per_token_ms/adv:0.0027379763151748507 - timing_per_token_ms/update_actor:0.08029448442664398 - perf/total_num_tokens:195666 - perf/time_per_step:30.46932632289827 - perf/throughput:802.7171241268688 (TaskRunner pid=2167046) Training Progress: 42%|████▏ | 42/100 [25:37<35:58, 37.22s/it] (TaskRunner pid=2167046) step:43 - global_seqlen/min:20322 - global_seqlen/max:27608 - global_seqlen/minmax_diff:7286 - global_seqlen/balanced_min:24259 - global_seqlen/balanced_max:24262 - global_seqlen/mean:24260.875 - actor/entropy:0.05833573266863823 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4241411089897156 - training/rollout_probs_diff_mean:0.002529443707317114 - training/rollout_probs_diff_std:0.012784117832779884 - training/rollout_actor_probs_pearson_corr:0.9950049519538879 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:0.9998300075531006 - rollout_corr/rollout_is_ratio_fraction_high:4.4535991037264466e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015587596863042563 - rollout_corr/rollout_is_max:4.229223728179932 - rollout_corr/rollout_is_min:0.31391194462776184 - rollout_corr/rollout_is_std:0.033042069524526596 - rollout_corr/rollout_is_eff_sample_size:0.9989088771294192 - rollout_corr/rollout_is_seq_mean:0.9998038411140442 - rollout_corr/rollout_is_seq_std:0.002055585151538253 - rollout_corr/rollout_is_seq_max:1.0087283849716187 - rollout_corr/rollout_is_seq_min:0.9881833791732788 - rollout_corr/rollout_is_seq_max_deviation:0.011816620826721191 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.058818947058171) - rollout_corr/training_log_ppl:np.float64(0.05630995506544423) - rollout_corr/kl:np.float64(0.0011501821353707697) - rollout_corr/k3_kl:np.float64(0.000877192553543793) - rollout_corr/rollout_ppl:np.float64(1.0575991161167622) - rollout_corr/rollout_log_ppl:np.float64(0.05515977229333657) - rollout_corr/log_ppl_diff:np.float64(0.0011501827721076552) - rollout_corr/log_ppl_abs_diff:np.float64(0.002000718541239621) - rollout_corr/log_ppl_diff_max:np.float64(0.014770753681659698) - rollout_corr/log_ppl_diff_min:np.float64(-0.004706133157014847) - rollout_corr/ppl_ratio:np.float64(1.0011541976127774) - rollout_corr/chi2_token:np.float64(0.0012625399976968765) - rollout_corr/chi2_seq:np.float64(0.3171506472863257) - actor/pg_loss:np.float64(0.0014347152318805456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021798857233079616) - actor/ppo_kl:np.float64(0.0003572766873529698) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(210587.42815934517) - self_distillation/token_reweight_w_std:np.float64(2840550.5963756745) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007291943766177) - self_distillation/token_reweight_w_clip_frac:np.float64(0.016290025698253885) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.28917961940169334) - perf/mfu/actor:np.float64(0.044912664362058916) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.87942504882812) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0033304570242762566 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0033304570242762566 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:350.83984375 - response_length/max:797.0 - response_length/min:144.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:350.83984375 - response_length_non_aborted/max:797.0 - response_length_non_aborted/min:144.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.3125 - prompt_length/max:494.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.729232311248779e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0937697906047106) - timing_s/agent_loop/generate_sequences/max:np.float64(6.558409214019775) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.737826870092249) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.558409214019775) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:797 - timing_s/gen:12.332078708335757 - timing_s/reward:0.06076156720519066 - timing_s/old_log_prob:2.2551023941487074 - timing_s/adv:0.5407032277435064 - timing_s/update_actor:15.401285748928785 - timing_s/step:30.679376512765884 - timing_s/stop_profile:0.00012132897973060608 - timing_per_token_ms/gen:0.13730533550448987 - timing_per_token_ms/adv:0.0027858807016621742 - timing_per_token_ms/update_actor:0.07935248496256207 - perf/total_num_tokens:194087 - perf/time_per_step:30.679376512765884 - perf/throughput:790.7877459603814 (TaskRunner pid=2167046) Training Progress: 43%|████▎ | 43/100 [26:08<33:30, 35.28s/it] (TaskRunner pid=2167046) step:44 - global_seqlen/min:21260 - global_seqlen/max:25228 - global_seqlen/minmax_diff:3968 - global_seqlen/balanced_min:22961 - global_seqlen/balanced_max:22964 - global_seqlen/mean:22962.875 - actor/entropy:0.049587756395339966 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3447294235229492 - training/rollout_probs_diff_mean:0.002002849942073226 - training/rollout_probs_diff_std:0.010673994198441505 - training/rollout_actor_probs_pearson_corr:0.996028482913971 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0000413656234741 - rollout_corr/rollout_is_ratio_fraction_high:2.309708906977903e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.08398108347319e-05 - rollout_corr/rollout_is_max:2.1141040325164795 - rollout_corr/rollout_is_min:0.37150999903678894 - rollout_corr/rollout_is_std:0.028536269441246986 - rollout_corr/rollout_is_eff_sample_size:0.9991863439184062 - rollout_corr/rollout_is_seq_mean:1.0000253915786743 - rollout_corr/rollout_is_seq_std:0.0017137716058641672 - rollout_corr/rollout_is_seq_max:1.0064252614974976 - rollout_corr/rollout_is_seq_min:0.9936391711235046 - rollout_corr/rollout_is_seq_max_deviation:0.006425261497497559 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0492094033397734) - rollout_corr/training_log_ppl:np.float64(0.04770922514126141) - rollout_corr/kl:np.float64(0.0005806892615183301) - rollout_corr/k3_kl:np.float64(0.0006233692477337627) - rollout_corr/rollout_ppl:np.float64(1.0486026802100241) - rollout_corr/rollout_log_ppl:np.float64(0.04712853589262522) - rollout_corr/log_ppl_diff:np.float64(0.0005806892486361903) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016339209532816312) - rollout_corr/log_ppl_diff_max:np.float64(0.011847276240587234) - rollout_corr/log_ppl_diff_min:np.float64(-0.009022675454616547) - rollout_corr/ppl_ratio:np.float64(1.0005835318006575) - rollout_corr/chi2_token:np.float64(0.001389569602906704) - rollout_corr/chi2_seq:np.float64(0.5874903546646237) - actor/pg_loss:np.float64(0.0005399615038186312) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.739950837683864e-05) - actor/ppo_kl:np.float64(0.00017125181667765332) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_w_mean:np.float64(4186.4649539021775) - self_distillation/token_reweight_w_std:np.float64(93057.81301470433) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997784104198217) - self_distillation/token_reweight_w_clip_frac:np.float64(0.005308339008479379) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.11222585383802652) - perf/mfu/actor:np.float64(0.04248327054451827) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.934814453125) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0022635147906839848 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0022635147906839848 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:338.24609375 - response_length/max:619.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:338.24609375 - response_length_non_aborted/max:619.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.34375 - prompt_length/max:524.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014463625848293304 - timing_s/agent_loop/generate_sequences/min:np.float64(1.685251161456108) - timing_s/agent_loop/generate_sequences/max:np.float64(5.734634621068835) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7367765212911763) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.734634621068835) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:588 - timing_s/gen:11.751267571002245 - timing_s/reward:0.05980514734983444 - timing_s/old_log_prob:2.2607326433062553 - timing_s/adv:0.5186716690659523 - timing_s/update_actor:15.34492513537407 - timing_s/step:30.02377664297819 - timing_s/stop_profile:0.00011668913066387177 - timing_per_token_ms/gen:0.13571003419526562 - timing_per_token_ms/adv:0.0028234251431166193 - timing_per_token_ms/update_actor:0.08353116244902951 - perf/total_num_tokens:183703 - perf/time_per_step:30.02377664297819 - perf/throughput:764.8230025508947 (TaskRunner pid=2167046) Training Progress: 44%|████▍ | 44/100 [26:38<31:28, 33.72s/it] (TaskRunner pid=2167046) step:45 - global_seqlen/min:19616 - global_seqlen/max:25545 - global_seqlen/minmax_diff:5929 - global_seqlen/balanced_min:23115 - global_seqlen/balanced_max:23117 - global_seqlen/mean:23116.125 - actor/entropy:0.055425405502319336 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47441259026527405 - training/rollout_probs_diff_mean:0.00222170352935791 - training/rollout_probs_diff_std:0.011589748784899712 - training/rollout_actor_probs_pearson_corr:0.9956302642822266 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.8125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0001627206802368 - rollout_corr/rollout_is_ratio_fraction_high:4.440448174136691e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.991008118959144e-05 - rollout_corr/rollout_is_max:3.4121227264404297 - rollout_corr/rollout_is_min:0.2129592001438141 - rollout_corr/rollout_is_std:0.03031672164797783 - rollout_corr/rollout_is_eff_sample_size:0.9990820973248383 - rollout_corr/rollout_is_seq_mean:1.0001908540725708 - rollout_corr/rollout_is_seq_std:0.0017676244024187326 - rollout_corr/rollout_is_seq_max:1.0077521800994873 - rollout_corr/rollout_is_seq_min:0.995261549949646 - rollout_corr/rollout_is_seq_max_deviation:0.007752180099487305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0531518626958132) - rollout_corr/training_log_ppl:np.float64(0.05147503435409817) - rollout_corr/kl:np.float64(0.0005863242159556137) - rollout_corr/k3_kl:np.float64(0.0007590526232590378) - rollout_corr/rollout_ppl:np.float64(1.0525323520414531) - rollout_corr/rollout_log_ppl:np.float64(0.05088870941017376) - rollout_corr/log_ppl_diff:np.float64(0.0005863249452886521) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017336888786303462) - rollout_corr/log_ppl_diff_max:np.float64(0.009906811639666557) - rollout_corr/log_ppl_diff_min:np.float64(-0.005315380170941353) - rollout_corr/ppl_ratio:np.float64(1.000589034985751) - rollout_corr/chi2_token:np.float64(0.0022609103471040726) - rollout_corr/chi2_seq:np.float64(0.6674370202235878) - actor/pg_loss:np.float64(0.0017804695526137948) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047525035915896297) - actor/ppo_kl:np.float64(0.0002818986581543115) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_w_mean:np.float64(136099.57786145085) - self_distillation/token_reweight_w_std:np.float64(1974545.3167731077) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999589323066175) - self_distillation/token_reweight_w_clip_frac:np.float64(0.022642265394097194) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.23706506937742233) - perf/mfu/actor:np.float64(0.042947523459426026) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.75249862670898) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:1 - critic/score/mean:0.8125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004180959425866604 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004180959425866604 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:351.87890625 - response_length/max:816.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:351.87890625 - response_length_non_aborted/max:816.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:370.5 - prompt_length/max:483.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.853338658809662e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.872569875791669) - timing_s/agent_loop/generate_sequences/max:np.float64(6.59431616961956) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.9382613372727064) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.59431616961956) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:816 - timing_s/gen:12.583950219675899 - timing_s/reward:0.06025470234453678 - timing_s/old_log_prob:2.280903497710824 - timing_s/adv:0.5497738718986511 - timing_s/update_actor:15.477132260799408 - timing_s/step:31.041685946285725 - timing_s/stop_profile:0.0001170281320810318 - timing_per_token_ms/gen:0.13969594275902686 - timing_per_token_ms/adv:0.0029728916065011497 - timing_per_token_ms/update_actor:0.0836922941280135 - perf/total_num_tokens:184929 - perf/time_per_step:31.041685946285725 - perf/throughput:744.6800744005964 (TaskRunner pid=2167046) Training Progress: 45%|████▌ | 45/100 [27:09<30:11, 32.93s/it] (TaskRunner pid=2167046) step:46 - global_seqlen/min:19925 - global_seqlen/max:25269 - global_seqlen/minmax_diff:5344 - global_seqlen/balanced_min:22809 - global_seqlen/balanced_max:22813 - global_seqlen/mean:22811.75 - actor/entropy:0.05739276111125946 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5554836392402649 - training/rollout_probs_diff_mean:0.002197186229750514 - training/rollout_probs_diff_std:0.011728445999324322 - training/rollout_actor_probs_pearson_corr:0.9958913922309875 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0001534223556519 - rollout_corr/rollout_is_ratio_fraction_high:1.1770522178267129e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.239364979090169e-05 - rollout_corr/rollout_is_max:2.795060396194458 - rollout_corr/rollout_is_min:0.011905263178050518 - rollout_corr/rollout_is_std:0.03081785887479782 - rollout_corr/rollout_is_eff_sample_size:0.9990516366740662 - rollout_corr/rollout_is_seq_mean:1.0001554489135742 - rollout_corr/rollout_is_seq_std:0.0020248249638825655 - rollout_corr/rollout_is_seq_max:1.0076408386230469 - rollout_corr/rollout_is_seq_min:0.9923105835914612 - rollout_corr/rollout_is_seq_max_deviation:0.007689416408538818 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0574439051561058) - rollout_corr/training_log_ppl:np.float64(0.054962404239631724) - rollout_corr/kl:np.float64(0.0004922806185305717) - rollout_corr/k3_kl:np.float64(0.0006965439166108922) - rollout_corr/rollout_ppl:np.float64(1.056904122699052) - rollout_corr/rollout_log_ppl:np.float64(0.05447012373679172) - rollout_corr/log_ppl_diff:np.float64(0.0004922805028400035) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017458072843510308) - rollout_corr/log_ppl_diff_max:np.float64(0.030868887901306152) - rollout_corr/log_ppl_diff_min:np.float64(-0.0061208326369524) - rollout_corr/ppl_ratio:np.float64(1.0004963488318026) - rollout_corr/chi2_token:np.float64(0.0016424430068582296) - rollout_corr/chi2_seq:np.float64(0.6610369915142655) - actor/pg_loss:np.float64(0.00031184148974716663) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019037215861317236) - actor/ppo_kl:np.float64(0.00011649080803977085) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_w_mean:np.float64(30034.95145637449) - self_distillation/token_reweight_w_std:np.float64(451229.26952795987) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999818910844624) - self_distillation/token_reweight_w_clip_frac:np.float64(0.00749496826028917) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.0970448236912489) - perf/mfu/actor:np.float64(0.0420408031250471) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.8536114692688) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:1 - critic/score/mean:0.765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005267308442853391 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005267308442853391 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:331.8671875 - response_length/max:592.0 - response_length/min:151.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.8671875 - response_length_non_aborted/max:592.0 - response_length_non_aborted/min:151.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:381.0 - prompt_length/max:481.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.152425289154053e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8468710575252771) - timing_s/agent_loop/generate_sequences/max:np.float64(5.481601318344474) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6845966974287876) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.481601318344474) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:583 - timing_s/gen:11.464920744299889 - timing_s/reward:0.060206545516848564 - timing_s/old_log_prob:2.274786591529846 - timing_s/adv:0.5359881762415171 - timing_s/update_actor:15.595661450177431 - timing_s/step:30.019388634711504 - timing_s/stop_profile:0.00013137422502040863 - timing_per_token_ms/gen:0.13494810075919736 - timing_per_token_ms/adv:0.0029370180731504436 - timing_per_token_ms/update_actor:0.08545848877320586 - perf/total_num_tokens:182494 - perf/time_per_step:30.019388634711504 - perf/throughput:759.900552192549 (TaskRunner pid=2167046) Training Progress: 46%|████▌ | 46/100 [27:39<28:51, 32.07s/it] (TaskRunner pid=2167046) step:47 - global_seqlen/min:20827 - global_seqlen/max:26713 - global_seqlen/minmax_diff:5886 - global_seqlen/balanced_min:24177 - global_seqlen/balanced_max:24180 - global_seqlen/mean:24179.125 - actor/entropy:0.055492036044597626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6224586963653564 - training/rollout_probs_diff_mean:0.0022652670741081238 - training/rollout_probs_diff_std:0.011492198333144188 - training/rollout_actor_probs_pearson_corr:0.9957019686698914 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:0.9999520778656006 - rollout_corr/rollout_is_ratio_fraction_high:2.1471898435265757e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.662354568718001e-05 - rollout_corr/rollout_is_max:2.4000132083892822 - rollout_corr/rollout_is_min:0.3577742278575897 - rollout_corr/rollout_is_std:0.030129367485642433 - rollout_corr/rollout_is_eff_sample_size:0.9990929255817853 - rollout_corr/rollout_is_seq_mean:0.9999911785125732 - rollout_corr/rollout_is_seq_std:0.0017208998324349523 - rollout_corr/rollout_is_seq_max:1.0061891078948975 - rollout_corr/rollout_is_seq_min:0.9937074184417725 - rollout_corr/rollout_is_seq_max_deviation:0.006292581558227539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0576613186858594) - rollout_corr/training_log_ppl:np.float64(0.05537509770147153) - rollout_corr/kl:np.float64(0.0005087664376974832) - rollout_corr/k3_kl:np.float64(0.0005032766369623687) - rollout_corr/rollout_ppl:np.float64(1.057125517167151) - rollout_corr/rollout_log_ppl:np.float64(0.05486633105647343) - rollout_corr/log_ppl_diff:np.float64(0.0005087666449981043) - rollout_corr/log_ppl_abs_diff:np.float64(0.001345907576251193) - rollout_corr/log_ppl_diff_max:np.float64(0.005119312554597855) - rollout_corr/log_ppl_diff_min:np.float64(-0.004925638437271118) - rollout_corr/ppl_ratio:np.float64(1.0005102870054543) - rollout_corr/chi2_token:np.float64(0.0010242487769573927) - rollout_corr/chi2_seq:np.float64(0.25397335505113006) - actor/pg_loss:np.float64(0.0007073208689689636) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021316877064236905) - actor/ppo_kl:np.float64(2.2292524672451464e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(16477.566876600962) - self_distillation/token_reweight_w_std:np.float64(290898.57240845717) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991839011199772) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014518271171255037) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.1594111081212759) - perf/mfu/actor:np.float64(0.04459250728956798) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.8540496826172) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:1 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00286381458863616 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00286381458863616 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:363.84765625 - response_length/max:616.0 - response_length/min:161.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:363.84765625 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:161.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.75 - prompt_length/max:490.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.4756179451942444e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.9349538125097752) - timing_s/agent_loop/generate_sequences/max:np.float64(5.945943394675851) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.1075278270218405) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.945943394675851) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:417 - timing_s/agent_loop/slowest/response_length:616 - timing_s/gen:11.789232794195414 - timing_s/reward:0.0615020003169775 - timing_s/old_log_prob:2.3247509114444256 - timing_s/adv:0.5406120996922255 - timing_s/update_actor:15.458580259233713 - timing_s/step:30.26202197559178 - timing_s/stop_profile:0.00011436082422733307 - timing_per_token_ms/gen:0.12656860587466223 - timing_per_token_ms/adv:0.0027948286987857576 - timing_per_token_ms/update_actor:0.07991697517607499 - perf/total_num_tokens:193433 - perf/time_per_step:30.26202197559178 - perf/throughput:798.9923812593217 (TaskRunner pid=2167046) Training Progress: 47%|████▋ | 47/100 [28:10<27:51, 31.54s/it] (TaskRunner pid=2167046) step:48 - global_seqlen/min:19556 - global_seqlen/max:26843 - global_seqlen/minmax_diff:7287 - global_seqlen/balanced_min:23908 - global_seqlen/balanced_max:23912 - global_seqlen/mean:23910.25 - actor/entropy:0.05349171906709671 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5511896014213562 - training/rollout_probs_diff_mean:0.001996841514483094 - training/rollout_probs_diff_std:0.01053113304078579 - training/rollout_actor_probs_pearson_corr:0.9963048100471497 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.8984375 - self_distillation/correct_sample_fraction:0.8203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8984375 - rollout_corr/rollout_is_mean:1.0001131296157837 - rollout_corr/rollout_is_ratio_fraction_high:3.288536026957445e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.865608444670215e-05 - rollout_corr/rollout_is_max:3.7064740657806396 - rollout_corr/rollout_is_min:0.019543590024113655 - rollout_corr/rollout_is_std:0.02837498113512993 - rollout_corr/rollout_is_eff_sample_size:0.9991958652393542 - rollout_corr/rollout_is_seq_mean:1.0001505613327026 - rollout_corr/rollout_is_seq_std:0.001532649272121489 - rollout_corr/rollout_is_seq_max:1.0046343803405762 - rollout_corr/rollout_is_seq_min:0.9958474040031433 - rollout_corr/rollout_is_seq_max_deviation:0.004634380340576172 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0504649495705962) - rollout_corr/training_log_ppl:np.float64(0.04875240848195972) - rollout_corr/kl:np.float64(0.0003140236587224621) - rollout_corr/k3_kl:np.float64(0.0005160192457793755) - rollout_corr/rollout_ppl:np.float64(1.050125792156905) - rollout_corr/rollout_log_ppl:np.float64(0.0484383839811926) - rollout_corr/log_ppl_diff:np.float64(0.0003140245007671183) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014380264019564493) - rollout_corr/log_ppl_diff_max:np.float64(0.012833669781684875) - rollout_corr/log_ppl_diff_min:np.float64(-0.007530331611633301) - rollout_corr/ppl_ratio:np.float64(1.000316001707688) - rollout_corr/chi2_token:np.float64(0.0014003217220306396) - rollout_corr/chi2_seq:np.float64(0.5536817323882133) - actor/pg_loss:np.float64(0.0007914919406175613) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001905762474052608) - actor/ppo_kl:np.float64(2.5524123401510224e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_w_mean:np.float64(199612.87102979445) - self_distillation/token_reweight_w_std:np.float64(2349294.0318124965) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000342559069395) - self_distillation/token_reweight_w_clip_frac:np.float64(0.013467349508573534) - self_distillation/empty_target_batch:np.float64(0.1015625) - actor/grad_norm:np.float64(0.18085278756916523) - perf/mfu/actor:np.float64(0.04381698070263809) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.83398818969727) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:1 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003589985426515341 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003589985426515341 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:356.3515625 - response_length/max:782.0 - response_length/min:144.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:356.3515625 - response_length_non_aborted/max:782.0 - response_length_non_aborted/min:144.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.84375 - prompt_length/max:488.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.644776582717896e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.739744871854782) - timing_s/agent_loop/generate_sequences/max:np.float64(6.505071012303233) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8486769011433353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.505071012303233) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:782 - timing_s/gen:12.31418189033866 - timing_s/reward:0.0610735472291708 - timing_s/old_log_prob:2.2674227822571993 - timing_s/adv:0.5372972805052996 - timing_s/update_actor:15.405583010986447 - timing_s/step:30.671290330588818 - timing_s/stop_profile:0.00011252425611019135 - timing_per_token_ms/gen:0.13498544154450112 - timing_per_token_ms/adv:0.002808927554632948 - timing_per_token_ms/update_actor:0.08053859229298338 - perf/total_num_tokens:191282 - perf/time_per_step:30.671290330588818 - perf/throughput:779.5645289873587 (TaskRunner pid=2167046) Training Progress: 48%|████▊ | 48/100 [28:41<27:07, 31.29s/it] (TaskRunner pid=2167046) step:49 - global_seqlen/min:20968 - global_seqlen/max:26342 - global_seqlen/minmax_diff:5374 - global_seqlen/balanced_min:23295 - global_seqlen/balanced_max:23298 - global_seqlen/mean:23296.875 - actor/entropy:0.05485960841178894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9957264065742493 - training/rollout_probs_diff_mean:0.002162784105166793 - training/rollout_probs_diff_std:0.012243074364960194 - training/rollout_actor_probs_pearson_corr:0.9952481389045715 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.000170350074768 - rollout_corr/rollout_is_ratio_fraction_high:4.6703327825525776e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.340665565105155e-05 - rollout_corr/rollout_is_max:3.282700300216675 - rollout_corr/rollout_is_min:0.004070968832820654 - rollout_corr/rollout_is_std:0.03059268184006214 - rollout_corr/rollout_is_eff_sample_size:0.9990654389263088 - rollout_corr/rollout_is_seq_mean:1.0001893043518066 - rollout_corr/rollout_is_seq_std:0.0018957956926897168 - rollout_corr/rollout_is_seq_max:1.0093369483947754 - rollout_corr/rollout_is_seq_min:0.9951988458633423 - rollout_corr/rollout_is_seq_max_deviation:0.00933694839477539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.055814309976995) - rollout_corr/training_log_ppl:np.float64(0.05374641695198079) - rollout_corr/kl:np.float64(0.0004828907792662207) - rollout_corr/k3_kl:np.float64(0.0005371262649731534) - rollout_corr/rollout_ppl:np.float64(1.0553037645295262) - rollout_corr/rollout_log_ppl:np.float64(0.05326352582505933) - rollout_corr/log_ppl_diff:np.float64(0.00048289112692145864) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015306145369322621) - rollout_corr/log_ppl_diff_max:np.float64(0.007901884615421295) - rollout_corr/log_ppl_diff_min:np.float64(-0.007005564868450165) - rollout_corr/ppl_ratio:np.float64(1.0004850262776017) - rollout_corr/chi2_token:np.float64(0.0012454038951545954) - rollout_corr/chi2_seq:np.float64(0.35367413074709475) - actor/pg_loss:np.float64(0.0008061726111918688) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020583227842507767) - actor/ppo_kl:np.float64(0.00010553301872917586) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(163828.87535567163) - self_distillation/token_reweight_w_std:np.float64(2273528.9927650653) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008936680387706) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014962271772674285) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.18702556192874908) - perf/mfu/actor:np.float64(0.04264424569583532) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.88611793518066) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0024738169740885496 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0024738169740885496 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.55859375 - response_length/max:767.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.55859375 - response_length_non_aborted/max:767.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:393.46875 - prompt_length/max:486.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.763054549694061e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5356009639799595) - timing_s/agent_loop/generate_sequences/max:np.float64(6.282565135508776) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.672979645591113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.282565135508776) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:767 - timing_s/gen:12.247308611869812 - timing_s/reward:0.06086792051792145 - timing_s/old_log_prob:2.283744128420949 - timing_s/adv:0.5211576409637928 - timing_s/update_actor:15.659554366022348 - timing_s/step:30.860737219452858 - timing_s/stop_profile:0.00010721012949943542 - timing_per_token_ms/gen:0.14299752019183173 - timing_per_token_ms/adv:0.002796285129249056 - timing_per_token_ms/update_actor:0.08402175380830233 - perf/total_num_tokens:186375 - perf/time_per_step:30.860737219452858 - perf/throughput:754.9033853058758 (TaskRunner pid=2167046) Training Progress: 49%|████▉ | 49/100 [29:11<26:30, 31.18s/it] (TaskRunner pid=2167046) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2167046) validation generation end (TaskRunner pid=2167046) [prompt] system (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume $ V $ of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c \cdot \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given that all angles $ \alpha, \beta, \gamma $ are 90.00º, the cosine of each angle is 0. Substituting these values into the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c \cdot \sqrt{1 + 2(0)(0)(0) - (0)^2 - (0)^2 - (0)^2} = a \cdot b \cdot c \cdot \sqrt{1} = a \cdot b \cdot c (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Now, substituting the given lattice parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = 7.82 \, \text{Å} \times 7.82 \, \text{Å} \times 17.88 \, \text{Å} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then multiply by 17.88: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 61.1524 \times 17.88 \approx 1094.31 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2167440) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2167046) step:50 - global_seqlen/min:18307 - global_seqlen/max:25358 - global_seqlen/minmax_diff:7051 - global_seqlen/balanced_min:22906 - global_seqlen/balanced_max:22909 - global_seqlen/mean:22907.375 - actor/entropy:0.0527101494371891 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.530918538570404 - training/rollout_probs_diff_mean:0.0021026728209108114 - training/rollout_probs_diff_std:0.01144232228398323 - training/rollout_actor_probs_pearson_corr:0.9958080053329468 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.999865710735321 - rollout_corr/rollout_is_ratio_fraction_high:2.4842560378601775e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012421280553098768 - rollout_corr/rollout_is_max:2.086440086364746 - rollout_corr/rollout_is_min:0.33655375242233276 - rollout_corr/rollout_is_std:0.029702994972467422 - rollout_corr/rollout_is_eff_sample_size:0.9991182717602443 - rollout_corr/rollout_is_seq_mean:0.9998937845230103 - rollout_corr/rollout_is_seq_std:0.0017050968017429113 - rollout_corr/rollout_is_seq_max:1.0061534643173218 - rollout_corr/rollout_is_seq_min:0.9932185411453247 - rollout_corr/rollout_is_seq_max_deviation:0.006781458854675293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0527593255974352) - rollout_corr/training_log_ppl:np.float64(0.05062973701751616) - rollout_corr/kl:np.float64(0.0005176823531662933) - rollout_corr/k3_kl:np.float64(0.0004894320357937687) - rollout_corr/rollout_ppl:np.float64(1.0521913496777415) - rollout_corr/rollout_log_ppl:np.float64(0.05011205441496713) - rollout_corr/log_ppl_diff:np.float64(0.00051768260254903) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014349543425851152) - rollout_corr/log_ppl_diff_max:np.float64(0.008032098412513733) - rollout_corr/log_ppl_diff_min:np.float64(-0.004613518714904785) - rollout_corr/ppl_ratio:np.float64(1.0005198165308684) - rollout_corr/chi2_token:np.float64(0.0008812225423753262) - rollout_corr/chi2_seq:np.float64(0.2764910035766661) - actor/pg_loss:np.float64(0.0006625362439081073) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022346714558807435) - actor/ppo_kl:np.float64(-5.137884922845615e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(111777.4753512349) - self_distillation/token_reweight_w_std:np.float64(1134010.0336442993) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999779810430482) - self_distillation/token_reweight_w_clip_frac:np.float64(0.008872004498698516) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.18878151662647724) - perf/mfu/actor:np.float64(0.04136322343812593) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.93096733093262) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7473404255319149) - val-aux/sciknoweval/reward/std@16:np.float64(0.05458479149126176) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7693617021276595) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.04424280678647665) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7248297872340426) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.048188095845902945) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7469255319148936) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.054892738931616274) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7870531914893617) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.03246858700919304) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7064468085106383) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.037573798242433266) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7496276595744681) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.043796060904346165) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8002127659574468) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.01884323781268916) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.6913191489361703) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.027674210919803686) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7489893617021277) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.030003411025740924) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8069255319148936) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.006948346270106342) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.6796914893617022) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.018065512761743046) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7481808510638299) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.01954575745336847) - val-aux/sciknoweval/score/mean@16:np.float64(0.7473404255319149) - val-aux/sciknoweval/score/std@16:np.float64(0.05458479149126176) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7693617021276595) - val-aux/sciknoweval/score/best@2/std:np.float64(0.04424280678647665) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7248297872340426) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.048188095845902945) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7469255319148936) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.054892738931616274) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7870531914893617) - val-aux/sciknoweval/score/best@4/std:np.float64(0.03246858700919304) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7064468085106383) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.037573798242433266) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7496276595744681) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.043796060904346165) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8002127659574468) - val-aux/sciknoweval/score/best@8/std:np.float64(0.01884323781268916) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.6913191489361703) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.027674210919803686) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7489893617021277) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.030003411025740924) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8069255319148936) - val-aux/sciknoweval/score/best@16/std:np.float64(0.006948346270106342) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.6796914893617022) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.018065512761743046) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7481808510638299) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.01954575745336847) - val-core/sciknoweval/acc/mean@16:np.float64(0.7473404255319149) - val-aux/sciknoweval/acc/std@16:np.float64(0.05458479149126176) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7693617021276595) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.04424280678647665) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7248297872340426) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.048188095845902945) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7469255319148936) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.054892738931616274) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7870531914893617) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.03246858700919304) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7064468085106383) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.037573798242433266) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7496276595744681) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.043796060904346165) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8002127659574468) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.01884323781268916) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.6913191489361703) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.027674210919803686) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7489893617021277) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.030003411025740924) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8069255319148936) - val-core/sciknoweval/acc/best@16/std:np.float64(0.006948346270106342) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.6796914893617022) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.018065512761743046) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7481808510638299) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.01954575745336847) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:1 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0014983168803155422 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0014983168803155422 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:314.48046875 - response_length/max:557.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.48046875 - response_length_non_aborted/max:557.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.375 - prompt_length/max:492.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000141998752951622 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6657594684511423) - timing_s/agent_loop/generate_sequences/max:np.float64(5.1641334146261215) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.496013310876151) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.1641334146261215) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:458 - timing_s/agent_loop/slowest/response_length:557 - timing_s/gen:10.932165456935763 - timing_s/reward:0.05853456258773804 - timing_s/old_log_prob:2.3450980577617884 - timing_s/adv:0.5179697405546904 - timing_s/update_actor:15.95227106846869 - timing_s/step:29.89210656285286 - timing_s/testing:37.86319975182414 - timing_s/save_checkpoint:6.8816672675311565 - timing_s/stop_profile:0.00012503191828727722 - timing_per_token_ms/gen:0.1357914896460651 - timing_per_token_ms/adv:0.0028264354850495222 - timing_per_token_ms/update_actor:0.08704768152433817 - perf/total_num_tokens:183259 - perf/time_per_step:29.89210656285286 - perf/throughput:766.3352514762931 (TaskRunner pid=2167046) Training Progress: 50%|█████ | 50/100 [30:26<36:51, 44.24s/it] (TaskRunner pid=2167046) step:51 - global_seqlen/min:20598 - global_seqlen/max:24762 - global_seqlen/minmax_diff:4164 - global_seqlen/balanced_min:22903 - global_seqlen/balanced_max:22909 - global_seqlen/mean:22907.125 - actor/entropy:0.05307382717728615 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.681950569152832 - training/rollout_probs_diff_mean:0.002189614810049534 - training/rollout_probs_diff_std:0.011771688237786293 - training/rollout_actor_probs_pearson_corr:0.9954410791397095 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.0001434087753296 - rollout_corr/rollout_is_ratio_fraction_high:4.8094843805301934e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.21422620699741e-05 - rollout_corr/rollout_is_max:3.7063121795654297 - rollout_corr/rollout_is_min:0.1226673275232315 - rollout_corr/rollout_is_std:0.030606316402554512 - rollout_corr/rollout_is_eff_sample_size:0.9990644870346002 - rollout_corr/rollout_is_seq_mean:1.0002003908157349 - rollout_corr/rollout_is_seq_std:0.002001448068767786 - rollout_corr/rollout_is_seq_max:1.0162816047668457 - rollout_corr/rollout_is_seq_min:0.9955185055732727 - rollout_corr/rollout_is_seq_max_deviation:0.016281604766845703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0550247388891876) - rollout_corr/training_log_ppl:np.float64(0.05237314046280517) - rollout_corr/kl:np.float64(0.0005226194430250075) - rollout_corr/k3_kl:np.float64(0.0006975394764396903) - rollout_corr/rollout_ppl:np.float64(1.0544638223946095) - rollout_corr/rollout_log_ppl:np.float64(0.051850520834705094) - rollout_corr/log_ppl_diff:np.float64(0.0005226196281000739) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016185973963729339) - rollout_corr/log_ppl_diff_max:np.float64(0.010919332504272461) - rollout_corr/log_ppl_diff_min:np.float64(-0.005789030343294144) - rollout_corr/ppl_ratio:np.float64(1.0005250833928585) - rollout_corr/chi2_token:np.float64(0.0018498406279832125) - rollout_corr/chi2_seq:np.float64(0.5595120217185467) - actor/pg_loss:np.float64(0.0013827746734023094) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044605470702663297) - actor/ppo_kl:np.float64(0.00020610883375127287) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.671875) - self_distillation/token_reweight_w_mean:np.float64(184521.5831220611) - self_distillation/token_reweight_w_std:np.float64(2546310.6744150682) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008375360630453) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01984426895796787) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.24625525996088982) - perf/mfu/actor:np.float64(0.0423324768937178) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.52999877929688) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0062553356401622295 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0062553356401622295 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:324.87890625 - response_length/max:634.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.87890625 - response_length_non_aborted/max:634.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.96875 - prompt_length/max:519.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.856024265289307e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.739304807037115) - timing_s/agent_loop/generate_sequences/max:np.float64(5.466823788359761) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.590949060060666) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.466823788359761) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:400 - timing_s/agent_loop/slowest/response_length:607 - timing_s/gen:11.270084258168936 - timing_s/reward:0.05573139898478985 - timing_s/old_log_prob:2.379555854946375 - timing_s/adv:0.5390975456684828 - timing_s/update_actor:15.496772347018123 - timing_s/step:29.842626152560115 - timing_s/stop_profile:0.00011540204286575317 - timing_per_token_ms/gen:0.1355082333341622 - timing_per_token_ms/adv:0.0029417569078860986 - timing_per_token_ms/update_actor:0.0845630581479459 - perf/total_num_tokens:183257 - perf/time_per_step:29.842626152560115 - perf/throughput:767.5974923552384 (TaskRunner pid=2167046) Training Progress: 51%|█████ | 51/100 [30:56<32:36, 39.93s/it] (TaskRunner pid=2167046) step:52 - global_seqlen/min:19845 - global_seqlen/max:26627 - global_seqlen/minmax_diff:6782 - global_seqlen/balanced_min:23480 - global_seqlen/balanced_max:23481 - global_seqlen/mean:23480.75 - actor/entropy:0.04984758421778679 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7995882630348206 - training/rollout_probs_diff_mean:0.00212426227517426 - training/rollout_probs_diff_std:0.01242454070597887 - training/rollout_actor_probs_pearson_corr:0.9947465062141418 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999185800552368 - rollout_corr/rollout_is_ratio_fraction_high:5.9063953813165426e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014175349497236311 - rollout_corr/rollout_is_max:5.277081489562988 - rollout_corr/rollout_is_min:0.02199946902692318 - rollout_corr/rollout_is_std:0.030726822093129158 - rollout_corr/rollout_is_eff_sample_size:0.9990565150127493 - rollout_corr/rollout_is_seq_mean:0.999808132648468 - rollout_corr/rollout_is_seq_std:0.0021094551775604486 - rollout_corr/rollout_is_seq_max:1.0133817195892334 - rollout_corr/rollout_is_seq_min:0.9917005896568298 - rollout_corr/rollout_is_seq_max_deviation:0.013381719589233398 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0494226575829089) - rollout_corr/training_log_ppl:np.float64(0.047722964010972646) - rollout_corr/kl:np.float64(0.0010447812739915463) - rollout_corr/k3_kl:np.float64(0.0007033883213365399) - rollout_corr/rollout_ppl:np.float64(1.0483222524635494) - rollout_corr/rollout_log_ppl:np.float64(0.04667818260804779) - rollout_corr/log_ppl_diff:np.float64(0.0010447814029248548) - rollout_corr/log_ppl_abs_diff:np.float64(0.001777110573129903) - rollout_corr/log_ppl_diff_max:np.float64(0.011939480900764465) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066554956138134) - rollout_corr/ppl_ratio:np.float64(1.0010481965728104) - rollout_corr/chi2_token:np.float64(0.0008801114745438099) - rollout_corr/chi2_seq:np.float64(0.4005721795838326) - actor/pg_loss:np.float64(0.0008483310230076313) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005638624024868477) - actor/ppo_kl:np.float64(0.00027875212417960427) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_w_mean:np.float64(222058.34980464377) - self_distillation/token_reweight_w_std:np.float64(2551187.82467389) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001434024889022) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01402115351811517) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.30175722017884254) - perf/mfu/actor:np.float64(0.04329011667396355) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.3583869934082) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:1 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027361377142369747 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027361377142369747 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:330.6796875 - response_length/max:598.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:330.6796875 - response_length_non_aborted/max:598.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:403.09375 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001457519829273224 - timing_s/agent_loop/generate_sequences/min:np.float64(1.724724542349577) - timing_s/agent_loop/generate_sequences/max:np.float64(5.4192838948220015) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6701025829606806) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.4192838948220015) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:583 - timing_s/gen:11.494549315422773 - timing_s/reward:0.05863015167415142 - timing_s/old_log_prob:2.3124916180968285 - timing_s/adv:0.5321205835789442 - timing_s/update_actor:15.649614898487926 - timing_s/step:30.1305893342942 - timing_s/stop_profile:0.00013644807040691376 - timing_per_token_ms/gen:0.13578270743760215 - timing_per_token_ms/adv:0.002832749079453085 - timing_per_token_ms/update_actor:0.08331087645458474 - perf/total_num_tokens:187846 - perf/time_per_step:30.1305893342942 - perf/throughput:779.2993936987004 (TaskRunner pid=2167046) Training Progress: 52%|█████▏ | 52/100 [31:26<29:36, 37.01s/it] (TaskRunner pid=2167046) step:53 - global_seqlen/min:20647 - global_seqlen/max:23970 - global_seqlen/minmax_diff:3323 - global_seqlen/balanced_min:22483 - global_seqlen/balanced_max:22486 - global_seqlen/mean:22485.375 - actor/entropy:0.04552474617958069 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975857734680176 - training/rollout_probs_diff_mean:0.0019856856670230627 - training/rollout_probs_diff_std:0.011346102692186832 - training/rollout_actor_probs_pearson_corr:0.995132327079773 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:0.9999587535858154 - rollout_corr/rollout_is_ratio_fraction_high:5.100284397485666e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010200568794971332 - rollout_corr/rollout_is_max:2.554445505142212 - rollout_corr/rollout_is_min:0.3733019232749939 - rollout_corr/rollout_is_std:0.029692960903048515 - rollout_corr/rollout_is_eff_sample_size:0.9991189857557254 - rollout_corr/rollout_is_seq_mean:1.0000665187835693 - rollout_corr/rollout_is_seq_std:0.0020678448490798473 - rollout_corr/rollout_is_seq_max:1.0102448463439941 - rollout_corr/rollout_is_seq_min:0.9950273036956787 - rollout_corr/rollout_is_seq_max_deviation:0.01024484634399414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0451452140696347) - rollout_corr/training_log_ppl:np.float64(0.04381734907747159) - rollout_corr/kl:np.float64(0.00043769131724502586) - rollout_corr/k3_kl:np.float64(0.0005817715615477614) - rollout_corr/rollout_ppl:np.float64(1.0446803770028055) - rollout_corr/rollout_log_ppl:np.float64(0.04337965739887295) - rollout_corr/log_ppl_diff:np.float64(0.00043769167859863956) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015039140798762674) - rollout_corr/log_ppl_diff_max:np.float64(0.006083883345127106) - rollout_corr/log_ppl_diff_min:np.float64(-0.006761221215128899) - rollout_corr/ppl_ratio:np.float64(1.0004397267475724) - rollout_corr/chi2_token:np.float64(0.0014745418448001146) - rollout_corr/chi2_seq:np.float64(0.42398698115721345) - actor/pg_loss:np.float64(0.0009315765928477049) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026180762870353647) - actor/ppo_kl:np.float64(4.117221907051771e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(49304.76177130593) - self_distillation/token_reweight_w_std:np.float64(677681.4286856137) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997115456499159) - self_distillation/token_reweight_w_clip_frac:np.float64(0.013133775428286754) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.19336532801389694) - perf/mfu/actor:np.float64(0.041706615520811945) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.17058563232422) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001973172416910529 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001973172416910529 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:306.35546875 - response_length/max:604.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:306.35546875 - response_length_non_aborted/max:604.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:502.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000256979838013649 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6869939118623734) - timing_s/agent_loop/generate_sequences/max:np.float64(5.25627869181335) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4931406362447888) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.25627869181335) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:321 - timing_s/agent_loop/slowest/response_length:578 - timing_s/gen:11.471845081076026 - timing_s/reward:0.05468622408807278 - timing_s/old_log_prob:2.644116973504424 - timing_s/adv:0.5500172451138496 - timing_s/update_actor:15.525353970006108 - timing_s/step:30.360052190721035 - timing_s/stop_profile:0.00016107968986034393 - timing_per_token_ms/gen:0.1462741795692303 - timing_per_token_ms/adv:0.003057638826981147 - timing_per_token_ms/update_actor:0.08630806674341715 - perf/total_num_tokens:179883 - perf/time_per_step:30.360052190721035 - perf/throughput:740.6237268219263 (TaskRunner pid=2167046) Training Progress: 53%|█████▎ | 53/100 [31:57<27:32, 35.16s/it] (TaskRunner pid=2167046) step:54 - global_seqlen/min:20588 - global_seqlen/max:26029 - global_seqlen/minmax_diff:5441 - global_seqlen/balanced_min:23719 - global_seqlen/balanced_max:23721 - global_seqlen/mean:23720.0 - actor/entropy:0.04974914714694023 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5220087766647339 - training/rollout_probs_diff_mean:0.002054911106824875 - training/rollout_probs_diff_std:0.011294582858681679 - training/rollout_actor_probs_pearson_corr:0.9953687191009521 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.625 - self_distillation/correct_sample_fraction:0.5703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.625 - rollout_corr/rollout_is_mean:1.0000876188278198 - rollout_corr/rollout_is_ratio_fraction_high:1.1769137017836329e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.061482028802857e-05 - rollout_corr/rollout_is_max:2.3180410861968994 - rollout_corr/rollout_is_min:0.4453998804092407 - rollout_corr/rollout_is_std:0.028263429179787636 - rollout_corr/rollout_is_eff_sample_size:0.9992019351761722 - rollout_corr/rollout_is_seq_mean:1.0000823736190796 - rollout_corr/rollout_is_seq_std:0.0018613808788359165 - rollout_corr/rollout_is_seq_max:1.0083881616592407 - rollout_corr/rollout_is_seq_min:0.9941661357879639 - rollout_corr/rollout_is_seq_max_deviation:0.008388161659240723 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0487789153121412) - rollout_corr/training_log_ppl:np.float64(0.04697248301545187) - rollout_corr/kl:np.float64(0.0006158747329720882) - rollout_corr/k3_kl:np.float64(0.0005426218764306157) - rollout_corr/rollout_ppl:np.float64(1.0481268735602498) - rollout_corr/rollout_log_ppl:np.float64(0.04635660783787898) - rollout_corr/log_ppl_diff:np.float64(0.0006158751775728888) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014905650741638965) - rollout_corr/log_ppl_diff_max:np.float64(0.015574745833873749) - rollout_corr/log_ppl_diff_min:np.float64(-0.006432611495256424) - rollout_corr/ppl_ratio:np.float64(1.0006186675745994) - rollout_corr/chi2_token:np.float64(0.0008958489634096622) - rollout_corr/chi2_seq:np.float64(0.19869494764134288) - actor/pg_loss:np.float64(0.00033523840829730034) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021035506097177858) - actor/ppo_kl:np.float64(0.00026634746228637596) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5703125) - self_distillation/token_reweight_w_mean:np.float64(87162.10803046962) - self_distillation/token_reweight_w_std:np.float64(1211193.186900845) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000149110564962) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011021143785910681) - self_distillation/empty_target_batch:np.float64(0.375) - actor/grad_norm:np.float64(0.19004646688699722) - perf/mfu/actor:np.float64(0.04410223401166184) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.18850708007812) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0010548088466748595 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0010548088466748595 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:331.90625 - response_length/max:750.0 - response_length/min:150.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.90625 - response_length_non_aborted/max:750.0 - response_length_non_aborted/min:150.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:409.34375 - prompt_length/max:524.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012475810945034027 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7376059517264366) - timing_s/agent_loop/generate_sequences/max:np.float64(6.052014265209436) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.617489763564663) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.052014265209436) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:750 - timing_s/gen:12.283002220094204 - timing_s/reward:0.06129976734519005 - timing_s/old_log_prob:2.295364612713456 - timing_s/adv:0.5475999526679516 - timing_s/update_actor:15.403776170685887 - timing_s/step:30.660206539556384 - timing_s/stop_profile:0.00013978779315948486 - timing_per_token_ms/gen:0.1445603311846131 - timing_per_token_ms/adv:0.0028857501721540452 - timing_per_token_ms/update_actor:0.08117504305799898 - perf/total_num_tokens:189760 - perf/time_per_step:30.660206539556384 - perf/throughput:773.6412332838512 (TaskRunner pid=2167046) Training Progress: 54%|█████▍ | 54/100 [32:28<25:55, 33.83s/it] (TaskRunner pid=2167046) step:55 - global_seqlen/min:21032 - global_seqlen/max:25317 - global_seqlen/minmax_diff:4285 - global_seqlen/balanced_min:23375 - global_seqlen/balanced_max:23376 - global_seqlen/mean:23375.25 - actor/entropy:0.0445924773812294 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3997594118118286 - training/rollout_probs_diff_mean:0.0019081803038716316 - training/rollout_probs_diff_std:0.010883488692343235 - training/rollout_actor_probs_pearson_corr:0.9953238368034363 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.73828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:0.9999055862426758 - rollout_corr/rollout_is_ratio_fraction_high:2.459117240505293e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011066027218475938 - rollout_corr/rollout_is_max:2.090315341949463 - rollout_corr/rollout_is_min:0.23710303008556366 - rollout_corr/rollout_is_std:0.027302302420139313 - rollout_corr/rollout_is_eff_sample_size:0.9992550205113556 - rollout_corr/rollout_is_seq_mean:0.9998701810836792 - rollout_corr/rollout_is_seq_std:0.001544875674881041 - rollout_corr/rollout_is_seq_max:1.0056719779968262 - rollout_corr/rollout_is_seq_min:0.9948750138282776 - rollout_corr/rollout_is_seq_max_deviation:0.005671977996826172 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.045014244504273) - rollout_corr/training_log_ppl:np.float64(0.043634509600451565) - rollout_corr/kl:np.float64(0.0006418770270268936) - rollout_corr/k3_kl:np.float64(0.0005075502466063142) - rollout_corr/rollout_ppl:np.float64(1.0443436088971794) - rollout_corr/rollout_log_ppl:np.float64(0.04299263210441495) - rollout_corr/log_ppl_diff:np.float64(0.0006418774960366136) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015478016607630707) - rollout_corr/log_ppl_diff_max:np.float64(0.008348822593688965) - rollout_corr/log_ppl_diff_min:np.float64(-0.004879988729953766) - rollout_corr/ppl_ratio:np.float64(1.000643964856863) - rollout_corr/chi2_token:np.float64(0.0007614318747073412) - rollout_corr/chi2_seq:np.float64(0.5452475808560848) - actor/pg_loss:np.float64(0.00022943562362343073) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.411085774336243e-05) - actor/ppo_kl:np.float64(0.00010750114318014425) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_w_mean:np.float64(19822.99445243622) - self_distillation/token_reweight_w_std:np.float64(292975.02256671095) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998247078619897) - self_distillation/token_reweight_w_clip_frac:np.float64(0.005360060549719492) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.09271686151623726) - perf/mfu/actor:np.float64(0.043276245005982816) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.02014541625977) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001120435306802392 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.001120435306802392 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:317.6953125 - response_length/max:613.0 - response_length/min:149.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:317.6953125 - response_length_non_aborted/max:613.0 - response_length_non_aborted/min:149.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.78125 - prompt_length/max:519.0 - prompt_length/min:271.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011121667921543121 - timing_s/agent_loop/generate_sequences/min:np.float64(1.770000969991088) - timing_s/agent_loop/generate_sequences/max:np.float64(5.573975896462798) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.556777519559546) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.573975896462798) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:271 - timing_s/agent_loop/slowest/response_length:606 - timing_s/gen:11.38779872842133 - timing_s/reward:0.054506201297044754 - timing_s/old_log_prob:2.281204592436552 - timing_s/adv:0.5317836664617062 - timing_s/update_actor:15.561219872906804 - timing_s/step:29.907985949888825 - timing_s/stop_profile:0.00011770427227020264 - timing_per_token_ms/gen:0.14001965730261073 - timing_per_token_ms/adv:0.0028437325080036906 - timing_per_token_ms/update_actor:0.08321418954292897 - perf/total_num_tokens:187002 - perf/time_per_step:29.907985949888825 - perf/throughput:781.5721874139402 (TaskRunner pid=2167046) Training Progress: 55%|█████▌ | 55/100 [32:58<24:29, 32.67s/it] (TaskRunner pid=2167046) step:56 - global_seqlen/min:19749 - global_seqlen/max:25955 - global_seqlen/minmax_diff:6206 - global_seqlen/balanced_min:23414 - global_seqlen/balanced_max:23416 - global_seqlen/mean:23415.5 - actor/entropy:0.0621294341981411 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5481129288673401 - training/rollout_probs_diff_mean:0.0021422128193080425 - training/rollout_probs_diff_std:0.011219512671232224 - training/rollout_actor_probs_pearson_corr:0.9964797496795654 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999027848243713 - rollout_corr/rollout_is_ratio_fraction_high:1.1152250408485997e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.921800326788798e-05 - rollout_corr/rollout_is_max:2.068037509918213 - rollout_corr/rollout_is_min:0.025016378611326218 - rollout_corr/rollout_is_std:0.02902502380311489 - rollout_corr/rollout_is_eff_sample_size:0.9991581380701209 - rollout_corr/rollout_is_seq_mean:0.9998484253883362 - rollout_corr/rollout_is_seq_std:0.0016506003448739648 - rollout_corr/rollout_is_seq_max:1.0049660205841064 - rollout_corr/rollout_is_seq_min:0.9948515295982361 - rollout_corr/rollout_is_seq_max_deviation:0.005148470401763916 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0614002938382328) - rollout_corr/training_log_ppl:np.float64(0.058645859207899775) - rollout_corr/kl:np.float64(0.0006652666341668834) - rollout_corr/k3_kl:np.float64(0.0005047853511887801) - rollout_corr/rollout_ppl:np.float64(1.060690845362842) - rollout_corr/rollout_log_ppl:np.float64(0.057980592235253425) - rollout_corr/log_ppl_diff:np.float64(0.00066526697264635) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014072957146709086) - rollout_corr/log_ppl_diff_max:np.float64(0.01607600599527359) - rollout_corr/log_ppl_diff_min:np.float64(-0.003986669704318047) - rollout_corr/ppl_ratio:np.float64(1.0006674323230982) - rollout_corr/chi2_token:np.float64(0.000626653665676713) - rollout_corr/chi2_seq:np.float64(0.17073907726444304) - actor/pg_loss:np.float64(0.000534118851646781) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000130811873532366) - actor/ppo_kl:np.float64(5.016306554850303e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(35010.49396638153) - self_distillation/token_reweight_w_std:np.float64(558753.1349300193) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996146762277931) - self_distillation/token_reweight_w_clip_frac:np.float64(0.009658753042458557) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.12083789333701134) - perf/mfu/actor:np.float64(0.043045165586583715) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.06346893310547) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-2.9274657208588906e-05 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-2.9274657208588906e-05 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:350.265625 - response_length/max:764.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:350.265625 - response_length_non_aborted/max:764.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:381.46875 - prompt_length/max:480.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013207830488681793 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5797028467059135) - timing_s/agent_loop/generate_sequences/max:np.float64(6.412713143974543) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6741961784573505) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.412713143974543) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:751 - timing_s/gen:12.2301533408463 - timing_s/reward:0.06069272197782993 - timing_s/old_log_prob:2.2678428273648024 - timing_s/adv:0.5279212314635515 - timing_s/update_actor:15.580040998756886 - timing_s/step:30.75347794033587 - timing_s/stop_profile:2.9560178518295288e-05 - timing_per_token_ms/gen:0.13639373400595864 - timing_per_token_ms/adv:0.0028182252752639893 - timing_per_token_ms/update_actor:0.08317162242295106 - perf/total_num_tokens:187324 - perf/time_per_step:30.75347794033587 - perf/throughput:761.3935583294964 (TaskRunner pid=2167046) Training Progress: 56%|█████▌ | 56/100 [33:29<23:32, 32.10s/it] (TaskRunner pid=2167046) step:57 - global_seqlen/min:20930 - global_seqlen/max:25923 - global_seqlen/minmax_diff:4993 - global_seqlen/balanced_min:23392 - global_seqlen/balanced_max:23395 - global_seqlen/mean:23393.875 - actor/entropy:0.054561372846364975 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967532455921173 - training/rollout_probs_diff_mean:0.002252341015264392 - training/rollout_probs_diff_std:0.011561992578208447 - training/rollout_actor_probs_pearson_corr:0.9957065582275391 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999447464942932 - rollout_corr/rollout_is_ratio_fraction_high:3.6330609873402864e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.266121974680573e-05 - rollout_corr/rollout_is_max:2.0692319869995117 - rollout_corr/rollout_is_min:0.24590204656124115 - rollout_corr/rollout_is_std:0.029926901683211327 - rollout_corr/rollout_is_eff_sample_size:0.9991050630279074 - rollout_corr/rollout_is_seq_mean:0.9999193549156189 - rollout_corr/rollout_is_seq_std:0.0018134000711143017 - rollout_corr/rollout_is_seq_max:1.0067952871322632 - rollout_corr/rollout_is_seq_min:0.9954656362533569 - rollout_corr/rollout_is_seq_max_deviation:0.006795287132263184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0550112552009523) - rollout_corr/training_log_ppl:np.float64(0.05320324154399714) - rollout_corr/kl:np.float64(0.0007456150652345173) - rollout_corr/k3_kl:np.float64(0.0006398622932053399) - rollout_corr/rollout_ppl:np.float64(1.0542202196083963) - rollout_corr/rollout_log_ppl:np.float64(0.052457625864917645) - rollout_corr/log_ppl_diff:np.float64(0.0007456156790794921) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015578475331494701) - rollout_corr/log_ppl_diff_max:np.float64(0.008377820253372192) - rollout_corr/log_ppl_diff_min:np.float64(-0.003737565129995346) - rollout_corr/ppl_ratio:np.float64(1.0007477451581508) - rollout_corr/chi2_token:np.float64(0.0011580914724618196) - rollout_corr/chi2_seq:np.float64(0.10339058190584183) - actor/pg_loss:np.float64(0.0008008657023310661) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002839279932231875) - actor/ppo_kl:np.float64(0.00018597565893951895) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_w_mean:np.float64(134570.97113014408) - self_distillation/token_reweight_w_std:np.float64(1767222.71986674) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998309221118689) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01680245746683795) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.23896951600909233) - perf/mfu/actor:np.float64(0.04274646358494116) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(145.98765563964844) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018664849922060966 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018664849922060966 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:322.55859375 - response_length/max:811.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:322.55859375 - response_length_non_aborted/max:811.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:408.5 - prompt_length/max:499.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.0463561415672302e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7961415387690067) - timing_s/agent_loop/generate_sequences/max:np.float64(6.301008032634854) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5712843748697196) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.301008032634854) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:811 - timing_s/gen:11.937636621296406 - timing_s/reward:0.04937828332185745 - timing_s/old_log_prob:2.224254881963134 - timing_s/adv:0.5445820018649101 - timing_s/update_actor:15.715799508616328 - timing_s/step:30.508695298805833 - timing_s/stop_profile:0.00011160597205162048 - timing_per_token_ms/gen:0.14456720098451598 - timing_per_token_ms/adv:0.0029098535506885356 - timing_per_token_ms/update_actor:0.0839739007999761 - perf/total_num_tokens:187151 - perf/time_per_step:30.508695298805833 - perf/throughput:766.7936885165876 (TaskRunner pid=2167046) Training Progress: 57%|█████▋ | 57/100 [33:59<22:40, 31.63s/it] (TaskRunner pid=2167046) step:58 - global_seqlen/min:20316 - global_seqlen/max:25776 - global_seqlen/minmax_diff:5460 - global_seqlen/balanced_min:22966 - global_seqlen/balanced_max:22968 - global_seqlen/mean:22967.125 - actor/entropy:0.05042630061507225 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48229533433914185 - training/rollout_probs_diff_mean:0.0019069842528551817 - training/rollout_probs_diff_std:0.01095327828079462 - training/rollout_actor_probs_pearson_corr:0.9958901405334473 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.8359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0001697540283203 - rollout_corr/rollout_is_ratio_fraction_high:3.494507836876437e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013978031347505748 - rollout_corr/rollout_is_max:2.1670167446136475 - rollout_corr/rollout_is_min:0.20139548182487488 - rollout_corr/rollout_is_std:0.029367996379733086 - rollout_corr/rollout_is_eff_sample_size:0.999138621031368 - rollout_corr/rollout_is_seq_mean:1.00016450881958 - rollout_corr/rollout_is_seq_std:0.0020526256412267685 - rollout_corr/rollout_is_seq_max:1.0117377042770386 - rollout_corr/rollout_is_seq_min:0.9942534565925598 - rollout_corr/rollout_is_seq_max_deviation:0.011737704277038574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0489178416319191) - rollout_corr/training_log_ppl:np.float64(0.047033522495439684) - rollout_corr/kl:np.float64(0.0005489920703753981) - rollout_corr/k3_kl:np.float64(0.0007287665723376691) - rollout_corr/rollout_ppl:np.float64(1.0483349319547415) - rollout_corr/rollout_log_ppl:np.float64(0.04648453013442122) - rollout_corr/log_ppl_diff:np.float64(0.0005489923610184633) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017202749072566803) - rollout_corr/log_ppl_diff_max:np.float64(0.008941486477851868) - rollout_corr/log_ppl_diff_min:np.float64(-0.007565155625343323) - rollout_corr/ppl_ratio:np.float64(1.000551925972104) - rollout_corr/chi2_token:np.float64(0.00182512984611094) - rollout_corr/chi2_seq:np.float64(0.32337166904471815) - actor/pg_loss:np.float64(0.00039528647903352976) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040576129458713694) - actor/ppo_kl:np.float64(0.00020430290850015353) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_w_mean:np.float64(142336.7662561175) - self_distillation/token_reweight_w_std:np.float64(1948578.4343058746) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.00024026283063) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014259436655265745) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.18195318803191185) - perf/mfu/actor:np.float64(0.04188602836543823) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.039306640625) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.8359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0026587380561977625 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0026587380561977625 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.34765625 - response_length/max:590.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.34765625 - response_length_non_aborted/max:590.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:382.375 - prompt_length/max:482.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018070824444293976 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6991853434592485) - timing_s/agent_loop/generate_sequences/max:np.float64(5.374585622921586) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5676894962161896) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.374585622921586) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:569 - timing_s/gen:11.22432673163712 - timing_s/reward:0.06401835568249226 - timing_s/old_log_prob:2.27885109372437 - timing_s/adv:0.5373944751918316 - timing_s/update_actor:15.59491010569036 - timing_s/step:29.785723138600588 - timing_s/stop_profile:0.00012184865772724152 - timing_per_token_ms/gen:0.1307449909915913 - timing_per_token_ms/adv:0.0029248027081743557 - timing_per_token_ms/update_actor:0.08487626392991264 - perf/total_num_tokens:183737 - perf/time_per_step:29.785723138600588 - perf/throughput:771.0783079909826 (TaskRunner pid=2167046) Training Progress: 58%|█████▊ | 58/100 [34:29<21:46, 31.10s/it] (TaskRunner pid=2167046) step:59 - global_seqlen/min:20116 - global_seqlen/max:24874 - global_seqlen/minmax_diff:4758 - global_seqlen/balanced_min:22054 - global_seqlen/balanced_max:22057 - global_seqlen/mean:22055.625 - actor/entropy:0.0500376932322979 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5391990542411804 - training/rollout_probs_diff_mean:0.002085200510919094 - training/rollout_probs_diff_std:0.011515925638377666 - training/rollout_actor_probs_pearson_corr:0.9953073859214783 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.000190019607544 - rollout_corr/rollout_is_ratio_fraction_high:0.00011397454363759607 - rollout_corr/rollout_is_ratio_fraction_low:0.00018995757272932678 - rollout_corr/rollout_is_max:2.691387176513672 - rollout_corr/rollout_is_min:0.25212857127189636 - rollout_corr/rollout_is_std:0.031689416617155075 - rollout_corr/rollout_is_eff_sample_size:0.9989971452998435 - rollout_corr/rollout_is_seq_mean:1.0002810955047607 - rollout_corr/rollout_is_seq_std:0.0024625116493552923 - rollout_corr/rollout_is_seq_max:1.0140364170074463 - rollout_corr/rollout_is_seq_min:0.9921025037765503 - rollout_corr/rollout_is_seq_max_deviation:0.014036417007446289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0487651862204075) - rollout_corr/training_log_ppl:np.float64(0.04702805262786569) - rollout_corr/kl:np.float64(0.00038048617622354897) - rollout_corr/k3_kl:np.float64(0.0007050287913870079) - rollout_corr/rollout_ppl:np.float64(1.0483574215322733) - rollout_corr/rollout_log_ppl:np.float64(0.04664756611236953) - rollout_corr/log_ppl_diff:np.float64(0.00038048651549615897) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018079983346979134) - rollout_corr/log_ppl_diff_max:np.float64(0.009737156331539154) - rollout_corr/log_ppl_diff_min:np.float64(-0.014218524098396301) - rollout_corr/ppl_ratio:np.float64(1.0003841938450933) - rollout_corr/chi2_token:np.float64(0.0023028196301311255) - rollout_corr/chi2_seq:np.float64(1.128127784235403) - actor/pg_loss:np.float64(0.00022415840066969395) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020612081243598368) - actor/ppo_kl:np.float64(0.00010932901325944044) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(50009.51330480375) - self_distillation/token_reweight_w_std:np.float64(780175.9249367046) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999425960704684) - self_distillation/token_reweight_w_clip_frac:np.float64(0.009332494970294647) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.09782755840569735) - perf/mfu/actor:np.float64(0.04061451686622483) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.05368041992188) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017222820315510035 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017222820315510035 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:308.45703125 - response_length/max:727.0 - response_length/min:132.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.45703125 - response_length_non_aborted/max:727.0 - response_length_non_aborted/min:132.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:380.78125 - prompt_length/max:518.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011824630200862885 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4989494867622852) - timing_s/agent_loop/generate_sequences/max:np.float64(5.6656965631991625) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.260860279238841) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.6656965631991625) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:727 - timing_s/gen:11.556723687797785 - timing_s/reward:0.05912894383072853 - timing_s/old_log_prob:2.2388291228562593 - timing_s/adv:0.5177878700196743 - timing_s/update_actor:15.652064887806773 - timing_s/step:30.112237552180886 - timing_s/stop_profile:0.00011169537901878357 - timing_per_token_ms/gen:0.14635248132460946 - timing_per_token_ms/adv:0.0029345567741770767 - timing_per_token_ms/update_actor:0.08870789700930473 - perf/total_num_tokens:176445 - perf/time_per_step:30.112237552180886 - perf/throughput:732.4472305248075 (TaskRunner pid=2167046) Training Progress: 59%|█████▉ | 59/100 [34:59<21:03, 30.82s/it] (TaskRunner pid=2167046) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2167046) validation generation end (TaskRunner pid=2167046) [prompt] system (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume $ V $ of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c \cdot \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given that all angles $ \alpha, \beta, \gamma $ are 90.00º, the cosine of each angle is 0. Substituting these values into the formula simplifies it to: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Using the given lattice parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) a = 7.82 \, \text{Å}, \quad b = 7.82 \, \text{Å}, \quad c = 17.88 \, \text{Å} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = 7.82 \times 7.82 \times 17.88 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then multiply by 17.88: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 61.1524 \times 17.88 \approx 1094.31 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2167440) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2167046) step:60 - global_seqlen/min:19789 - global_seqlen/max:26080 - global_seqlen/minmax_diff:6291 - global_seqlen/balanced_min:22973 - global_seqlen/balanced_max:22975 - global_seqlen/mean:22974.125 - actor/entropy:0.04030780494213104 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.548633337020874 - training/rollout_probs_diff_mean:0.0017152910586446524 - training/rollout_probs_diff_std:0.010967686772346497 - training/rollout_actor_probs_pearson_corr:0.9947925209999084 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65234375 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65234375 - rollout_corr/rollout_is_mean:1.0000240802764893 - rollout_corr/rollout_is_ratio_fraction_high:3.4909295209217817e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016291004430968314 - rollout_corr/rollout_is_max:2.5243659019470215 - rollout_corr/rollout_is_min:0.3289499878883362 - rollout_corr/rollout_is_std:0.02684214897453785 - rollout_corr/rollout_is_eff_sample_size:0.99928001780181 - rollout_corr/rollout_is_seq_mean:1.0000858306884766 - rollout_corr/rollout_is_seq_std:0.00180041731800884 - rollout_corr/rollout_is_seq_max:1.0107128620147705 - rollout_corr/rollout_is_seq_min:0.9954133033752441 - rollout_corr/rollout_is_seq_max_deviation:0.010712862014770508 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0415030517615378) - rollout_corr/training_log_ppl:np.float64(0.04021239400572085) - rollout_corr/kl:np.float64(0.0004116895854844316) - rollout_corr/k3_kl:np.float64(0.00043681999613109923) - rollout_corr/rollout_ppl:np.float64(1.0410730261355639) - rollout_corr/rollout_log_ppl:np.float64(0.039800704115805274) - rollout_corr/log_ppl_diff:np.float64(0.00041168988991557853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012860779506809195) - rollout_corr/log_ppl_diff_max:np.float64(0.007598474621772766) - rollout_corr/log_ppl_diff_min:np.float64(-0.0053790658712387085) - rollout_corr/ppl_ratio:np.float64(1.0004133372567594) - rollout_corr/chi2_token:np.float64(0.0008693772833794355) - rollout_corr/chi2_seq:np.float64(0.27017642348073423) - actor/pg_loss:np.float64(7.360754534602165e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.8638998123351485e-05) - actor/ppo_kl:np.float64(8.944077894434965e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(39921.53101348528) - self_distillation/token_reweight_w_std:np.float64(599336.2581787109) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000242586946115) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0022192828328115866) - self_distillation/empty_target_batch:np.float64(0.34765625) - actor/grad_norm:np.float64(0.05733739724382758) - perf/mfu/actor:np.float64(0.04283900249480353) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.04742813110352) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7606382978723404) - val-aux/sciknoweval/reward/std@16:np.float64(0.030302917070648274) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7705531914893617) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.014673598442778796) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7500212765957448) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.03543724585981525) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7601702127659575) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.030115812466110246) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7747765957446809) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.005226268367911392) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7361914893617021) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.03868224938421085) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.765) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.02065664987576481) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7762659574468085) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0018438045478039392) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7198297872340426) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.03827583722765706) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7670957446808511) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.009773121229997817) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7765851063829786) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.0003362442687080662) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.7018191489361703) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.031682913847846) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7673404255319148) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.005752764524641576) - val-aux/sciknoweval/score/mean@16:np.float64(0.7606382978723404) - val-aux/sciknoweval/score/std@16:np.float64(0.030302917070648274) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7705531914893617) - val-aux/sciknoweval/score/best@2/std:np.float64(0.014673598442778796) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7500212765957448) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.03543724585981525) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7601702127659575) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.030115812466110246) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7747765957446809) - val-aux/sciknoweval/score/best@4/std:np.float64(0.005226268367911392) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7361914893617021) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.03868224938421085) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.765) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.02065664987576481) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7762659574468085) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0018438045478039392) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7198297872340426) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.03827583722765706) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7670957446808511) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.009773121229997817) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7765851063829786) - val-aux/sciknoweval/score/best@16/std:np.float64(0.0003362442687080662) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.7018191489361703) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.031682913847846) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7673404255319148) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.005752764524641576) - val-core/sciknoweval/acc/mean@16:np.float64(0.7606382978723404) - val-aux/sciknoweval/acc/std@16:np.float64(0.030302917070648274) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7705531914893617) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.014673598442778796) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7500212765957448) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.03543724585981525) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7601702127659575) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.030115812466110246) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7747765957446809) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.005226268367911392) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7361914893617021) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.03868224938421085) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.765) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.02065664987576481) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7762659574468085) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0018438045478039392) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7198297872340426) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.03827583722765706) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7670957446808511) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.009773121229997817) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7765851063829786) - val-core/sciknoweval/acc/best@16/std:np.float64(0.0003362442687080662) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.7018191489361703) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.031682913847846) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7673404255319148) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.005752764524641576) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00016581914678681642 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:0.00016581914678681642 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:335.69140625 - response_length/max:621.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.69140625 - response_length_non_aborted/max:621.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:382.25 - prompt_length/max:495.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011176057159900665 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5634933914989233) - timing_s/agent_loop/generate_sequences/max:np.float64(5.524224644526839) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.710239077008737) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.524224644526839) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:621 - timing_s/gen:11.40767808444798 - timing_s/reward:0.059924423694610596 - timing_s/old_log_prob:2.298730293288827 - timing_s/adv:0.5085990130901337 - timing_s/update_actor:15.473632106557488 - timing_s/step:29.836819676682353 - timing_s/testing:37.287454918026924 - timing_s/save_checkpoint:6.898927379399538 - timing_s/stop_profile:0.00013572163879871368 - timing_per_token_ms/gen:0.13274466276979624 - timing_per_token_ms/adv:0.0027672382141329306 - timing_per_token_ms/update_actor:0.08419054102472612 - perf/total_num_tokens:183793 - perf/time_per_step:29.836819676682353 - perf/throughput:769.9924204037876 (TaskRunner pid=2167046) Training Progress: 60%|██████ | 60/100 [36:13<29:11, 43.79s/it] (TaskRunner pid=2167046) step:61 - global_seqlen/min:19769 - global_seqlen/max:26095 - global_seqlen/minmax_diff:6326 - global_seqlen/balanced_min:23471 - global_seqlen/balanced_max:23472 - global_seqlen/mean:23471.5 - actor/entropy:0.04185541719198227 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4555684030056 - training/rollout_probs_diff_mean:0.0017948037711903453 - training/rollout_probs_diff_std:0.010674769058823586 - training/rollout_actor_probs_pearson_corr:0.9954621195793152 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.83984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0000584125518799 - rollout_corr/rollout_is_ratio_fraction_high:1.218739362229826e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013406132347881794 - rollout_corr/rollout_is_max:3.2528841495513916 - rollout_corr/rollout_is_min:0.2887685298919678 - rollout_corr/rollout_is_std:0.028913918882608414 - rollout_corr/rollout_is_eff_sample_size:0.9991648025994506 - rollout_corr/rollout_is_seq_mean:1.0000717639923096 - rollout_corr/rollout_is_seq_std:0.0017382517689839005 - rollout_corr/rollout_is_seq_max:1.0059905052185059 - rollout_corr/rollout_is_seq_min:0.9953637719154358 - rollout_corr/rollout_is_seq_max_deviation:0.005990505218505859 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0430718776769936) - rollout_corr/training_log_ppl:np.float64(0.041932695094146766) - rollout_corr/kl:np.float64(0.00048433673313930115) - rollout_corr/k3_kl:np.float64(0.0005176498646184413) - rollout_corr/rollout_ppl:np.float64(1.0425654179416597) - rollout_corr/rollout_log_ppl:np.float64(0.04144835810348013) - rollout_corr/log_ppl_diff:np.float64(0.00048433699066663394) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014789276692681597) - rollout_corr/log_ppl_diff_max:np.float64(0.00832807645201683) - rollout_corr/log_ppl_diff_min:np.float64(-0.005034495145082474) - rollout_corr/ppl_ratio:np.float64(1.0004862751811743) - rollout_corr/chi2_token:np.float64(0.0011503582354635) - rollout_corr/chi2_seq:np.float64(0.4335774367209524) - actor/pg_loss:np.float64(0.0006099274614825845) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.566592052578926e-05) - actor/ppo_kl:np.float64(0.0001280484544565752) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_w_mean:np.float64(108078.03998975107) - self_distillation/token_reweight_w_std:np.float64(1306800.5061330106) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000541384797543) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006611851000343449) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.14281610306352377) - perf/mfu/actor:np.float64(0.04358161947414355) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.56989288330078) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.83984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.83984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001662055728957057 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.001662055728957057 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:320.515625 - response_length/max:624.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:320.515625 - response_length_non_aborted/max:624.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.96875 - prompt_length/max:509.0 - prompt_length/min:269.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017317011952400208 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6565555464476347) - timing_s/agent_loop/generate_sequences/max:np.float64(5.370139880105853) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5645750863914145) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.370139880105853) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:287 - timing_s/agent_loop/slowest/response_length:624 - timing_s/gen:11.2309581451118 - timing_s/reward:0.054320551455020905 - timing_s/old_log_prob:2.3640927094966173 - timing_s/adv:0.5403022803366184 - timing_s/update_actor:15.53094632178545 - timing_s/step:29.81940203718841 - timing_s/stop_profile:2.927519381046295e-05 - timing_per_token_ms/gen:0.13687610472763367 - timing_per_token_ms/adv:0.002877437958463554 - timing_per_token_ms/update_actor:0.08271172657150933 - perf/total_num_tokens:187772 - perf/time_per_step:29.81940203718841 - perf/throughput:787.121752834889 (TaskRunner pid=2167046) Training Progress: 61%|██████ | 61/100 [36:43<25:44, 39.61s/it] (TaskRunner pid=2167046) step:62 - global_seqlen/min:18082 - global_seqlen/max:26343 - global_seqlen/minmax_diff:8261 - global_seqlen/balanced_min:22891 - global_seqlen/balanced_max:22894 - global_seqlen/mean:22892.875 - actor/entropy:0.04360160604119301 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9823201894760132 - training/rollout_probs_diff_mean:0.001706660958006978 - training/rollout_probs_diff_std:0.011156455613672733 - training/rollout_actor_probs_pearson_corr:0.9951454997062683 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.82421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:1.000069260597229 - rollout_corr/rollout_is_ratio_fraction_high:4.773326509166509e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011933316272916272 - rollout_corr/rollout_is_max:3.4115493297576904 - rollout_corr/rollout_is_min:0.0067672124132514 - rollout_corr/rollout_is_std:0.02733721025288105 - rollout_corr/rollout_is_eff_sample_size:0.999253473101148 - rollout_corr/rollout_is_seq_mean:1.000159740447998 - rollout_corr/rollout_is_seq_std:0.0017442916287109256 - rollout_corr/rollout_is_seq_max:1.0067665576934814 - rollout_corr/rollout_is_seq_min:0.9937759637832642 - rollout_corr/rollout_is_seq_max_deviation:0.006766557693481445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0414578090421855) - rollout_corr/training_log_ppl:np.float64(0.039926511135945475) - rollout_corr/kl:np.float64(0.00042428443190600795) - rollout_corr/k3_kl:np.float64(0.0007075274741481508) - rollout_corr/rollout_ppl:np.float64(1.041011284571141) - rollout_corr/rollout_log_ppl:np.float64(0.03950222592038699) - rollout_corr/log_ppl_diff:np.float64(0.0004242852155584842) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015441864725289633) - rollout_corr/log_ppl_diff_max:np.float64(0.024814952164888382) - rollout_corr/log_ppl_diff_min:np.float64(-0.005558319389820099) - rollout_corr/ppl_ratio:np.float64(1.000428590690717) - rollout_corr/chi2_token:np.float64(0.001744407694786787) - rollout_corr/chi2_seq:np.float64(0.31756631121970713) - actor/pg_loss:np.float64(0.0008215298876166344) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003732443005901587) - actor/ppo_kl:np.float64(7.225550693901184e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.82421875) - self_distillation/token_reweight_w_mean:np.float64(257022.4825459295) - self_distillation/token_reweight_w_std:np.float64(3306871.5474428586) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000569719588384) - self_distillation/token_reweight_w_clip_frac:np.float64(0.016503850085427985) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.21665046736598015) - perf/mfu/actor:np.float64(0.04265161260571607) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.3099021911621) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.82421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.82421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036590532399713993 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036590532399713993 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:327.33984375 - response_length/max:790.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.33984375 - response_length_non_aborted/max:790.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.0625 - prompt_length/max:541.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.878856897354126e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9862233083695173) - timing_s/agent_loop/generate_sequences/max:np.float64(6.082181049510837) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2972556818203884) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.082181049510837) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:790 - timing_s/gen:12.123094467446208 - timing_s/reward:0.05876629427075386 - timing_s/old_log_prob:2.234193356707692 - timing_s/adv:0.5234870482236147 - timing_s/update_actor:15.39575981348753 - timing_s/step:30.42191405594349 - timing_s/stop_profile:0.00013401173055171967 - timing_per_token_ms/gen:0.14466872477531006 - timing_per_token_ms/adv:0.0028583513878423675 - timing_per_token_ms/update_actor:0.08406414557743146 - perf/total_num_tokens:183143 - perf/time_per_step:30.42191405594349 - perf/throughput:752.5126445989498 (TaskRunner pid=2167046) Training Progress: 62%|██████▏ | 62/100 [37:13<23:20, 36.87s/it] (TaskRunner pid=2167046) step:63 - global_seqlen/min:20795 - global_seqlen/max:25263 - global_seqlen/minmax_diff:4468 - global_seqlen/balanced_min:23732 - global_seqlen/balanced_max:23734 - global_seqlen/mean:23732.875 - actor/entropy:0.04523954167962074 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6751085519790649 - training/rollout_probs_diff_mean:0.0018330732127651572 - training/rollout_probs_diff_std:0.010520632378757 - training/rollout_actor_probs_pearson_corr:0.9957168698310852 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.000042200088501 - rollout_corr/rollout_is_ratio_fraction_high:1.1687840924423654e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001285662583541125 - rollout_corr/rollout_is_max:2.0695643424987793 - rollout_corr/rollout_is_min:0.005992737133055925 - rollout_corr/rollout_is_std:0.028119659051299095 - rollout_corr/rollout_is_eff_sample_size:0.9992100285399822 - rollout_corr/rollout_is_seq_mean:1.000035285949707 - rollout_corr/rollout_is_seq_std:0.001678181579336524 - rollout_corr/rollout_is_seq_max:1.0064517259597778 - rollout_corr/rollout_is_seq_min:0.992402970790863 - rollout_corr/rollout_is_seq_max_deviation:0.007597029209136963 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0453694420866668) - rollout_corr/training_log_ppl:np.float64(0.043938727694694535) - rollout_corr/kl:np.float64(0.0006067258537747477) - rollout_corr/k3_kl:np.float64(0.0005313017011587817) - rollout_corr/rollout_ppl:np.float64(1.0447322241961956) - rollout_corr/rollout_log_ppl:np.float64(0.04333200147084426) - rollout_corr/log_ppl_diff:np.float64(0.0006067262238502735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013585495998995611) - rollout_corr/log_ppl_diff_max:np.float64(0.016803178936243057) - rollout_corr/log_ppl_diff_min:np.float64(-0.006356984376907349) - rollout_corr/ppl_ratio:np.float64(1.0006088686641306) - rollout_corr/chi2_token:np.float64(0.0007685464806854725) - rollout_corr/chi2_seq:np.float64(0.4505869532003999) - actor/pg_loss:np.float64(0.0004227783065289259) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.712959288459388e-05) - actor/ppo_kl:np.float64(0.00015468594890499787) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(139172.26291169436) - self_distillation/token_reweight_w_std:np.float64(1867242.3280806076) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001536742784083) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0052675219340017065) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.1391439251601696) - perf/mfu/actor:np.float64(0.043777487128157024) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.31776428222656) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001742949360050261 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:0.001742949360050261 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:334.21484375 - response_length/max:613.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.21484375 - response_length_non_aborted/max:613.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.4375 - prompt_length/max:483.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014797411859035492 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5788266826421022) - timing_s/agent_loop/generate_sequences/max:np.float64(5.438034595921636) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.615576795367815) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.438034595921636) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:283 - timing_s/agent_loop/slowest/response_length:613 - timing_s/gen:11.706054279580712 - timing_s/reward:0.06105692870914936 - timing_s/old_log_prob:2.2562557756900787 - timing_s/adv:0.5328396391123533 - timing_s/update_actor:15.465085987001657 - timing_s/step:30.10845929943025 - timing_s/stop_profile:0.0001232922077178955 - timing_per_token_ms/gen:0.13681850278264954 - timing_per_token_ms/adv:0.0028064427461503994 - timing_per_token_ms/update_actor:0.08145392197006082 - perf/total_num_tokens:189863 - perf/time_per_step:30.10845929943025 - perf/throughput:788.2460794149338 (TaskRunner pid=2167046) Training Progress: 63%|██████▎ | 63/100 [37:44<21:29, 34.86s/it] (TaskRunner pid=2167046) step:64 - global_seqlen/min:20088 - global_seqlen/max:25585 - global_seqlen/minmax_diff:5497 - global_seqlen/balanced_min:23119 - global_seqlen/balanced_max:23121 - global_seqlen/mean:23120.25 - actor/entropy:0.042551927268505096 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083586573600769 - training/rollout_probs_diff_mean:0.0017879114020615816 - training/rollout_probs_diff_std:0.010721568018198013 - training/rollout_actor_probs_pearson_corr:0.9953984022140503 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001569986343384 - rollout_corr/rollout_is_ratio_fraction_high:7.074637687765062e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011791062570409849 - rollout_corr/rollout_is_max:10.94495964050293 - rollout_corr/rollout_is_min:0.27272939682006836 - rollout_corr/rollout_is_std:0.028316102921962738 - rollout_corr/rollout_is_eff_sample_size:0.9991989597077329 - rollout_corr/rollout_is_seq_mean:1.000223994255066 - rollout_corr/rollout_is_seq_std:0.0025728975888341665 - rollout_corr/rollout_is_seq_max:1.0256704092025757 - rollout_corr/rollout_is_seq_min:0.9949301481246948 - rollout_corr/rollout_is_seq_max_deviation:0.025670409202575684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0416441741399467) - rollout_corr/training_log_ppl:np.float64(0.04043069246199593) - rollout_corr/kl:np.float64(0.0003189195853421367) - rollout_corr/k3_kl:np.float64(0.0004776339879981606) - rollout_corr/rollout_ppl:np.float64(1.0413051219657063) - rollout_corr/rollout_log_ppl:np.float64(0.040111772246746114) - rollout_corr/log_ppl_diff:np.float64(0.0003189202152498183) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013647890837091836) - rollout_corr/log_ppl_diff_max:np.float64(0.006027184426784515) - rollout_corr/log_ppl_diff_min:np.float64(-0.007712792605161667) - rollout_corr/ppl_ratio:np.float64(1.000320767518133) - rollout_corr/chi2_token:np.float64(0.0015714333858340979) - rollout_corr/chi2_seq:np.float64(0.45822112914174795) - actor/pg_loss:np.float64(0.0006036241538822651) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.704760395834455e-05) - actor/ppo_kl:np.float64(3.3609087846109276e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(25306.706190291326) - self_distillation/token_reweight_w_std:np.float64(409929.1284506519) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997767452150583) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006463439349317923) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.1090348232537508) - perf/mfu/actor:np.float64(0.04248081094347521) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.26595306396484) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:3.537318843882531e-05 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:3.537318843882531e-05 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:331.2890625 - response_length/max:733.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.2890625 - response_length_non_aborted/max:733.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.21875 - prompt_length/max:473.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014341622591018677 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6459696050733328) - timing_s/agent_loop/generate_sequences/max:np.float64(5.94431322067976) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6513856843157555) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.94431322067976) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:733 - timing_s/gen:11.87289572879672 - timing_s/reward:0.05930803343653679 - timing_s/old_log_prob:2.3765826784074306 - timing_s/adv:0.5368497855961323 - timing_s/update_actor:15.625425288453698 - timing_s/step:30.555814055725932 - timing_s/stop_profile:0.00010952353477478027 - timing_per_token_ms/gen:0.13999405410678836 - timing_per_token_ms/adv:0.0029024869194544407 - timing_per_token_ms/update_actor:0.08447911078196439 - perf/total_num_tokens:184962 - perf/time_per_step:30.555814055725932 - perf/throughput:756.6563259559906 (TaskRunner pid=2167046) Training Progress: 64%|██████▍ | 64/100 [38:14<20:08, 33.58s/it] (TaskRunner pid=2167046) step:65 - global_seqlen/min:20020 - global_seqlen/max:25600 - global_seqlen/minmax_diff:5580 - global_seqlen/balanced_min:23287 - global_seqlen/balanced_max:23289 - global_seqlen/mean:23288.125 - actor/entropy:0.04803186282515526 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967540204524994 - training/rollout_probs_diff_mean:0.001957700354978442 - training/rollout_probs_diff_std:0.01209981832653284 - training/rollout_actor_probs_pearson_corr:0.9949580430984497 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000300407409668 - rollout_corr/rollout_is_ratio_fraction_high:3.559521064744331e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001898411283036694 - rollout_corr/rollout_is_max:2.54838228225708 - rollout_corr/rollout_is_min:0.11307276040315628 - rollout_corr/rollout_is_std:0.02980916015803814 - rollout_corr/rollout_is_eff_sample_size:0.9991120838421531 - rollout_corr/rollout_is_seq_mean:1.0000916719436646 - rollout_corr/rollout_is_seq_std:0.001912038424052298 - rollout_corr/rollout_is_seq_max:1.0090073347091675 - rollout_corr/rollout_is_seq_min:0.9943963289260864 - rollout_corr/rollout_is_seq_max_deviation:0.00900733470916748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.047498204279691) - rollout_corr/training_log_ppl:np.float64(0.0458258127400768) - rollout_corr/kl:np.float64(0.0006745162798118542) - rollout_corr/k3_kl:np.float64(0.0006516177231024045) - rollout_corr/rollout_ppl:np.float64(1.0467801187187433) - rollout_corr/rollout_log_ppl:np.float64(0.045151295971663785) - rollout_corr/log_ppl_diff:np.float64(0.0006745167684130138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016444117263745284) - rollout_corr/log_ppl_diff_max:np.float64(0.010444030165672302) - rollout_corr/log_ppl_diff_min:np.float64(-0.007610205560922623) - rollout_corr/ppl_ratio:np.float64(1.0006771273911) - rollout_corr/chi2_token:np.float64(0.0012553422711789608) - rollout_corr/chi2_seq:np.float64(0.21363996737636626) - actor/pg_loss:np.float64(0.0007566583808511496) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003426368589316553) - actor/ppo_kl:np.float64(0.0002434091673113592) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(132425.8512902113) - self_distillation/token_reweight_w_std:np.float64(1851148.4486397996) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997992021963) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0151770984812174) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.18636032566428185) - perf/mfu/actor:np.float64(0.0430304463908559) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.26295471191406) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0038264852482825518 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0038264852482825518 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:329.22265625 - response_length/max:741.0 - response_length/min:139.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:329.22265625 - response_length_non_aborted/max:741.0 - response_length_non_aborted/min:139.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:398.53125 - prompt_length/max:490.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.276345372200012e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5382776949554682) - timing_s/agent_loop/generate_sequences/max:np.float64(5.978386091068387) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.5521642115418217) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.978386091068387) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:741 - timing_s/gen:12.125503525137901 - timing_s/reward:0.05911542475223541 - timing_s/old_log_prob:2.2964321319013834 - timing_s/adv:0.5274523887783289 - timing_s/update_actor:15.513752859085798 - timing_s/step:30.610530648380518 - timing_s/stop_profile:0.00011078454554080963 - timing_per_token_ms/gen:0.1438699531939334 - timing_per_token_ms/adv:0.0028311230980291933 - timing_per_token_ms/update_actor:0.08327072735077319 - perf/total_num_tokens:186305 - perf/time_per_step:30.610530648380518 - perf/throughput:760.7880198977238 (TaskRunner pid=2167046) Training Progress: 65%|██████▌ | 65/100 [38:45<19:04, 32.70s/it] (TaskRunner pid=2167046) step:66 - global_seqlen/min:17558 - global_seqlen/max:25411 - global_seqlen/minmax_diff:7853 - global_seqlen/balanced_min:22498 - global_seqlen/balanced_max:22502 - global_seqlen/mean:22500.875 - actor/entropy:0.039825305342674255 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400331974029541 - training/rollout_probs_diff_mean:0.0018288729479536414 - training/rollout_probs_diff_std:0.011651387438178062 - training/rollout_actor_probs_pearson_corr:0.994309663772583 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.72265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0000613927841187 - rollout_corr/rollout_is_ratio_fraction_high:1.240217807207955e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016122830857057124 - rollout_corr/rollout_is_max:2.01051664352417 - rollout_corr/rollout_is_min:0.2722620368003845 - rollout_corr/rollout_is_std:0.02900242619216442 - rollout_corr/rollout_is_eff_sample_size:0.9991596851850771 - rollout_corr/rollout_is_seq_mean:1.000057339668274 - rollout_corr/rollout_is_seq_std:0.0020476924255490303 - rollout_corr/rollout_is_seq_max:1.007577657699585 - rollout_corr/rollout_is_seq_min:0.9939850568771362 - rollout_corr/rollout_is_seq_max_deviation:0.007577657699584961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0409869737923145) - rollout_corr/training_log_ppl:np.float64(0.03976573963245755) - rollout_corr/kl:np.float64(0.0005508014146560214) - rollout_corr/k3_kl:np.float64(0.0007626194736367609) - rollout_corr/rollout_ppl:np.float64(1.040397803299129) - rollout_corr/rollout_log_ppl:np.float64(0.03921493800089593) - rollout_corr/log_ppl_diff:np.float64(0.000550801631561626) - rollout_corr/log_ppl_abs_diff:np.float64(0.001862790147697524) - rollout_corr/log_ppl_diff_max:np.float64(0.012821070849895477) - rollout_corr/log_ppl_diff_min:np.float64(-0.006489671766757965) - rollout_corr/ppl_ratio:np.float64(1.0005543171428144) - rollout_corr/chi2_token:np.float64(0.002054796786978841) - rollout_corr/chi2_seq:np.float64(0.5148234020452946) - actor/pg_loss:np.float64(0.0013034224975854158) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003754316767299315) - actor/ppo_kl:np.float64(0.00012462263222268177) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.72265625) - self_distillation/token_reweight_w_mean:np.float64(184329.6079326775) - self_distillation/token_reweight_w_std:np.float64(2585825.313373127) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004278225824237) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015970634303812403) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.3207257315516472) - perf/mfu/actor:np.float64(0.041364441977581275) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.18589401245117) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.72265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.72265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0010541850933805108 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0010541850933805108 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:314.96484375 - response_length/max:643.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.96484375 - response_length_non_aborted/max:643.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.1875 - prompt_length/max:475.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001494530588388443 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5015150532126427) - timing_s/agent_loop/generate_sequences/max:np.float64(5.508818969130516) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.419145724561531) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.508818969130516) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:431 - timing_s/agent_loop/slowest/response_length:643 - timing_s/gen:11.514163475483656 - timing_s/reward:0.05881977640092373 - timing_s/old_log_prob:2.2771597616374493 - timing_s/adv:0.5236806087195873 - timing_s/update_actor:15.603892534971237 - timing_s/step:30.062982780858874 - timing_s/stop_profile:3.5936012864112854e-05 - timing_per_token_ms/gen:0.14280070289942648 - timing_per_token_ms/adv:0.002909223578636316 - timing_per_token_ms/update_actor:0.08668492078069874 - perf/total_num_tokens:180007 - perf/time_per_step:30.062982780858874 - perf/throughput:748.4578348069416 (TaskRunner pid=2167046) Training Progress: 66%|██████▌ | 66/100 [39:15<18:05, 31.92s/it] (TaskRunner pid=2167046) step:67 - global_seqlen/min:19699 - global_seqlen/max:23633 - global_seqlen/minmax_diff:3934 - global_seqlen/balanced_min:21599 - global_seqlen/balanced_max:21601 - global_seqlen/mean:21600.25 - actor/entropy:0.03609660267829895 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3185198903083801 - training/rollout_probs_diff_mean:0.0016948558622971177 - training/rollout_probs_diff_std:0.01066279411315918 - training/rollout_actor_probs_pearson_corr:0.9946100115776062 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.80078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001091957092285 - rollout_corr/rollout_is_ratio_fraction_high:4.045634705107659e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.394179606810212e-05 - rollout_corr/rollout_is_max:3.5274367332458496 - rollout_corr/rollout_is_min:0.22336265444755554 - rollout_corr/rollout_is_std:0.027806321159005165 - rollout_corr/rollout_is_eff_sample_size:0.9992275249362932 - rollout_corr/rollout_is_seq_mean:1.0002102851867676 - rollout_corr/rollout_is_seq_std:0.00230899965390563 - rollout_corr/rollout_is_seq_max:1.0178509950637817 - rollout_corr/rollout_is_seq_min:0.995329737663269 - rollout_corr/rollout_is_seq_max_deviation:0.01785099506378174 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0353782731108367) - rollout_corr/training_log_ppl:np.float64(0.034475101392672514) - rollout_corr/kl:np.float64(0.0006411960302656894) - rollout_corr/k3_kl:np.float64(0.0007077014818066552) - rollout_corr/rollout_ppl:np.float64(1.0347134619951248) - rollout_corr/rollout_log_ppl:np.float64(0.03383390524686547) - rollout_corr/log_ppl_diff:np.float64(0.0006411961458070436) - rollout_corr/log_ppl_abs_diff:np.float64(0.001701803797914181) - rollout_corr/log_ppl_diff_max:np.float64(0.011103812605142593) - rollout_corr/log_ppl_diff_min:np.float64(-0.006912939250469208) - rollout_corr/ppl_ratio:np.float64(1.000644029583782) - rollout_corr/chi2_token:np.float64(0.0017004713881760836) - rollout_corr/chi2_seq:np.float64(0.27700750855728984) - actor/pg_loss:np.float64(0.00027153268456459045) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010332653255318291) - actor/ppo_kl:np.float64(0.00038016634875148725) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_w_mean:np.float64(33479.28833753336) - self_distillation/token_reweight_w_std:np.float64(415466.60549042985) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001371470279992) - self_distillation/token_reweight_w_clip_frac:np.float64(0.002036116988165304) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.11304023303091526) - perf/mfu/actor:np.float64(0.03992824987806824) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.28586959838867) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:2 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000815869658254087 - critic/advantages/max:0.375 - critic/advantages/min:-0.625 - critic/returns/mean:0.000815869658254087 - critic/returns/max:0.375 - critic/returns/min:-0.625 - response_length/mean:289.6640625 - response_length/max:748.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.6640625 - response_length_non_aborted/max:748.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.34375 - prompt_length/max:472.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.121823728084564e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4565069302916527) - timing_s/agent_loop/generate_sequences/max:np.float64(5.718095608055592) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1236164262954844) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.718095608055592) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:748 - timing_s/gen:11.893555466085672 - timing_s/reward:0.058034129440784454 - timing_s/old_log_prob:2.292636835947633 - timing_s/adv:0.5232712011784315 - timing_s/update_actor:15.200434923171997 - timing_s/step:30.055124912410975 - timing_s/stop_profile:0.00011288747191429138 - timing_per_token_ms/gen:0.1603899380490017 - timing_per_token_ms/adv:0.0030281547735467847 - timing_per_token_ms/update_actor:0.08796446177227114 - perf/total_num_tokens:172802 - perf/time_per_step:30.055124912410975 - perf/throughput:718.6877466970827 (TaskRunner pid=2167046) Training Progress: 67%|██████▋ | 67/100 [39:45<17:15, 31.38s/it] (TaskRunner pid=2167046) step:68 - global_seqlen/min:16526 - global_seqlen/max:23491 - global_seqlen/minmax_diff:6965 - global_seqlen/balanced_min:21012 - global_seqlen/balanced_max:21016 - global_seqlen/mean:21015.25 - actor/entropy:0.05100872367620468 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2822123169898987 - training/rollout_probs_diff_mean:0.001955451210960746 - training/rollout_probs_diff_std:0.010564039461314678 - training/rollout_actor_probs_pearson_corr:0.9961839318275452 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.87109375 - self_distillation/correct_sample_fraction:0.8203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.87109375 - rollout_corr/rollout_is_mean:1.000083565711975 - rollout_corr/rollout_is_ratio_fraction_high:5.4052594350650907e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.459204011363909e-05 - rollout_corr/rollout_is_max:4.378950119018555 - rollout_corr/rollout_is_min:0.22583216428756714 - rollout_corr/rollout_is_std:0.028775470331311226 - rollout_corr/rollout_is_eff_sample_size:0.9991726573374431 - rollout_corr/rollout_is_seq_mean:1.0001046657562256 - rollout_corr/rollout_is_seq_std:0.0021570385433733463 - rollout_corr/rollout_is_seq_max:1.0077972412109375 - rollout_corr/rollout_is_seq_min:0.9944034814834595 - rollout_corr/rollout_is_seq_max_deviation:0.0077972412109375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0534572070464492) - rollout_corr/training_log_ppl:np.float64(0.05067192515161878) - rollout_corr/kl:np.float64(0.0007594044693028934) - rollout_corr/k3_kl:np.float64(0.0008045174472961492) - rollout_corr/rollout_ppl:np.float64(1.0526364957913756) - rollout_corr/rollout_log_ppl:np.float64(0.04991252027957671) - rollout_corr/log_ppl_diff:np.float64(0.0007594048720420687) - rollout_corr/log_ppl_abs_diff:np.float64(0.002040182679593272) - rollout_corr/log_ppl_diff_max:np.float64(0.01110206451267004) - rollout_corr/log_ppl_diff_min:np.float64(-0.00639449805021286) - rollout_corr/ppl_ratio:np.float64(1.000763491494581) - rollout_corr/chi2_token:np.float64(0.0016917763277888298) - rollout_corr/chi2_seq:np.float64(0.38484108657576144) - actor/pg_loss:np.float64(0.0016131881857290864) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00037061344210087555) - actor/ppo_kl:np.float64(0.00039652320700866994) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.87109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.87109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.8203125) - self_distillation/token_reweight_w_mean:np.float64(208727.49827101128) - self_distillation/token_reweight_w_std:np.float64(2129898.3024121746) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995387236122042) - self_distillation/token_reweight_w_clip_frac:np.float64(0.022508326030219905) - self_distillation/empty_target_batch:np.float64(0.12890625) - actor/grad_norm:np.float64(0.2767256274819374) - perf/mfu/actor:np.float64(0.03884775522311526) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.36953353881836) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:2 - critic/score/mean:0.8203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013580714585259557 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0013580714585259557 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:289.0703125 - response_length/max:598.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.0703125 - response_length_non_aborted/max:598.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:367.65625 - prompt_length/max:491.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013464875519275665 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5636711083352566) - timing_s/agent_loop/generate_sequences/max:np.float64(5.099744895473123) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1056349851496634) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.099744895473123) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:552 - timing_s/gen:10.939020650461316 - timing_s/reward:0.05838066153228283 - timing_s/old_log_prob:2.3343949746340513 - timing_s/adv:0.5343663319945335 - timing_s/update_actor:15.248274385929108 - timing_s/step:29.201415199786425 - timing_s/stop_profile:0.00010869652032852173 - timing_per_token_ms/gen:0.14782060823303853 - timing_per_token_ms/adv:0.003178443820526365 - timing_per_token_ms/update_actor:0.09069767422424851 - perf/total_num_tokens:168122 - perf/time_per_step:29.201415199786425 - perf/throughput:719.6654633421226 (TaskRunner pid=2167046) Training Progress: 68%|██████▊ | 68/100 [40:14<16:23, 30.74s/it] (TaskRunner pid=2167046) step:69 - global_seqlen/min:17617 - global_seqlen/max:23611 - global_seqlen/minmax_diff:5994 - global_seqlen/balanced_min:20846 - global_seqlen/balanced_max:20850 - global_seqlen/mean:20847.125 - actor/entropy:0.04170255362987518 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.430754691362381 - training/rollout_probs_diff_mean:0.0017275827703997493 - training/rollout_probs_diff_std:0.010673193261027336 - training/rollout_actor_probs_pearson_corr:0.9951395392417908 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.0000512599945068 - rollout_corr/rollout_is_ratio_fraction_high:4.057508340338245e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020287542429286987 - rollout_corr/rollout_is_max:2.476072311401367 - rollout_corr/rollout_is_min:0.26981011033058167 - rollout_corr/rollout_is_std:0.028634270653128624 - rollout_corr/rollout_is_eff_sample_size:0.9991807502269672 - rollout_corr/rollout_is_seq_mean:0.9999962449073792 - rollout_corr/rollout_is_seq_std:0.0020436632912606 - rollout_corr/rollout_is_seq_max:1.011947512626648 - rollout_corr/rollout_is_seq_min:0.9935392141342163 - rollout_corr/rollout_is_seq_max_deviation:0.01194751262664795 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0424060276709497) - rollout_corr/training_log_ppl:np.float64(0.040941476891930506) - rollout_corr/kl:np.float64(0.0003941324239971067) - rollout_corr/k3_kl:np.float64(0.0005813903811582588) - rollout_corr/rollout_ppl:np.float64(1.0419914699159563) - rollout_corr/rollout_log_ppl:np.float64(0.04054734420378736) - rollout_corr/log_ppl_diff:np.float64(0.00039413268814314506) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016268106724055542) - rollout_corr/log_ppl_diff_max:np.float64(0.010919928550720215) - rollout_corr/log_ppl_diff_min:np.float64(-0.007021263241767883) - rollout_corr/ppl_ratio:np.float64(1.0003970407415181) - rollout_corr/chi2_token:np.float64(0.001484232721850276) - rollout_corr/chi2_seq:np.float64(0.34094764827750623) - actor/pg_loss:np.float64(0.00032893416937440634) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021659644880855922) - actor/ppo_kl:np.float64(-7.26069886276548e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_w_mean:np.float64(98340.4069654087) - self_distillation/token_reweight_w_std:np.float64(1267416.7012998087) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006331857293844) - self_distillation/token_reweight_w_clip_frac:np.float64(0.009909781474561896) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.21244935132563114) - perf/mfu/actor:np.float64(0.03856798348514151) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.3545265197754) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:2 - critic/score/mean:0.828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00031783815938979387 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00031783815938979387 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:288.81640625 - response_length/max:635.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:288.81640625 - response_length_non_aborted/max:635.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:362.65625 - prompt_length/max:461.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001394525170326233 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1610720288008451) - timing_s/agent_loop/generate_sequences/max:np.float64(5.278335407376289) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.103333387945895) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.278335407376289) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:635 - timing_s/gen:11.23056485876441 - timing_s/reward:0.057341909036040306 - timing_s/old_log_prob:2.28742160461843 - timing_s/adv:0.5086922198534012 - timing_s/update_actor:15.477266134694219 - timing_s/step:29.65025979653001 - timing_s/stop_profile:0.00012357719242572784 - timing_per_token_ms/gen:0.1518937048942263 - timing_per_token_ms/adv:0.0030501341303261313 - timing_per_token_ms/update_actor:0.09280216177706889 - perf/total_num_tokens:166777 - perf/time_per_step:29.65025979653001 - perf/throughput:703.1009219838187 (TaskRunner pid=2167046) Training Progress: 69%|██████▉ | 69/100 [40:44<15:43, 30.43s/it] (TaskRunner pid=2167046) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2167046) validation generation end (TaskRunner pid=2167046) [prompt] system (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume $ V $ of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c \cdot \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given that all angles $ \alpha, \beta, \gamma $ are 90.00º, their cosines are 0. Substituting these values into the formula simplifies it to: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Substituting the given lattice parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = 7.82 \, \text{Å} \times 7.82 \, \text{Å} \times 17.88 \, \text{Å} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then multiply by 17.88: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 61.1524 \times 17.88 \approx 1094.31 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2167440) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2167046) step:70 - global_seqlen/min:18920 - global_seqlen/max:23896 - global_seqlen/minmax_diff:4976 - global_seqlen/balanced_min:21167 - global_seqlen/balanced_max:21171 - global_seqlen/mean:21168.875 - actor/entropy:0.044530898332595825 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5623742341995239 - training/rollout_probs_diff_mean:0.0020385412499308586 - training/rollout_probs_diff_std:0.012092887423932552 - training/rollout_actor_probs_pearson_corr:0.9944683313369751 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9999082684516907 - rollout_corr/rollout_is_ratio_fraction_high:1.431864711776143e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002577356353867799 - rollout_corr/rollout_is_max:2.489572763442993 - rollout_corr/rollout_is_min:1.5778536180732772e-05 - rollout_corr/rollout_is_std:0.0301333237439394 - rollout_corr/rollout_is_eff_sample_size:0.9990926875955936 - rollout_corr/rollout_is_seq_mean:0.9998764991760254 - rollout_corr/rollout_is_seq_std:0.0022292432840913534 - rollout_corr/rollout_is_seq_max:1.0073078870773315 - rollout_corr/rollout_is_seq_min:0.9925515055656433 - rollout_corr/rollout_is_seq_max_deviation:0.0074484944343566895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0505306674167514) - rollout_corr/training_log_ppl:np.float64(0.04811194552348752) - rollout_corr/kl:np.float64(0.0014961613618638125) - rollout_corr/k3_kl:np.float64(0.0010580281840617545) - rollout_corr/rollout_ppl:np.float64(1.0489219408482313) - rollout_corr/rollout_log_ppl:np.float64(0.04661578337800165) - rollout_corr/log_ppl_diff:np.float64(0.0014961621454858687) - rollout_corr/log_ppl_abs_diff:np.float64(0.00244221169850789) - rollout_corr/log_ppl_diff_max:np.float64(0.03330881893634796) - rollout_corr/log_ppl_diff_min:np.float64(-0.006231758743524551) - rollout_corr/ppl_ratio:np.float64(1.0015048009809107) - rollout_corr/chi2_token:np.float64(0.0012201927602291107) - rollout_corr/chi2_seq:np.float64(0.11082092486321926) - actor/pg_loss:np.float64(0.000838939449749887) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002512547634978546) - actor/ppo_kl:np.float64(0.0007138061801215123) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(323953.5671025575) - self_distillation/token_reweight_w_std:np.float64(3053798.2800295064) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005679957102984) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014940109875169583) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.32167357206344604) - perf/mfu/actor:np.float64(0.03943325036909458) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.27843856811523) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7666223404255319) - val-aux/sciknoweval/reward/std@16:np.float64(0.032785963866969546) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.778436170212766) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.02531669454454758) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7547553191489362) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.02932435814267925) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7669255319148935) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.03212767894847751) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7876063829787233) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.018373306211109434) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7434787234042554) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.02673062493039757) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7678829787234043) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.023418665335771412) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7945000000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.014540498308215178) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7321382978723404) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.02210945095685271) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7684787234042553) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.015149678269704906) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8006595744680851) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.011295678198616167) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.7221702127659575) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.01594366053527828) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7687659574468084) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.009281693465605173) - val-aux/sciknoweval/score/mean@16:np.float64(0.7666223404255319) - val-aux/sciknoweval/score/std@16:np.float64(0.032785963866969546) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.778436170212766) - val-aux/sciknoweval/score/best@2/std:np.float64(0.02531669454454758) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7547553191489362) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.02932435814267925) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7669255319148935) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.03212767894847751) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7876063829787233) - val-aux/sciknoweval/score/best@4/std:np.float64(0.018373306211109434) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7434787234042554) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.02673062493039757) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7678829787234043) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.023418665335771412) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7945000000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.014540498308215178) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7321382978723404) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.02210945095685271) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7684787234042553) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.015149678269704906) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8006595744680851) - val-aux/sciknoweval/score/best@16/std:np.float64(0.011295678198616167) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.7221702127659575) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.01594366053527828) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7687659574468084) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.009281693465605173) - val-core/sciknoweval/acc/mean@16:np.float64(0.7666223404255319) - val-aux/sciknoweval/acc/std@16:np.float64(0.032785963866969546) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.778436170212766) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.02531669454454758) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7547553191489362) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.02932435814267925) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7669255319148935) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.03212767894847751) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7876063829787233) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.018373306211109434) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7434787234042554) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.02673062493039757) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7678829787234043) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.023418665335771412) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7945000000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.014540498308215178) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7321382978723404) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.02210945095685271) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7684787234042553) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.015149678269704906) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8006595744680851) - val-core/sciknoweval/acc/best@16/std:np.float64(0.011295678198616167) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.7221702127659575) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.01594366053527828) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7687659574468084) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.009281693465605173) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:2 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00045640688040293753 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.00045640688040293753 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:272.80859375 - response_length/max:768.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.80859375 - response_length_non_aborted/max:768.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:388.71875 - prompt_length/max:512.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002062879502773285 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3907919973134995) - timing_s/agent_loop/generate_sequences/max:np.float64(5.686534350737929) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9022413608036004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.686534350737929) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:768 - timing_s/gen:11.487954104319215 - timing_s/reward:0.05927581526339054 - timing_s/old_log_prob:2.3286957256495953 - timing_s/adv:0.5266923401504755 - timing_s/update_actor:15.42138602770865 - timing_s/step:29.9099238589406 - timing_s/testing:36.23174679465592 - timing_s/save_checkpoint:6.721334571018815 - timing_s/stop_profile:0.0001097600907087326 - timing_per_token_ms/gen:0.16449196157332172 - timing_per_token_ms/adv:0.0031100633604199297 - timing_per_token_ms/update_actor:0.09106167679971568 - perf/total_num_tokens:169351 - perf/time_per_step:29.9099238589406 - perf/throughput:707.754225648831 (TaskRunner pid=2167046) Training Progress: 70%|███████ | 70/100 [41:57<21:35, 43.18s/it] (TaskRunner pid=2167046) step:71 - global_seqlen/min:19003 - global_seqlen/max:24898 - global_seqlen/minmax_diff:5895 - global_seqlen/balanced_min:22877 - global_seqlen/balanced_max:22879 - global_seqlen/mean:22878.5 - actor/entropy:0.041066378355026245 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417160868644714 - training/rollout_probs_diff_mean:0.0017004754627123475 - training/rollout_probs_diff_std:0.0106857530772686 - training/rollout_actor_probs_pearson_corr:0.9951813220977783 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.78515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999306201934814 - rollout_corr/rollout_is_ratio_fraction_high:6.025548282195814e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.640877397032455e-05 - rollout_corr/rollout_is_max:4.695746421813965 - rollout_corr/rollout_is_min:0.13378670811653137 - rollout_corr/rollout_is_std:0.027188541367650032 - rollout_corr/rollout_is_eff_sample_size:0.9992613292333393 - rollout_corr/rollout_is_seq_mean:0.9999163746833801 - rollout_corr/rollout_is_seq_std:0.00201141694560647 - rollout_corr/rollout_is_seq_max:1.0119117498397827 - rollout_corr/rollout_is_seq_min:0.9936652779579163 - rollout_corr/rollout_is_seq_max_deviation:0.011911749839782715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0394931584596634) - rollout_corr/training_log_ppl:np.float64(0.03842429356336652) - rollout_corr/kl:np.float64(0.00044337915408476647) - rollout_corr/k3_kl:np.float64(0.0005245370683890638) - rollout_corr/rollout_ppl:np.float64(1.0390328136272728) - rollout_corr/rollout_log_ppl:np.float64(0.03798091409771587) - rollout_corr/log_ppl_diff:np.float64(0.00044337946565065067) - rollout_corr/log_ppl_abs_diff:np.float64(0.001516479414931382) - rollout_corr/log_ppl_diff_max:np.float64(0.00839998945593834) - rollout_corr/log_ppl_diff_min:np.float64(-0.007669523358345032) - rollout_corr/ppl_ratio:np.float64(1.000445791753009) - rollout_corr/chi2_token:np.float64(0.0013469457626342773) - rollout_corr/chi2_seq:np.float64(0.7061824975535274) - actor/pg_loss:np.float64(0.00031901232432574034) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.6846769439289346e-05) - actor/ppo_kl:np.float64(-8.79328427032533e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78515625) - self_distillation/token_reweight_w_mean:np.float64(78056.61540558911) - self_distillation/token_reweight_w_std:np.float64(985978.0488043064) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999284080695361) - self_distillation/token_reweight_w_clip_frac:np.float64(0.003921596973668784) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.06964961811900139) - perf/mfu/actor:np.float64(0.04247672842872548) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.3838882446289) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:2 - critic/score/mean:0.78515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0023409256245940924 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0023409256245940924 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:324.140625 - response_length/max:611.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.140625 - response_length_non_aborted/max:611.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.8125 - prompt_length/max:482.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013821013271808624 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9531577304005623) - timing_s/agent_loop/generate_sequences/max:np.float64(5.360064445063472) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.3088946127318195) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.360064445063472) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:597 - timing_s/gen:11.14386217109859 - timing_s/reward:0.059097979217767715 - timing_s/old_log_prob:2.4689029324799776 - timing_s/adv:0.548030748963356 - timing_s/update_actor:15.49778139963746 - timing_s/step:29.82078117504716 - timing_s/stop_profile:0.00012038648128509521 - timing_per_token_ms/gen:0.1342957600759049 - timing_per_token_ms/adv:0.0029942454103380686 - timing_per_token_ms/update_actor:0.08467437441067739 - perf/total_num_tokens:183028 - perf/time_per_step:29.82078117504716 - perf/throughput:767.1998887522041 (TaskRunner pid=2167046) Training Progress: 71%|███████ | 71/100 [42:27<18:56, 39.19s/it] (TaskRunner pid=2167046) step:72 - global_seqlen/min:21927 - global_seqlen/max:24576 - global_seqlen/minmax_diff:2649 - global_seqlen/balanced_min:22883 - global_seqlen/balanced_max:22884 - global_seqlen/mean:22883.375 - actor/entropy:0.03342093899846077 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5599771738052368 - training/rollout_probs_diff_mean:0.0015238827327266335 - training/rollout_probs_diff_std:0.010387895628809929 - training/rollout_actor_probs_pearson_corr:0.9947546124458313 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.7890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.0000593662261963 - rollout_corr/rollout_is_ratio_fraction_high:6.152558489702642e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00019688188331201673 - rollout_corr/rollout_is_max:2.7454488277435303 - rollout_corr/rollout_is_min:0.009296300821006298 - rollout_corr/rollout_is_std:0.027067698538303375 - rollout_corr/rollout_is_eff_sample_size:0.9992679951393448 - rollout_corr/rollout_is_seq_mean:1.0000627040863037 - rollout_corr/rollout_is_seq_std:0.0017418585484847426 - rollout_corr/rollout_is_seq_max:1.0082443952560425 - rollout_corr/rollout_is_seq_min:0.9941825270652771 - rollout_corr/rollout_is_seq_max_deviation:0.00824439525604248 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0355365513823926) - rollout_corr/training_log_ppl:np.float64(0.03459935712726292) - rollout_corr/kl:np.float64(0.0005253947009382642) - rollout_corr/k3_kl:np.float64(0.0011788816827351667) - rollout_corr/rollout_ppl:np.float64(1.0349831562489271) - rollout_corr/rollout_log_ppl:np.float64(0.03407396220063674) - rollout_corr/log_ppl_diff:np.float64(0.000525394926626177) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018047186922558467) - rollout_corr/log_ppl_diff_max:np.float64(0.01845739781856537) - rollout_corr/log_ppl_diff_min:np.float64(-0.006578452885150909) - rollout_corr/ppl_ratio:np.float64(1.000529364682734) - rollout_corr/chi2_token:np.float64(0.008092012256383896) - rollout_corr/chi2_seq:np.float64(0.6922405788209289) - actor/pg_loss:np.float64(0.0009021894074976444) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004105189245819929) - actor/ppo_kl:np.float64(-7.494122602480502e-06) - actor/pg_clipfrac_lower:np.float64(1.333191084995633e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7890625) - self_distillation/token_reweight_w_mean:np.float64(197008.8856558064) - self_distillation/token_reweight_w_std:np.float64(2194593.5106205577) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999858915572986) - self_distillation/token_reweight_w_clip_frac:np.float64(0.009888861561194062) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.37002852745354176) - perf/mfu/actor:np.float64(0.04237542423474614) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.3786392211914) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:2 - critic/score/mean:0.7890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017073351191356778 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0017073351191356778 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:317.44921875 - response_length/max:657.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:317.44921875 - response_length_non_aborted/max:657.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:397.65625 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001435931771993637 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4516130425035954) - timing_s/agent_loop/generate_sequences/max:np.float64(5.389261782169342) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.509800940846617) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.389261782169342) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:657 - timing_s/gen:11.444258995354176 - timing_s/reward:0.05996984802186489 - timing_s/old_log_prob:2.266350820660591 - timing_s/adv:0.5192318316549063 - timing_s/update_actor:15.472687847912312 - timing_s/step:29.851040318608284 - timing_s/stop_profile:0.00011741556227207184 - timing_per_token_ms/gen:0.14082295390938726 - timing_per_token_ms/adv:0.0028362939888396392 - timing_per_token_ms/update_actor:0.08451926260829265 - perf/total_num_tokens:183067 - perf/time_per_step:29.851040318608284 - perf/throughput:766.5855111165141 (TaskRunner pid=2167046) Training Progress: 72%|███████▏ | 72/100 [42:57<16:59, 36.40s/it] (TaskRunner pid=2167046) step:73 - global_seqlen/min:18163 - global_seqlen/max:25690 - global_seqlen/minmax_diff:7527 - global_seqlen/balanced_min:21913 - global_seqlen/balanced_max:21917 - global_seqlen/mean:21916.0 - actor/entropy:0.034132421016693115 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351723432540894 - training/rollout_probs_diff_mean:0.0017482901457697153 - training/rollout_probs_diff_std:0.011693055741488934 - training/rollout_actor_probs_pearson_corr:0.9933797121047974 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.000124454498291 - rollout_corr/rollout_is_ratio_fraction_high:9.308510925620794e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3297872101247776e-05 - rollout_corr/rollout_is_max:4.1709747314453125 - rollout_corr/rollout_is_min:0.43125781416893005 - rollout_corr/rollout_is_std:0.029188847169280052 - rollout_corr/rollout_is_eff_sample_size:0.9991488554809967 - rollout_corr/rollout_is_seq_mean:1.0000839233398438 - rollout_corr/rollout_is_seq_std:0.0020293211564421654 - rollout_corr/rollout_is_seq_max:1.0085396766662598 - rollout_corr/rollout_is_seq_min:0.9942524433135986 - rollout_corr/rollout_is_seq_max_deviation:0.008539676666259766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0363794229924679) - rollout_corr/training_log_ppl:np.float64(0.03516461659455672) - rollout_corr/kl:np.float64(0.0003281725150534598) - rollout_corr/k3_kl:np.float64(0.000549684357956437) - rollout_corr/rollout_ppl:np.float64(1.0360313290730119) - rollout_corr/rollout_log_ppl:np.float64(0.03483644372136041) - rollout_corr/log_ppl_diff:np.float64(0.0003281728731963085) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015313210878957761) - rollout_corr/log_ppl_diff_max:np.float64(0.010771263390779495) - rollout_corr/log_ppl_diff_min:np.float64(-0.006783545948565006) - rollout_corr/ppl_ratio:np.float64(1.0003303056582808) - rollout_corr/chi2_token:np.float64(0.0016802684403955936) - rollout_corr/chi2_seq:np.float64(0.5110847533214837) - actor/pg_loss:np.float64(0.00020943302661180496) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.329807704081759e-05) - actor/ppo_kl:np.float64(-7.272707647376642e-05) - actor/pg_clipfrac_lower:np.float64(1.268262985831825e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(30931.851405258058) - self_distillation/token_reweight_w_std:np.float64(314383.1071442036) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003154934383929) - self_distillation/token_reweight_w_clip_frac:np.float64(0.002947135504655307) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.05511173512786627) - perf/mfu/actor:np.float64(0.04059150667566496) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.59348678588867) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:2 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00014461435785051435 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00014461435785051435 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.75 - response_length/max:586.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.75 - response_length_non_aborted/max:586.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.125 - prompt_length/max:494.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001238752156496048 - timing_s/agent_loop/generate_sequences/min:np.float64(1.461754359304905) - timing_s/agent_loop/generate_sequences/max:np.float64(5.0036635641008615) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.123489771867753) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.0036635641008615) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:570 - timing_s/gen:10.982410997152328 - timing_s/reward:0.05244344286620617 - timing_s/old_log_prob:2.4749830458313227 - timing_s/adv:0.5284104067832232 - timing_s/update_actor:15.08427819609642 - timing_s/step:29.220373958349228 - timing_s/stop_profile:0.00011274777352809906 - timing_per_token_ms/gen:0.14604269943021714 - timing_per_token_ms/adv:0.003013839242923111 - timing_per_token_ms/update_actor:0.08603462194342273 - perf/total_num_tokens:175328 - perf/time_per_step:29.220373958349228 - perf/throughput:750.0246242994393 (TaskRunner pid=2167046) Training Progress: 73%|███████▎ | 73/100 [43:26<15:25, 34.26s/it] (TaskRunner pid=2167046) step:74 - global_seqlen/min:18699 - global_seqlen/max:24638 - global_seqlen/minmax_diff:5939 - global_seqlen/balanced_min:22666 - global_seqlen/balanced_max:22668 - global_seqlen/mean:22667.375 - actor/entropy:0.04269804805517197 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4211313724517822 - training/rollout_probs_diff_mean:0.0018847070168703794 - training/rollout_probs_diff_std:0.010922428220510483 - training/rollout_actor_probs_pearson_corr:0.9953285455703735 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.84375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0002391338348389 - rollout_corr/rollout_is_ratio_fraction_high:6.0671511164400727e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.707442222861573e-05 - rollout_corr/rollout_is_max:5.670766830444336 - rollout_corr/rollout_is_min:0.28868961334228516 - rollout_corr/rollout_is_std:0.029143888503313065 - rollout_corr/rollout_is_eff_sample_size:0.9991514736296966 - rollout_corr/rollout_is_seq_mean:1.0002909898757935 - rollout_corr/rollout_is_seq_std:0.0020770577248185873 - rollout_corr/rollout_is_seq_max:1.013790249824524 - rollout_corr/rollout_is_seq_min:0.9944962859153748 - rollout_corr/rollout_is_seq_max_deviation:0.013790249824523926 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0447109127417207) - rollout_corr/training_log_ppl:np.float64(0.04343737147610227) - rollout_corr/kl:np.float64(0.000363680085879281) - rollout_corr/k3_kl:np.float64(0.0005548008483327038) - rollout_corr/rollout_ppl:np.float64(1.044323279056698) - rollout_corr/rollout_log_ppl:np.float64(0.043073691116660484) - rollout_corr/log_ppl_diff:np.float64(0.000363680359441787) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015776339314470533) - rollout_corr/log_ppl_diff_max:np.float64(0.0066324397921562195) - rollout_corr/log_ppl_diff_min:np.float64(-0.008656047284603119) - rollout_corr/ppl_ratio:np.float64(1.0003659820649773) - rollout_corr/chi2_token:np.float64(0.0016228372696787119) - rollout_corr/chi2_seq:np.float64(0.7596687572076917) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.00016098882441184514) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_w_mean:np.float64(0.999999993480742) - self_distillation/token_reweight_w_std:np.float64(0.0) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999999993480742) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.04193188049080382) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.48727798461914) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:2 - critic/score/mean:0.84375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:321.91796875 - response_length/max:589.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:321.91796875 - response_length_non_aborted/max:589.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:386.4375 - prompt_length/max:481.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017289817333221436 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7890728898346424) - timing_s/agent_loop/generate_sequences/max:np.float64(5.667955011129379) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6804275711692753) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.667955011129379) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:589 - timing_s/gen:11.574067967012525 - timing_s/reward:0.06021520681679249 - timing_s/old_log_prob:2.247141635045409 - timing_s/adv:0.535718709230423 - timing_s/update_actor:15.435283839702606 - timing_s/step:29.93964694440365 - timing_s/stop_profile:0.00011217407882213593 - timing_per_token_ms/gen:0.1404432413999651 - timing_per_token_ms/adv:0.0029542387971171285 - timing_per_token_ms/update_actor:0.08511839063688785 - perf/total_num_tokens:181339 - perf/time_per_step:29.93964694440365 - perf/throughput:757.1022812023176 (TaskRunner pid=2167046) Training Progress: 74%|███████▍ | 74/100 [43:56<14:17, 32.98s/it] (TaskRunner pid=2167046) step:75 - global_seqlen/min:20181 - global_seqlen/max:24422 - global_seqlen/minmax_diff:4241 - global_seqlen/balanced_min:22420 - global_seqlen/balanced_max:22422 - global_seqlen/mean:22421.0 - actor/entropy:0.0377618744969368 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7632365226745605 - training/rollout_probs_diff_mean:0.0016583539545536041 - training/rollout_probs_diff_std:0.011271195486187935 - training/rollout_actor_probs_pearson_corr:0.9941364526748657 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.66015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9999065399169922 - rollout_corr/rollout_is_ratio_fraction_high:5.097359462524764e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015292078023776412 - rollout_corr/rollout_is_max:3.3628957271575928 - rollout_corr/rollout_is_min:0.10317613184452057 - rollout_corr/rollout_is_std:0.026788802817463875 - rollout_corr/rollout_is_eff_sample_size:0.9992826366379459 - rollout_corr/rollout_is_seq_mean:0.9999075531959534 - rollout_corr/rollout_is_seq_std:0.0017399507341906428 - rollout_corr/rollout_is_seq_max:1.0067952871322632 - rollout_corr/rollout_is_seq_min:0.9949383735656738 - rollout_corr/rollout_is_seq_max_deviation:0.006795287132263184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0384952994063497) - rollout_corr/training_log_ppl:np.float64(0.03707945561973247) - rollout_corr/kl:np.float64(0.0004538998949357165) - rollout_corr/k3_kl:np.float64(0.00041609512296765516) - rollout_corr/rollout_ppl:np.float64(1.038021398242563) - rollout_corr/rollout_log_ppl:np.float64(0.03662555549408353) - rollout_corr/log_ppl_diff:np.float64(0.0004539001256489428) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012520181262516417) - rollout_corr/log_ppl_diff_max:np.float64(0.006798019632697105) - rollout_corr/log_ppl_diff_min:np.float64(-0.00393272191286087) - rollout_corr/ppl_ratio:np.float64(1.0004553578328341) - rollout_corr/chi2_token:np.float64(0.0007796948775649071) - rollout_corr/chi2_seq:np.float64(0.202988384058699) - actor/pg_loss:np.float64(8.659297600388527e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.3403119232389145e-05) - actor/ppo_kl:np.float64(-7.427042106478154e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66015625) - self_distillation/token_reweight_w_mean:np.float64(65656.72851561918) - self_distillation/token_reweight_w_std:np.float64(913989.140625) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001852267887443) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0024617357703391463) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.07150154281407595) - perf/mfu/actor:np.float64(0.04083979931261415) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.35407638549805) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:2 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00022141655790619552 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:-0.00022141655790619552 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:306.53125 - response_length/max:618.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:306.53125 - response_length_non_aborted/max:618.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.125 - prompt_length/max:493.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013413839042186737 - timing_s/agent_loop/generate_sequences/min:np.float64(1.786161419004202) - timing_s/agent_loop/generate_sequences/max:np.float64(5.231584556400776) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4409076798401657) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.231584556400776) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:618 - timing_s/gen:11.410260079428554 - timing_s/reward:0.05926189012825489 - timing_s/old_log_prob:2.2375035118311644 - timing_s/adv:0.5120990797877312 - timing_s/update_actor:15.626816725358367 - timing_s/step:29.932239281013608 - timing_s/stop_profile:0.00010907649993896484 - timing_per_token_ms/gen:0.14540549596580377 - timing_per_token_ms/adv:0.0028550191772653494 - timing_per_token_ms/update_actor:0.08712154188795307 - perf/total_num_tokens:179368 - perf/time_per_step:29.932239281013608 - perf/throughput:749.0585582155866 (TaskRunner pid=2167046) Training Progress: 75%|███████▌ | 75/100 [44:26<13:22, 32.08s/it] (TaskRunner pid=2167046) step:76 - global_seqlen/min:17637 - global_seqlen/max:23218 - global_seqlen/minmax_diff:5581 - global_seqlen/balanced_min:20738 - global_seqlen/balanced_max:20744 - global_seqlen/mean:20742.5 - actor/entropy:0.04465673118829727 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29321733117103577 - training/rollout_probs_diff_mean:0.0018123796908184886 - training/rollout_probs_diff_std:0.010481405071914196 - training/rollout_actor_probs_pearson_corr:0.995635449886322 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.0001299381256104 - rollout_corr/rollout_is_ratio_fraction_high:1.4342263966682367e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4342263966682367e-05 - rollout_corr/rollout_is_max:2.523247241973877 - rollout_corr/rollout_is_min:0.47835788130760193 - rollout_corr/rollout_is_std:0.027491578832268715 - rollout_corr/rollout_is_eff_sample_size:0.9992450219453466 - rollout_corr/rollout_is_seq_mean:1.0001345872879028 - rollout_corr/rollout_is_seq_std:0.0018495450494810939 - rollout_corr/rollout_is_seq_max:1.0072492361068726 - rollout_corr/rollout_is_seq_min:0.9933908581733704 - rollout_corr/rollout_is_seq_max_deviation:0.007249236106872559 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.043047591112554) - rollout_corr/training_log_ppl:np.float64(0.04124056530963571) - rollout_corr/kl:np.float64(0.0005134855972919183) - rollout_corr/k3_kl:np.float64(0.000566033174557834) - rollout_corr/rollout_ppl:np.float64(1.0424995855428278) - rollout_corr/rollout_log_ppl:np.float64(0.04072707948353127) - rollout_corr/log_ppl_diff:np.float64(0.0005134858261044428) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018579333004709042) - rollout_corr/log_ppl_diff_max:np.float64(0.0113980732858181) - rollout_corr/log_ppl_diff_min:np.float64(-0.010368887335062027) - rollout_corr/ppl_ratio:np.float64(1.0005171394441277) - rollout_corr/chi2_token:np.float64(0.0013373454567044973) - rollout_corr/chi2_seq:np.float64(0.5531120426021516) - actor/pg_loss:np.float64(4.2089493945240974e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014904459294484695) - actor/ppo_kl:np.float64(0.00027603819385291217) - actor/pg_clipfrac_lower:np.float64(1.641281596675981e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(104931.70410155761) - self_distillation/token_reweight_w_std:np.float64(1154588.890625) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000077618053183) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0010802401811815798) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.06183192692697048) - perf/mfu/actor:np.float64(0.038350335680134795) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.40346145629883) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:2 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000932247145101428 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:0.000932247145101428 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:272.359375 - response_length/max:591.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.359375 - response_length_non_aborted/max:591.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:375.84375 - prompt_length/max:471.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.789317846298218e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2649769186973572) - timing_s/agent_loop/generate_sequences/max:np.float64(4.929914198815823) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.903962667020096) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.929914198815823) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:278 - timing_s/agent_loop/slowest/response_length:587 - timing_s/gen:10.682688023895025 - timing_s/reward:0.058021435514092445 - timing_s/old_log_prob:2.286804085597396 - timing_s/adv:0.5014898627996445 - timing_s/update_actor:15.547459477558732 - timing_s/step:29.164036449044943 - timing_s/stop_profile:0.0001047961413860321 - timing_per_token_ms/gen:0.15321392954929472 - timing_per_token_ms/adv:0.0030221156008174307 - timing_per_token_ms/update_actor:0.09369325947667068 - perf/total_num_tokens:165940 - perf/time_per_step:29.164036449044943 - perf/throughput:711.2355670053097 (TaskRunner pid=2167046) Training Progress: 76%|███████▌ | 76/100 [44:55<12:29, 31.22s/it] (TaskRunner pid=2167046) step:77 - global_seqlen/min:18473 - global_seqlen/max:24783 - global_seqlen/minmax_diff:6310 - global_seqlen/balanced_min:22922 - global_seqlen/balanced_max:22924 - global_seqlen/mean:22923.0 - actor/entropy:0.029721224680542946 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7918662428855896 - training/rollout_probs_diff_mean:0.001373450388200581 - training/rollout_probs_diff_std:0.01168912835419178 - training/rollout_actor_probs_pearson_corr:0.9923051595687866 - self_distillation/success_group_fraction:0.5 - self_distillation/success_sample_fraction:0.5 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5 - rollout_corr/rollout_is_mean:0.9999530911445618 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00025326714967377484 - rollout_corr/rollout_is_max:1.9884089231491089 - rollout_corr/rollout_is_min:0.15751142799854279 - rollout_corr/rollout_is_std:0.02383342757821083 - rollout_corr/rollout_is_eff_sample_size:0.9994322902127667 - rollout_corr/rollout_is_seq_mean:0.9998948574066162 - rollout_corr/rollout_is_seq_std:0.001541188801638782 - rollout_corr/rollout_is_seq_max:1.0081614255905151 - rollout_corr/rollout_is_seq_min:0.9954944849014282 - rollout_corr/rollout_is_seq_max_deviation:0.008161425590515137 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.028552311938256) - rollout_corr/training_log_ppl:np.float64(0.02791491772450172) - rollout_corr/kl:np.float64(0.0004828657112110335) - rollout_corr/k3_kl:np.float64(0.0004112039243029386) - rollout_corr/rollout_ppl:np.float64(1.028052968904376) - rollout_corr/rollout_log_ppl:np.float64(0.027432051831056015) - rollout_corr/log_ppl_diff:np.float64(0.0004828658934457053) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012247920726622397) - rollout_corr/log_ppl_diff_max:np.float64(0.006324823945760727) - rollout_corr/log_ppl_diff_min:np.float64(-0.0035044103860855103) - rollout_corr/ppl_ratio:np.float64(1.000484426273033) - rollout_corr/chi2_token:np.float64(0.0005913209170103073) - rollout_corr/chi2_seq:np.float64(0.18623502389527857) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(-2.6408372267638924e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(0.9999999969732016) - self_distillation/token_reweight_w_std:np.float64(0.0) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999999969732016) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0) - self_distillation/empty_target_batch:np.float64(0.5) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.042214026092317475) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.41495895385742) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:2 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:308.46875 - response_length/max:617.0 - response_length/min:144.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.46875 - response_length_non_aborted/max:617.0 - response_length_non_aborted/min:144.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.875 - prompt_length/max:510.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017272494733333588 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7466902062296867) - timing_s/agent_loop/generate_sequences/max:np.float64(5.335060682147741) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4707724340623827) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.335060682147741) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:617 - timing_s/gen:11.125146919861436 - timing_s/reward:0.05995953641831875 - timing_s/old_log_prob:2.2617794070392847 - timing_s/adv:0.49137057177722454 - timing_s/update_actor:15.607751056551933 - timing_s/step:29.633765883743763 - timing_s/stop_profile:0.00011434778571128845 - timing_per_token_ms/gen:0.14088171056455065 - timing_per_token_ms/adv:0.0026794626127537003 - timing_per_token_ms/update_actor:0.08510966636430622 - perf/total_num_tokens:183384 - perf/time_per_step:29.633765883743763 - perf/throughput:773.5432644615345 (TaskRunner pid=2167046) Training Progress: 77%|███████▋ | 77/100 [45:25<11:47, 30.76s/it] (TaskRunner pid=2167046) step:78 - global_seqlen/min:21618 - global_seqlen/max:23393 - global_seqlen/minmax_diff:1775 - global_seqlen/balanced_min:22640 - global_seqlen/balanced_max:22641 - global_seqlen/mean:22640.375 - actor/entropy:0.035092517733573914 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351723432540894 - training/rollout_probs_diff_mean:0.0017425608821213245 - training/rollout_probs_diff_std:0.011111678555607796 - training/rollout_actor_probs_pearson_corr:0.994017481803894 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.6328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:0.9998819828033447 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.965578798670322e-05 - rollout_corr/rollout_is_max:1.9659780263900757 - rollout_corr/rollout_is_min:0.4614439904689789 - rollout_corr/rollout_is_std:0.028242332860827446 - rollout_corr/rollout_is_eff_sample_size:0.9992027683105108 - rollout_corr/rollout_is_seq_mean:0.9999055862426758 - rollout_corr/rollout_is_seq_std:0.0017442753305658698 - rollout_corr/rollout_is_seq_max:1.0089272260665894 - rollout_corr/rollout_is_seq_min:0.994486927986145 - rollout_corr/rollout_is_seq_max_deviation:0.008927226066589355 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0356996422633529) - rollout_corr/training_log_ppl:np.float64(0.0347957349367789) - rollout_corr/kl:np.float64(0.0006287374900068726) - rollout_corr/k3_kl:np.float64(0.00041870254129960927) - rollout_corr/rollout_ppl:np.float64(1.0350423878990114) - rollout_corr/rollout_log_ppl:np.float64(0.03416699716126459) - rollout_corr/log_ppl_diff:np.float64(0.0006287377755143098) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013772529555353685) - rollout_corr/log_ppl_diff_max:np.float64(0.009740196168422699) - rollout_corr/log_ppl_diff_min:np.float64(-0.006851349025964737) - rollout_corr/ppl_ratio:np.float64(1.0006305843126029) - rollout_corr/chi2_token:np.float64(0.0004125772975385189) - rollout_corr/chi2_seq:np.float64(0.027380227576941252) - actor/pg_loss:np.float64(0.0002134254900738597) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001747056612657616) - actor/ppo_kl:np.float64(0.00013020554026610398) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_w_mean:np.float64(73278.90783314756) - self_distillation/token_reweight_w_std:np.float64(810226.5895272555) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998990218155086) - self_distillation/token_reweight_w_clip_frac:np.float64(0.003775067198148463) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.09012467134743929) - perf/mfu/actor:np.float64(0.04218577321591646) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.53057479858398) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010013086721301079 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010013086721301079 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:295.51171875 - response_length/max:569.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:295.51171875 - response_length_non_aborted/max:569.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.0 - prompt_length/max:493.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017403066158294678 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0237001329660416) - timing_s/agent_loop/generate_sequences/max:np.float64(5.066724296659231) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.6675877660454717) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.066724296659231) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:271 - timing_s/agent_loop/slowest/response_length:555 - timing_s/gen:11.020616475492716 - timing_s/reward:0.07347750663757324 - timing_s/old_log_prob:2.33992669172585 - timing_s/adv:0.5275673251599073 - timing_s/update_actor:15.446424208581448 - timing_s/step:29.495602479204535 - timing_s/stop_profile:0.00011941604316234589 - timing_per_token_ms/gen:0.14567707598700236 - timing_per_token_ms/adv:0.002912757215593311 - timing_per_token_ms/update_actor:0.08528140660535354 - perf/total_num_tokens:181123 - perf/time_per_step:29.495602479204535 - perf/throughput:767.5847616932823 (TaskRunner pid=2167046) Training Progress: 78%|███████▊ | 78/100 [45:54<11:08, 30.40s/it] (TaskRunner pid=2167046) step:79 - global_seqlen/min:19064 - global_seqlen/max:24752 - global_seqlen/minmax_diff:5688 - global_seqlen/balanced_min:22609 - global_seqlen/balanced_max:22611 - global_seqlen/mean:22609.875 - actor/entropy:0.03175250440835953 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7543225288391113 - training/rollout_probs_diff_mean:0.0016824816120788455 - training/rollout_probs_diff_std:0.01265895739197731 - training/rollout_actor_probs_pearson_corr:0.9913443922996521 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.80078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.0001208782196045 - rollout_corr/rollout_is_ratio_fraction_high:7.674302469240502e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015348604938481003 - rollout_corr/rollout_is_max:4.387160778045654 - rollout_corr/rollout_is_min:0.029560457915067673 - rollout_corr/rollout_is_std:0.02866755798459053 - rollout_corr/rollout_is_eff_sample_size:0.9991788460058731 - rollout_corr/rollout_is_seq_mean:1.0002002716064453 - rollout_corr/rollout_is_seq_std:0.00217239186167717 - rollout_corr/rollout_is_seq_max:1.0119655132293701 - rollout_corr/rollout_is_seq_min:0.9956554174423218 - rollout_corr/rollout_is_seq_max_deviation:0.011965513229370117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.033602093346417) - rollout_corr/training_log_ppl:np.float64(0.03276922370059765) - rollout_corr/kl:np.float64(0.0006594466274378874) - rollout_corr/k3_kl:np.float64(0.000879152259249949) - rollout_corr/rollout_ppl:np.float64(1.0329237426631153) - rollout_corr/rollout_log_ppl:np.float64(0.03210977676053517) - rollout_corr/log_ppl_diff:np.float64(0.0006594469382434909) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018515425176701683) - rollout_corr/log_ppl_diff_max:np.float64(0.02449878305196762) - rollout_corr/log_ppl_diff_min:np.float64(-0.008788060396909714) - rollout_corr/ppl_ratio:np.float64(1.0006653296295553) - rollout_corr/chi2_token:np.float64(0.001893435139209032) - rollout_corr/chi2_seq:np.float64(0.5223376241046935) - actor/pg_loss:np.float64(0.0001657054526731372) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.394158183160471e-05) - actor/ppo_kl:np.float64(0.00019161261689504272) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_w_mean:np.float64(32477.99980539549) - self_distillation/token_reweight_w_std:np.float64(403046.4446096127) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000050456728786) - self_distillation/token_reweight_w_clip_frac:np.float64(0.002672293430805439) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.09365954529494047) - perf/mfu/actor:np.float64(0.04209851712862714) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.6111183166504) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.80078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.80078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006427228217944503 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006427228217944503 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:305.40234375 - response_length/max:569.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.40234375 - response_length_non_aborted/max:569.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:401.15625 - prompt_length/max:494.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002107582986354828 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7130346596240997) - timing_s/agent_loop/generate_sequences/max:np.float64(5.07829512655735) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.373994250316173) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.07829512655735) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:569 - timing_s/gen:10.9527677744627 - timing_s/reward:0.05182774364948273 - timing_s/old_log_prob:2.3623594883829355 - timing_s/adv:0.5516835059970617 - timing_s/update_actor:15.409146958962083 - timing_s/step:29.38063520565629 - timing_s/stop_profile:2.9552727937698364e-05 - timing_per_token_ms/gen:0.14009142364021207 - timing_per_token_ms/adv:0.003050014130977403 - timing_per_token_ms/update_actor:0.08519035907408866 - perf/total_num_tokens:180879 - perf/time_per_step:29.38063520565629 - perf/throughput:769.5502442931255 (TaskRunner pid=2167046) Training Progress: 79%|███████▉ | 79/100 [46:24<10:34, 30.24s/it] (TaskRunner pid=2167046) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2167046) validation generation end (TaskRunner pid=2167046) [prompt] system (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume $ V $ of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c \cdot \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given that all angles $ \alpha, \beta, \gamma $ are 90.00º, the cosine of each angle is 0. Substituting these values into the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c \cdot \sqrt{1 + 2(0)(0)(0) - (0)^2 - (0)^2 - (0)^2} = a \cdot b \cdot c \cdot \sqrt{1} = a \cdot b \cdot c (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Now, substituting the given lattice parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = 7.82 \cdot 7.82 \cdot 17.88 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate $ 7.82 \cdot 7.82 $: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 7.82 \cdot 7.82 = 61.1524 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then multiply by 17.88: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 61.1524 \cdot 17.88 \approx 1094.31 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2167440) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2167046) step:80 - global_seqlen/min:19660 - global_seqlen/max:25315 - global_seqlen/minmax_diff:5655 - global_seqlen/balanced_min:22802 - global_seqlen/balanced_max:22805 - global_seqlen/mean:22803.375 - actor/entropy:0.050714973360300064 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6335345506668091 - training/rollout_probs_diff_mean:0.001896037021651864 - training/rollout_probs_diff_std:0.011774555779993534 - training/rollout_actor_probs_pearson_corr:0.9953871369361877 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.000020146369934 - rollout_corr/rollout_is_ratio_fraction_high:9.482830500928685e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002607778296805918 - rollout_corr/rollout_is_max:5.18784761428833 - rollout_corr/rollout_is_min:0.19069860875606537 - rollout_corr/rollout_is_std:0.03135092556476593 - rollout_corr/rollout_is_eff_sample_size:0.9990177275875637 - rollout_corr/rollout_is_seq_mean:1.0000741481781006 - rollout_corr/rollout_is_seq_std:0.0026087467558681965 - rollout_corr/rollout_is_seq_max:1.0172052383422852 - rollout_corr/rollout_is_seq_min:0.9944260120391846 - rollout_corr/rollout_is_seq_max_deviation:0.017205238342285156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.048643831629306) - rollout_corr/training_log_ppl:np.float64(0.046666585320053855) - rollout_corr/kl:np.float64(0.0012581646745264408) - rollout_corr/k3_kl:np.float64(0.0014205996778304808) - rollout_corr/rollout_ppl:np.float64(1.0472968486137688) - rollout_corr/rollout_log_ppl:np.float64(0.04540842036749382) - rollout_corr/log_ppl_diff:np.float64(0.001258164952560037) - rollout_corr/log_ppl_abs_diff:np.float64(0.002346423177186807) - rollout_corr/log_ppl_diff_max:np.float64(0.030772797763347626) - rollout_corr/log_ppl_diff_min:np.float64(-0.010465197265148163) - rollout_corr/ppl_ratio:np.float64(1.0012677058111876) - rollout_corr/chi2_token:np.float64(0.0037672733888030052) - rollout_corr/chi2_seq:np.float64(0.9763107842300087) - actor/pg_loss:np.float64(0.000655685318633914) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002488995287421858) - actor/ppo_kl:np.float64(0.0006420044184398677) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(110585.75917376974) - self_distillation/token_reweight_w_std:np.float64(1607578.4135247644) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001894689630717) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011867681285366416) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.17190873017534614) - perf/mfu/actor:np.float64(0.041936679075923314) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(146.61267471313477) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7579787234042553) - val-aux/sciknoweval/reward/std@16:np.float64(0.020546155153965868) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7650425531914894) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.011835424325934357) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7508829787234043) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.02186188398559465) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7582872340425532) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.020059577602687874) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7680425531914893) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.005994382106336695) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7418936170212765) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.023130927252613374) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7609148936170212) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.013964251181075955) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7701489361702128) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.005536783555435272) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7317446808510639) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.02167952980531925) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.763127659574468) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.007406332487350029) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7728829787234042) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.005070786835475184) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.7220425531914894) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.015633326854271365) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7643085106382979) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.004304937399897404) - val-aux/sciknoweval/score/mean@16:np.float64(0.7579787234042553) - val-aux/sciknoweval/score/std@16:np.float64(0.020546155153965868) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7650425531914894) - val-aux/sciknoweval/score/best@2/std:np.float64(0.011835424325934357) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7508829787234043) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.02186188398559465) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7582872340425532) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.020059577602687874) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7680425531914893) - val-aux/sciknoweval/score/best@4/std:np.float64(0.005994382106336695) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7418936170212765) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.023130927252613374) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7609148936170212) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.013964251181075955) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7701489361702128) - val-aux/sciknoweval/score/best@8/std:np.float64(0.005536783555435272) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7317446808510639) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.02167952980531925) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.763127659574468) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.007406332487350029) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7728829787234042) - val-aux/sciknoweval/score/best@16/std:np.float64(0.005070786835475184) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.7220425531914894) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.015633326854271365) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7643085106382979) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.004304937399897404) - val-core/sciknoweval/acc/mean@16:np.float64(0.7579787234042553) - val-aux/sciknoweval/acc/std@16:np.float64(0.020546155153965868) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7650425531914894) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.011835424325934357) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7508829787234043) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.02186188398559465) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7582872340425532) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.020059577602687874) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7680425531914893) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.005994382106336695) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7418936170212765) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.023130927252613374) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7609148936170212) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.013964251181075955) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7701489361702128) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.005536783555435272) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7317446808510639) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.02167952980531925) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.763127659574468) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.007406332487350029) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7728829787234042) - val-core/sciknoweval/acc/best@16/std:np.float64(0.005070786835475184) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.7220425531914894) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.015633326854271365) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7643085106382979) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.004304937399897404) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007615897920913994 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007615897920913994 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:329.54296875 - response_length/max:765.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:329.54296875 - response_length_non_aborted/max:765.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:383.0625 - prompt_length/max:488.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.960782825946808e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5280603226274252) - timing_s/agent_loop/generate_sequences/max:np.float64(6.235287645831704) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.443311409551825) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.235287645831704) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:765 - timing_s/gen:12.49514858983457 - timing_s/reward:0.05852593667805195 - timing_s/old_log_prob:2.2642095126211643 - timing_s/adv:0.5320028979331255 - timing_s/update_actor:15.528907306492329 - timing_s/step:30.968016492202878 - timing_s/testing:37.4534579962492 - timing_s/save_checkpoint:6.927084492519498 - timing_s/stop_profile:0.00014102086424827576 - timing_per_token_ms/gen:0.14811171473080106 - timing_per_token_ms/adv:0.002916250872585338 - timing_per_token_ms/update_actor:0.08512395263032517 - perf/total_num_tokens:182427 - perf/time_per_step:30.968016492202878 - perf/throughput:736.3524559521411 (TaskRunner pid=2167046) Training Progress: 80%|████████ | 80/100 [47:40<14:35, 43.80s/it] (TaskRunner pid=2167046) step:81 - global_seqlen/min:18437 - global_seqlen/max:24732 - global_seqlen/minmax_diff:6295 - global_seqlen/balanced_min:20958 - global_seqlen/balanced_max:20960 - global_seqlen/mean:20959.375 - actor/entropy:0.03692462667822838 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.669526994228363 - training/rollout_probs_diff_mean:0.0017158862901851535 - training/rollout_probs_diff_std:0.011773472651839256 - training/rollout_actor_probs_pearson_corr:0.9934026002883911 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0001301765441895 - rollout_corr/rollout_is_ratio_fraction_high:0.00014152478252071887 - rollout_corr/rollout_is_ratio_fraction_low:0.0001981347013497725 - rollout_corr/rollout_is_max:3.121652126312256 - rollout_corr/rollout_is_min:0.06086057052016258 - rollout_corr/rollout_is_std:0.02928262948989868 - rollout_corr/rollout_is_eff_sample_size:0.9991435002195741 - rollout_corr/rollout_is_seq_mean:1.0000874996185303 - rollout_corr/rollout_is_seq_std:0.0021697808988392353 - rollout_corr/rollout_is_seq_max:1.0103951692581177 - rollout_corr/rollout_is_seq_min:0.9942268133163452 - rollout_corr/rollout_is_seq_max_deviation:0.010395169258117676 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.035950763616711) - rollout_corr/training_log_ppl:np.float64(0.03461706963435063) - rollout_corr/kl:np.float64(0.00124807827964446) - rollout_corr/k3_kl:np.float64(0.0011595660929395457) - rollout_corr/rollout_ppl:np.float64(1.0346353389322758) - rollout_corr/rollout_log_ppl:np.float64(0.03336899111445746) - rollout_corr/log_ppl_diff:np.float64(0.0012480785198931699) - rollout_corr/log_ppl_abs_diff:np.float64(0.002425113902972953) - rollout_corr/log_ppl_diff_max:np.float64(0.0498611256480217) - rollout_corr/log_ppl_diff_min:np.float64(-0.013390138745307922) - rollout_corr/ppl_ratio:np.float64(1.001260134857148) - rollout_corr/chi2_token:np.float64(0.0016280324198305607) - rollout_corr/chi2_seq:np.float64(0.5298847754020244) - actor/pg_loss:np.float64(0.0006970755057409406) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016883460511962767) - actor/ppo_kl:np.float64(0.0007997096730001374) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(92006.85765184229) - self_distillation/token_reweight_w_std:np.float64(980502.2056374671) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004657644312829) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011001636754372157) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.259071440435946) - perf/mfu/actor:np.float64(0.0382410973678092) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.5239028930664) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011233530240133405 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011233530240133405 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:276.01171875 - response_length/max:610.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:276.01171875 - response_length_non_aborted/max:610.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:378.96875 - prompt_length/max:524.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012356601655483246 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6116145700216293) - timing_s/agent_loop/generate_sequences/max:np.float64(5.111679133027792) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0472230411542114) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.111679133027792) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:189 - timing_s/agent_loop/slowest/response_length:610 - timing_s/gen:10.71186687424779 - timing_s/reward:0.05499126575887203 - timing_s/old_log_prob:2.601982841268182 - timing_s/adv:0.5419840794056654 - timing_s/update_actor:15.367881076410413 - timing_s/step:29.381832733750343 - timing_s/stop_profile:0.00011327676475048065 - timing_per_token_ms/gen:0.15159946891758713 - timing_per_token_ms/adv:0.003232348766397289 - timing_per_token_ms/update_actor:0.09165278709652848 - perf/total_num_tokens:167675 - perf/time_per_step:29.381832733750343 - perf/throughput:713.3447116770348 (TaskRunner pid=2167046) Training Progress: 81%|████████ | 81/100 [48:09<12:30, 39.49s/it] (TaskRunner pid=2167046) step:82 - global_seqlen/min:18511 - global_seqlen/max:24253 - global_seqlen/minmax_diff:5742 - global_seqlen/balanced_min:21238 - global_seqlen/balanced_max:21240 - global_seqlen/mean:21239.375 - actor/entropy:0.03823944181203842 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3175753355026245 - training/rollout_probs_diff_mean:0.0017360865604132414 - training/rollout_probs_diff_std:0.01078617200255394 - training/rollout_actor_probs_pearson_corr:0.9948719143867493 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.7109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000228881835938 - rollout_corr/rollout_is_ratio_fraction_high:5.631661042571068e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.447491563856602e-05 - rollout_corr/rollout_is_max:3.3646881580352783 - rollout_corr/rollout_is_min:0.23710303008556366 - rollout_corr/rollout_is_std:0.02898392453789711 - rollout_corr/rollout_is_eff_sample_size:0.9991605182489611 - rollout_corr/rollout_is_seq_mean:1.000075340270996 - rollout_corr/rollout_is_seq_std:0.0025008083321154118 - rollout_corr/rollout_is_seq_max:1.0169435739517212 - rollout_corr/rollout_is_seq_min:0.991999626159668 - rollout_corr/rollout_is_seq_max_deviation:0.01694357395172119 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0359556558541954) - rollout_corr/training_log_ppl:np.float64(0.034949058814163436) - rollout_corr/kl:np.float64(0.0005831834411189618) - rollout_corr/k3_kl:np.float64(0.0009131975491611932) - rollout_corr/rollout_ppl:np.float64(1.0353414616547525) - rollout_corr/rollout_log_ppl:np.float64(0.03436587487749421) - rollout_corr/log_ppl_diff:np.float64(0.0005831839366692293) - rollout_corr/log_ppl_abs_diff:np.float64(0.001916763685130718) - rollout_corr/log_ppl_diff_max:np.float64(0.013963993638753891) - rollout_corr/log_ppl_diff_min:np.float64(-0.015550876036286354) - rollout_corr/ppl_ratio:np.float64(1.0005876692011952) - rollout_corr/chi2_token:np.float64(0.004429039778187871) - rollout_corr/chi2_seq:np.float64(0.7082133176736534) - actor/pg_loss:np.float64(0.00022834050469100475) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011931026392630883) - actor/ppo_kl:np.float64(0.00015909244585188276) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_w_mean:np.float64(30193.831576111028) - self_distillation/token_reweight_w_std:np.float64(321280.0764630872) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998594501521438) - self_distillation/token_reweight_w_clip_frac:np.float64(0.003927729223505594) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.08594363182783127) - perf/mfu/actor:np.float64(0.03934670631693053) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.38236618041992) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0003238205099478364 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0003238205099478364 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:277.44921875 - response_length/max:743.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:277.44921875 - response_length_non_aborted/max:743.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:386.28125 - prompt_length/max:510.0 - prompt_length/min:266.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.356066584587097e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3338458444923162) - timing_s/agent_loop/generate_sequences/max:np.float64(5.63678365573287) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9356736195113626) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.63678365573287) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:698 - timing_s/gen:11.589976023882627 - timing_s/reward:0.05485660955309868 - timing_s/old_log_prob:2.29955842345953 - timing_s/adv:0.5150378718972206 - timing_s/update_actor:15.009910188615322 - timing_s/step:29.562939513474703 - timing_s/stop_profile:0.00011094287037849426 - timing_per_token_ms/gen:0.16317704568519897 - timing_per_token_ms/adv:0.003031150115629701 - timing_per_token_ms/update_actor:0.08833775822390796 - perf/total_num_tokens:169915 - perf/time_per_step:29.562939513474703 - perf/throughput:718.4459782938417 (TaskRunner pid=2167046) Training Progress: 82%|████████▏ | 82/100 [48:39<10:57, 36.52s/it] (TaskRunner pid=2167046) step:83 - global_seqlen/min:20094 - global_seqlen/max:24098 - global_seqlen/minmax_diff:4004 - global_seqlen/balanced_min:21362 - global_seqlen/balanced_max:21364 - global_seqlen/mean:21363.375 - actor/entropy:0.028036868199706078 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.938510537147522 - training/rollout_probs_diff_mean:0.0014980696141719818 - training/rollout_probs_diff_std:0.013240817934274673 - training/rollout_actor_probs_pearson_corr:0.9897127151489258 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.6953125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:1.0001375675201416 - rollout_corr/rollout_is_ratio_fraction_high:0.000113104579213541 - rollout_corr/rollout_is_ratio_fraction_low:0.0003393137303646654 - rollout_corr/rollout_is_max:3.193481922149658 - rollout_corr/rollout_is_min:0.04810234531760216 - rollout_corr/rollout_is_std:0.02907426841557026 - rollout_corr/rollout_is_eff_sample_size:0.9991558769107263 - rollout_corr/rollout_is_seq_mean:1.0002412796020508 - rollout_corr/rollout_is_seq_std:0.0025487029924988747 - rollout_corr/rollout_is_seq_max:1.015842080116272 - rollout_corr/rollout_is_seq_min:0.9927569627761841 - rollout_corr/rollout_is_seq_max_deviation:0.015842080116271973 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0276337102986872) - rollout_corr/training_log_ppl:np.float64(0.027054207998844504) - rollout_corr/kl:np.float64(0.0004606549058898679) - rollout_corr/k3_kl:np.float64(0.0007622376264420083) - rollout_corr/rollout_ppl:np.float64(1.0271579953841865) - rollout_corr/rollout_log_ppl:np.float64(0.026593552976464707) - rollout_corr/log_ppl_diff:np.float64(0.00046065502192504937) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017162600461233524) - rollout_corr/log_ppl_diff_max:np.float64(0.008437082171440125) - rollout_corr/log_ppl_diff_min:np.float64(-0.013399794697761536) - rollout_corr/ppl_ratio:np.float64(1.0004641304258257) - rollout_corr/chi2_token:np.float64(0.0028343098238110542) - rollout_corr/chi2_seq:np.float64(0.47074131085537374) - actor/pg_loss:np.float64(0.00014830310828983784) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010468151140230475) - actor/ppo_kl:np.float64(3.112560744966686e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6953125) - self_distillation/token_reweight_w_mean:np.float64(141561.96050062706) - self_distillation/token_reweight_w_std:np.float64(1861106.9816170656) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002270576078445) - self_distillation/token_reweight_w_clip_frac:np.float64(0.005041797267040238) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.059840649366378784) - perf/mfu/actor:np.float64(0.03939250038014562) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.21709823608398) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016824306221678853 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016824306221678853 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:276.29296875 - response_length/max:574.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:276.29296875 - response_length_non_aborted/max:574.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.3125 - prompt_length/max:492.0 - prompt_length/min:267.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010314397513866425 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4022988248616457) - timing_s/agent_loop/generate_sequences/max:np.float64(5.110102530568838) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.998306116562162) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.110102530568838) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:574 - timing_s/gen:11.175709571689367 - timing_s/reward:0.05753273330628872 - timing_s/old_log_prob:2.2706907521933317 - timing_s/adv:0.5061775743961334 - timing_s/update_actor:15.045591592788696 - timing_s/step:29.142487639561296 - timing_s/stop_profile:0.00010825134813785553 - timing_per_token_ms/gen:0.15800299121586528 - timing_per_token_ms/adv:0.0029617135307280183 - timing_per_token_ms/update_actor:0.08803379377549601 - perf/total_num_tokens:170907 - perf/time_per_step:29.142487639561296 - perf/throughput:733.0662798668892 (TaskRunner pid=2167046) Training Progress: 83%|████████▎ | 83/100 [49:08<09:43, 34.32s/it] (TaskRunner pid=2167046) step:84 - global_seqlen/min:19137 - global_seqlen/max:26002 - global_seqlen/minmax_diff:6865 - global_seqlen/balanced_min:22053 - global_seqlen/balanced_max:22057 - global_seqlen/mean:22055.875 - actor/entropy:0.030365219339728355 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535175025463104 - training/rollout_probs_diff_mean:0.0013786866329610348 - training/rollout_probs_diff_std:0.009515224024653435 - training/rollout_actor_probs_pearson_corr:0.9947026371955872 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.84375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.000044822692871 - rollout_corr/rollout_is_ratio_fraction_high:2.6288464141543955e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6288464141543955e-05 - rollout_corr/rollout_is_max:5.18784761428833 - rollout_corr/rollout_is_min:0.2994607388973236 - rollout_corr/rollout_is_std:0.024178551509976387 - rollout_corr/rollout_is_eff_sample_size:0.9994156201342632 - rollout_corr/rollout_is_seq_mean:0.9999868273735046 - rollout_corr/rollout_is_seq_std:0.0017935303039848804 - rollout_corr/rollout_is_seq_max:1.0134811401367188 - rollout_corr/rollout_is_seq_min:0.9930511713027954 - rollout_corr/rollout_is_seq_max_deviation:0.01348114013671875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0276782847940922) - rollout_corr/training_log_ppl:np.float64(0.027085085138423892) - rollout_corr/kl:np.float64(0.00033988594790629323) - rollout_corr/k3_kl:np.float64(0.0003952844845345749) - rollout_corr/rollout_ppl:np.float64(1.027325198519975) - rollout_corr/rollout_log_ppl:np.float64(0.026745198925254954) - rollout_corr/log_ppl_diff:np.float64(0.0003398862131689384) - rollout_corr/log_ppl_abs_diff:np.float64(0.001232237534622982) - rollout_corr/log_ppl_diff_max:np.float64(0.012646578252315521) - rollout_corr/log_ppl_diff_min:np.float64(-0.004848688840866089) - rollout_corr/ppl_ratio:np.float64(1.0003415157552809) - rollout_corr/chi2_token:np.float64(0.0012291001621633768) - rollout_corr/chi2_seq:np.float64(0.22799045592546463) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(2.268064918098389e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_w_mean:np.float64(0.9999999923165888) - self_distillation/token_reweight_w_std:np.float64(0.0) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999999923165888) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.040526068192393905) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.13608169555664) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.84375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.84375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:297.18359375 - response_length/max:750.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:297.18359375 - response_length_non_aborted/max:750.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:392.0625 - prompt_length/max:519.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010252371430397034 - timing_s/agent_loop/generate_sequences/min:np.float64(1.35355931147933) - timing_s/agent_loop/generate_sequences/max:np.float64(5.65272818878293) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.111852749600075) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.65272818878293) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:750 - timing_s/gen:11.531932955607772 - timing_s/reward:0.059469351544976234 - timing_s/old_log_prob:2.3598664570599794 - timing_s/adv:0.5169887784868479 - timing_s/update_actor:15.03002911619842 - timing_s/step:29.58426152355969 - timing_s/stop_profile:0.00011780299246311188 - timing_per_token_ms/gen:0.1515783981861982 - timing_per_token_ms/adv:0.0029299947207198076 - timing_per_token_ms/update_actor:0.08518155092576479 - perf/total_num_tokens:176447 - perf/time_per_step:29.58426152355969 - perf/throughput:745.5273129746912 (TaskRunner pid=2167046) Training Progress: 84%|████████▍ | 84/100 [49:38<08:46, 32.92s/it] (TaskRunner pid=2167046) step:85 - global_seqlen/min:17600 - global_seqlen/max:24927 - global_seqlen/minmax_diff:7327 - global_seqlen/balanced_min:21543 - global_seqlen/balanced_max:21545 - global_seqlen/mean:21544.25 - actor/entropy:0.03369623050093651 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4004964232444763 - training/rollout_probs_diff_mean:0.0016652214108034968 - training/rollout_probs_diff_std:0.010939822532236576 - training/rollout_actor_probs_pearson_corr:0.9941464066505432 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.00010085105896 - rollout_corr/rollout_is_ratio_fraction_high:9.723032417241484e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.334028098033741e-05 - rollout_corr/rollout_is_max:3.8197498321533203 - rollout_corr/rollout_is_min:0.18857713043689728 - rollout_corr/rollout_is_std:0.02875475026667118 - rollout_corr/rollout_is_eff_sample_size:0.9991739664724485 - rollout_corr/rollout_is_seq_mean:1.0000773668289185 - rollout_corr/rollout_is_seq_std:0.0023834542371332645 - rollout_corr/rollout_is_seq_max:1.023589015007019 - rollout_corr/rollout_is_seq_min:0.9939945936203003 - rollout_corr/rollout_is_seq_max_deviation:0.023589015007019043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0349912350066006) - rollout_corr/training_log_ppl:np.float64(0.034070851845172) - rollout_corr/kl:np.float64(0.000908018300947333) - rollout_corr/k3_kl:np.float64(0.0008562989545923827) - rollout_corr/rollout_ppl:np.float64(1.03404114022851) - rollout_corr/rollout_log_ppl:np.float64(0.03316283329104408) - rollout_corr/log_ppl_diff:np.float64(0.0009080185541279207) - rollout_corr/log_ppl_abs_diff:np.float64(0.001848379795319488) - rollout_corr/log_ppl_diff_max:np.float64(0.026849649846553802) - rollout_corr/log_ppl_diff_min:np.float64(-0.007245257496833801) - rollout_corr/ppl_ratio:np.float64(1.0009134255815297) - rollout_corr/chi2_token:np.float64(0.0018449586350470781) - rollout_corr/chi2_seq:np.float64(0.9608175957109779) - actor/pg_loss:np.float64(0.0002799328649416566) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002830132093549764) - actor/ppo_kl:np.float64(0.00048529359718330056) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(166367.9715071246) - self_distillation/token_reweight_w_std:np.float64(1815967.0959693075) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000693320762366) - self_distillation/token_reweight_w_clip_frac:np.float64(0.007904627491370775) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.19982873648405075) - perf/mfu/actor:np.float64(0.039475850572369256) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.0826416015625) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002161638578400016 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002161638578400016 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.2265625 - response_length/max:586.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.2265625 - response_length_non_aborted/max:586.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:392.03125 - prompt_length/max:541.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014371797442436218 - timing_s/agent_loop/generate_sequences/min:np.float64(1.240128206089139) - timing_s/agent_loop/generate_sequences/max:np.float64(5.077182488515973) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.134709821853903) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.077182488515973) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:554 - timing_s/gen:10.776293391361833 - timing_s/reward:0.05615835078060627 - timing_s/old_log_prob:2.3090774826705456 - timing_s/adv:0.5167031604796648 - timing_s/update_actor:15.053321877494454 - timing_s/step:28.79759426228702 - timing_s/stop_profile:0.00011362694203853607 - timing_per_token_ms/gen:0.14968321514795446 - timing_per_token_ms/adv:0.0029979180087474893 - timing_per_token_ms/update_actor:0.08733955624757449 - perf/total_num_tokens:172354 - perf/time_per_step:28.79759426228702 - perf/throughput:748.126729051603 (TaskRunner pid=2167046) Training Progress: 85%|████████▌ | 85/100 [50:06<07:55, 31.70s/it] (TaskRunner pid=2167046) step:86 - global_seqlen/min:18423 - global_seqlen/max:23773 - global_seqlen/minmax_diff:5350 - global_seqlen/balanced_min:21398 - global_seqlen/balanced_max:21401 - global_seqlen/mean:21400.0 - actor/entropy:0.039314959198236465 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41023650765419006 - training/rollout_probs_diff_mean:0.0018119228770956397 - training/rollout_probs_diff_std:0.011676772497594357 - training/rollout_actor_probs_pearson_corr:0.9939371347427368 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.81640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:1.0000507831573486 - rollout_corr/rollout_is_ratio_fraction_high:9.458437125431374e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001756566925905645 - rollout_corr/rollout_is_max:2.5501339435577393 - rollout_corr/rollout_is_min:0.18535448610782623 - rollout_corr/rollout_is_std:0.030159026384353638 - rollout_corr/rollout_is_eff_sample_size:0.9990913786735653 - rollout_corr/rollout_is_seq_mean:1.0000463724136353 - rollout_corr/rollout_is_seq_std:0.002288077026605606 - rollout_corr/rollout_is_seq_max:1.0129072666168213 - rollout_corr/rollout_is_seq_min:0.9928169250488281 - rollout_corr/rollout_is_seq_max_deviation:0.012907266616821289 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0370242814533412) - rollout_corr/training_log_ppl:np.float64(0.036050161899311206) - rollout_corr/kl:np.float64(0.0012159123268471461) - rollout_corr/k3_kl:np.float64(0.001352685779481999) - rollout_corr/rollout_ppl:np.float64(1.0357652008533478) - rollout_corr/rollout_log_ppl:np.float64(0.03483424940327495) - rollout_corr/log_ppl_diff:np.float64(0.0012159124960362533) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024147777801317716) - rollout_corr/log_ppl_diff_max:np.float64(0.016132771968841553) - rollout_corr/log_ppl_diff_min:np.float64(-0.022180378437042236) - rollout_corr/ppl_ratio:np.float64(1.0012233622837812) - rollout_corr/chi2_token:np.float64(0.007148264441639185) - rollout_corr/chi2_seq:np.float64(2.3162417996209115) - actor/pg_loss:np.float64(0.000582026899792254) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015828989535293658) - actor/ppo_kl:np.float64(0.0007632725505146709) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.81640625) - self_distillation/token_reweight_w_mean:np.float64(79115.98013575724) - self_distillation/token_reweight_w_std:np.float64(957300.9500577603) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999438552185893) - self_distillation/token_reweight_w_clip_frac:np.float64(0.005835453615873121) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.1951440516859293) - perf/mfu/actor:np.float64(0.039398913577472425) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.11903762817383) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009458436979912221 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009458436979912221 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:289.09375 - response_length/max:846.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.09375 - response_length_non_aborted/max:846.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.65625 - prompt_length/max:483.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010889209806919098 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2017557490617037) - timing_s/agent_loop/generate_sequences/max:np.float64(6.2804846949875355) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.039875644535641) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.2804846949875355) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:267 - timing_s/agent_loop/slowest/response_length:846 - timing_s/gen:12.230041805654764 - timing_s/reward:0.05486845597624779 - timing_s/old_log_prob:2.268472572788596 - timing_s/adv:0.5610908549278975 - timing_s/update_actor:15.074780009686947 - timing_s/step:30.28217082284391 - timing_s/stop_profile:0.0001094285398721695 - timing_per_token_ms/gen:0.16525297002560216 - timing_per_token_ms/adv:0.003277399853550803 - timing_per_token_ms/update_actor:0.08805362155190974 - perf/total_num_tokens:171200 - perf/time_per_step:30.28217082284391 - perf/throughput:706.6864567006709 (TaskRunner pid=2167046) Training Progress: 86%|████████▌ | 86/100 [50:37<07:18, 31.29s/it] (TaskRunner pid=2167046) step:87 - global_seqlen/min:19044 - global_seqlen/max:24169 - global_seqlen/minmax_diff:5125 - global_seqlen/balanced_min:21777 - global_seqlen/balanced_max:21778 - global_seqlen/mean:21777.125 - actor/entropy:0.027197115123271942 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.875576913356781 - training/rollout_probs_diff_mean:0.0015121392207220197 - training/rollout_probs_diff_std:0.011220099404454231 - training/rollout_actor_probs_pearson_corr:0.9922543168067932 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:0.9999395608901978 - rollout_corr/rollout_is_ratio_fraction_high:5.836263517267071e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00016049724945332855 - rollout_corr/rollout_is_max:3.3198060989379883 - rollout_corr/rollout_is_min:0.0059265922755002975 - rollout_corr/rollout_is_std:0.027668792754411697 - rollout_corr/rollout_is_eff_sample_size:0.9992347855255346 - rollout_corr/rollout_is_seq_mean:0.99995356798172 - rollout_corr/rollout_is_seq_std:0.0020202931482344866 - rollout_corr/rollout_is_seq_max:1.010077953338623 - rollout_corr/rollout_is_seq_min:0.9906949400901794 - rollout_corr/rollout_is_seq_max_deviation:0.010077953338623047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.030908034183085) - rollout_corr/training_log_ppl:np.float64(0.030158942176967685) - rollout_corr/kl:np.float64(0.0022416504295428297) - rollout_corr/k3_kl:np.float64(0.0021501615189396077) - rollout_corr/rollout_ppl:np.float64(1.0285401679575443) - rollout_corr/rollout_log_ppl:np.float64(0.02791729165619472) - rollout_corr/log_ppl_diff:np.float64(0.0022416505207729642) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032178409592233947) - rollout_corr/log_ppl_diff_max:np.float64(0.07320429384708405) - rollout_corr/log_ppl_diff_min:np.float64(-0.006806857883930206) - rollout_corr/ppl_ratio:np.float64(1.002269170479849) - rollout_corr/chi2_token:np.float64(0.004207434365525842) - rollout_corr/chi2_seq:np.float64(0.16284064133651555) - actor/pg_loss:np.float64(0.0011388499988242984) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020194145145069342) - actor/ppo_kl:np.float64(0.0016807883800389334) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(184774.13582514809) - self_distillation/token_reweight_w_std:np.float64(1593571.5521162562) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002801124937832) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006642151573032606) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.26061127707362175) - perf/mfu/actor:np.float64(0.040144429518951516) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.14898681640625) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004906108952127397 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004906108952127397 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:267.72265625 - response_length/max:557.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:267.72265625 - response_length_non_aborted/max:557.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:412.8125 - prompt_length/max:518.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001679845154285431 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2957756333053112) - timing_s/agent_loop/generate_sequences/max:np.float64(4.838256204500794) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0168341065655113) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.838256204500794) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:557 - timing_s/gen:10.759811215102673 - timing_s/reward:0.04805499501526356 - timing_s/old_log_prob:2.255372365936637 - timing_s/adv:0.5169291161000729 - timing_s/update_actor:15.184918504208326 - timing_s/step:28.803423115983605 - timing_s/stop_profile:7.556378841400146e-05 - timing_per_token_ms/gen:0.15699273699027785 - timing_per_token_ms/adv:0.0029671565696807595 - timing_per_token_ms/update_actor:0.08716094585607792 - perf/total_num_tokens:174217 - perf/time_per_step:28.803423115983605 - perf/throughput:756.0603096482456 (TaskRunner pid=2167046) Training Progress: 87%|████████▋ | 87/100 [51:06<06:37, 30.55s/it] (TaskRunner pid=2167046) step:88 - global_seqlen/min:18952 - global_seqlen/max:23483 - global_seqlen/minmax_diff:4531 - global_seqlen/balanced_min:21164 - global_seqlen/balanced_max:21170 - global_seqlen/mean:21167.875 - actor/entropy:0.03469604626297951 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7770388722419739 - training/rollout_probs_diff_mean:0.0016765209147706628 - training/rollout_probs_diff_std:0.012051005847752094 - training/rollout_actor_probs_pearson_corr:0.9929908514022827 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.00003981590271 - rollout_corr/rollout_is_ratio_fraction_high:0.00011329037806717679 - rollout_corr/rollout_is_ratio_fraction_low:0.00011329037806717679 - rollout_corr/rollout_is_max:3.9595537185668945 - rollout_corr/rollout_is_min:0.06086057052016258 - rollout_corr/rollout_is_std:0.02961859479546547 - rollout_corr/rollout_is_eff_sample_size:0.9991232697500432 - rollout_corr/rollout_is_seq_mean:1.0000507831573486 - rollout_corr/rollout_is_seq_std:0.0024844331201165915 - rollout_corr/rollout_is_seq_max:1.0169881582260132 - rollout_corr/rollout_is_seq_min:0.9927829504013062 - rollout_corr/rollout_is_seq_max_deviation:0.016988158226013184 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0361049468629062) - rollout_corr/training_log_ppl:np.float64(0.034795299737993446) - rollout_corr/kl:np.float64(0.00043864597136611394) - rollout_corr/k3_kl:np.float64(0.0006331470921853155) - rollout_corr/rollout_ppl:np.float64(1.0356523073278368) - rollout_corr/rollout_log_ppl:np.float64(0.03435665349843475) - rollout_corr/log_ppl_diff:np.float64(0.0004386462377397038) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017799362379946615) - rollout_corr/log_ppl_diff_max:np.float64(0.018246829509735107) - rollout_corr/log_ppl_diff_min:np.float64(-0.006508391350507736) - rollout_corr/ppl_ratio:np.float64(1.0004424944054335) - rollout_corr/chi2_token:np.float64(0.0017431846354156733) - rollout_corr/chi2_seq:np.float64(0.6097231933381408) - actor/pg_loss:np.float64(0.00028333289083093405) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020099821449548472) - actor/ppo_kl:np.float64(-5.0402789252501634e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(131093.8798913369) - self_distillation/token_reweight_w_std:np.float64(1555027.8888158603) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006680756341666) - self_distillation/token_reweight_w_clip_frac:np.float64(0.010408774585812353) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.1889299713075161) - perf/mfu/actor:np.float64(0.038953613110281254) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.14186477661133) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00011329037806717679 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00011329037806717679 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.83984375 - response_length/max:742.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.83984375 - response_length_non_aborted/max:742.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:385.65625 - prompt_length/max:490.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.541477680206299e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5102975443005562) - timing_s/agent_loop/generate_sequences/max:np.float64(5.756528107449412) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9962517545209266) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.756528107449412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:728 - timing_s/gen:11.626551555469632 - timing_s/reward:0.0533559899777174 - timing_s/old_log_prob:2.2899036556482315 - timing_s/adv:0.5903963912278414 - timing_s/update_actor:15.109874501824379 - timing_s/step:29.769190276041627 - timing_s/stop_profile:4.385039210319519e-05 - timing_per_token_ms/gen:0.1646470516953853 - timing_per_token_ms/adv:0.0034863938351620168 - timing_per_token_ms/update_actor:0.08922644869775768 - perf/total_num_tokens:169343 - perf/time_per_step:29.769190276041627 - perf/throughput:711.0665356939855 (TaskRunner pid=2167046) Training Progress: 88%|████████▊ | 88/100 [51:35<06:03, 30.32s/it] (TaskRunner pid=2167046) step:89 - global_seqlen/min:20201 - global_seqlen/max:24059 - global_seqlen/minmax_diff:3858 - global_seqlen/balanced_min:22120 - global_seqlen/balanced_max:22122 - global_seqlen/mean:22121.375 - actor/entropy:0.0294000506401062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9222151041030884 - training/rollout_probs_diff_mean:0.0016106810653582215 - training/rollout_probs_diff_std:0.01242443360388279 - training/rollout_actor_probs_pearson_corr:0.9914741516113281 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.73828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0001593828201294 - rollout_corr/rollout_is_ratio_fraction_high:0.00010028222459368408 - rollout_corr/rollout_is_ratio_fraction_low:0.0002292165154358372 - rollout_corr/rollout_is_max:2.4047913551330566 - rollout_corr/rollout_is_min:0.0020848875865340233 - rollout_corr/rollout_is_std:0.02992490865290165 - rollout_corr/rollout_is_eff_sample_size:0.9991055390121959 - rollout_corr/rollout_is_seq_mean:1.0000648498535156 - rollout_corr/rollout_is_seq_std:0.0025368130300194025 - rollout_corr/rollout_is_seq_max:1.0112179517745972 - rollout_corr/rollout_is_seq_min:0.9908243417739868 - rollout_corr/rollout_is_seq_max_deviation:0.011217951774597168 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0337297716178) - rollout_corr/training_log_ppl:np.float64(0.03286470607508818) - rollout_corr/kl:np.float64(0.0006779706385913187) - rollout_corr/k3_kl:np.float64(0.001490014383888183) - rollout_corr/rollout_ppl:np.float64(1.0330208893865347) - rollout_corr/rollout_log_ppl:np.float64(0.03218673507581116) - rollout_corr/log_ppl_diff:np.float64(0.0006779709992770222) - rollout_corr/log_ppl_abs_diff:np.float64(0.002508861419300956) - rollout_corr/log_ppl_diff_max:np.float64(0.03041791543364525) - rollout_corr/log_ppl_diff_min:np.float64(-0.01392044872045517) - rollout_corr/ppl_ratio:np.float64(1.0006865614559501) - rollout_corr/chi2_token:np.float64(0.0062223493587225676) - rollout_corr/chi2_seq:np.float64(0.9317113894503564) - actor/pg_loss:np.float64(0.0008178113494068384) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023295565551961772) - actor/ppo_kl:np.float64(0.0001360127029670366) - actor/pg_clipfrac_lower:np.float64(2.396472336840816e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_w_mean:np.float64(144165.8516279969) - self_distillation/token_reweight_w_std:np.float64(1638643.5006887442) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003369075711817) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006198407252668403) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.32256839238107204) - perf/mfu/actor:np.float64(0.04058150884673786) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.04215621948242) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:3 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0013556007761508226 - critic/advantages/max:0.75 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0013556007761508226 - critic/returns/max:0.75 - critic/returns/min:-0.75 - response_length/mean:272.66796875 - response_length/max:573.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.66796875 - response_length_non_aborted/max:573.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:418.625 - prompt_length/max:512.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.828883826732635e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.143798092380166) - timing_s/agent_loop/generate_sequences/max:np.float64(5.059799168258905) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0779781756937155) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.059799168258905) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:563 - timing_s/gen:10.962892504408956 - timing_s/reward:0.05267326347529888 - timing_s/old_log_prob:2.2350706923753023 - timing_s/adv:0.5168185103684664 - timing_s/update_actor:15.059026561677456 - timing_s/step:28.923460172489285 - timing_s/stop_profile:3.618001937866211e-05 - timing_per_token_ms/gen:0.15705474699381053 - timing_per_token_ms/adv:0.002920357066233826 - timing_per_token_ms/update_actor:0.0850931879329238 - perf/total_num_tokens:176971 - perf/time_per_step:28.923460172489285 - perf/throughput:764.8246395167088 (TaskRunner pid=2167046) Training Progress: 89%|████████▉ | 89/100 [52:04<05:29, 29.91s/it] (TaskRunner pid=2167046) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2167046) validation generation end (TaskRunner pid=2167046) [prompt] system (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume $ V $ of a unit cell in a crystal structure can be calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c \cdot \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given that all angles $ \alpha, \beta, \gamma $ are 90.00º, the cosine of each angle is 0. Substituting these values into the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = a \cdot b \cdot c \cdot \sqrt{1 + 2(0)(0)(0) - (0)^2 - (0)^2 - (0)^2} = a \cdot b \cdot c \cdot \sqrt{1} = a \cdot b \cdot c (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Now, substitute the given lattice parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = 7.82 \, \text{Å} \times 7.82 \, \text{Å} \times 17.88 \, \text{Å} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 7.82 \times 7.82 = 61.1524 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then multiply by 17.88: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 61.1524 \times 17.88 \approx 1094.31 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume of the unit cell is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2167440) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2167046) step:90 - global_seqlen/min:18160 - global_seqlen/max:22613 - global_seqlen/minmax_diff:4453 - global_seqlen/balanced_min:21124 - global_seqlen/balanced_max:21127 - global_seqlen/mean:21126.0 - actor/entropy:0.03373492509126663 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032570362091064 - training/rollout_probs_diff_mean:0.0016449851682409644 - training/rollout_probs_diff_std:0.010777240619063377 - training/rollout_actor_probs_pearson_corr:0.9940168261528015 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.78125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.9999426603317261 - rollout_corr/rollout_is_ratio_fraction_high:4.441023338586092e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013323070015758276 - rollout_corr/rollout_is_max:3.03580379486084 - rollout_corr/rollout_is_min:0.14749553799629211 - rollout_corr/rollout_is_std:0.027225784957408905 - rollout_corr/rollout_is_eff_sample_size:0.9992594247051011 - rollout_corr/rollout_is_seq_mean:0.9998467564582825 - rollout_corr/rollout_is_seq_std:0.0020057959482073784 - rollout_corr/rollout_is_seq_max:1.0077792406082153 - rollout_corr/rollout_is_seq_min:0.9924387335777283 - rollout_corr/rollout_is_seq_max_deviation:0.007779240608215332 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0404213974252343) - rollout_corr/training_log_ppl:np.float64(0.03911794312261918) - rollout_corr/kl:np.float64(0.006334157641134652) - rollout_corr/k3_kl:np.float64(0.005529801496451547) - rollout_corr/rollout_ppl:np.float64(1.0335820065811276) - rollout_corr/rollout_log_ppl:np.float64(0.03278378526101733) - rollout_corr/log_ppl_diff:np.float64(0.006334157883429725) - rollout_corr/log_ppl_abs_diff:np.float64(0.007246610600304848) - rollout_corr/log_ppl_diff_max:np.float64(0.0966530591249466) - rollout_corr/log_ppl_diff_min:np.float64(-0.011298129335045815) - rollout_corr/ppl_ratio:np.float64(1.0064935444388539) - rollout_corr/chi2_token:np.float64(0.006344154244288802) - rollout_corr/chi2_seq:np.float64(1.2312942533753812) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.005713566887692423) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_w_mean:np.float64(0.9999999932479113) - self_distillation/token_reweight_w_std:np.float64(0.0) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999999932479113) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.03905662064435432) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.02539443969727) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7639627659574468) - val-aux/sciknoweval/reward/std@16:np.float64(0.03739248197220592) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7783723404255319) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.025295053531890987) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7499042553191487) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.03754459470577036) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7638936170212767) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.03753661391631336) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7872446808510639) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.015370639381592657) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7354255319148936) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.035222086964244725) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7674361702127659) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.027474048017876722) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7931595744680849) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.00918105401702876) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7204787234042552) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.03113333199731948) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7699893617021277) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.01673564647493126) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7965851063829787) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.004315713066034739) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.7062978723404255) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.023264301416585198) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7715744680851063) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.011274235759200167) - val-aux/sciknoweval/score/mean@16:np.float64(0.7639627659574468) - val-aux/sciknoweval/score/std@16:np.float64(0.03739248197220592) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7783723404255319) - val-aux/sciknoweval/score/best@2/std:np.float64(0.025295053531890987) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7499042553191487) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.03754459470577036) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7638936170212767) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.03753661391631336) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7872446808510639) - val-aux/sciknoweval/score/best@4/std:np.float64(0.015370639381592657) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7354255319148936) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.035222086964244725) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7674361702127659) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.027474048017876722) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7931595744680849) - val-aux/sciknoweval/score/best@8/std:np.float64(0.00918105401702876) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7204787234042552) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.03113333199731948) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7699893617021277) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.01673564647493126) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7965851063829787) - val-aux/sciknoweval/score/best@16/std:np.float64(0.004315713066034739) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.7062978723404255) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.023264301416585198) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7715744680851063) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.011274235759200167) - val-core/sciknoweval/acc/mean@16:np.float64(0.7639627659574468) - val-aux/sciknoweval/acc/std@16:np.float64(0.03739248197220592) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7783723404255319) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.025295053531890987) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7499042553191487) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.03754459470577036) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7638936170212767) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.03753661391631336) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7872446808510639) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.015370639381592657) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7354255319148936) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.035222086964244725) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7674361702127659) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.027474048017876722) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7931595744680849) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.00918105401702876) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7204787234042552) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.03113333199731948) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7699893617021277) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.01673564647493126) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7965851063829787) - val-core/sciknoweval/acc/best@16/std:np.float64(0.004315713066034739) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.7062978723404255) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.023264301416585198) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7715744680851063) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.011274235759200167) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:3 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:263.875 - response_length/max:635.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.875 - response_length_non_aborted/max:635.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.3125 - prompt_length/max:509.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.10409814119339e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9039298556745052) - timing_s/agent_loop/generate_sequences/max:np.float64(5.1769066993147135) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8741497110022465) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.1769066993147135) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:287 - timing_s/agent_loop/slowest/response_length:635 - timing_s/gen:11.141404872760177 - timing_s/reward:0.05317526310682297 - timing_s/old_log_prob:2.2556243874132633 - timing_s/adv:0.5280161388218403 - timing_s/update_actor:14.898592561483383 - timing_s/step:28.927802968770266 - timing_s/testing:37.425777373835444 - timing_s/save_checkpoint:6.877587698400021 - timing_s/stop_profile:0.0001491345465183258 - timing_per_token_ms/gen:0.16493079217136689 - timing_per_token_ms/adv:0.003124207959515764 - timing_per_token_ms/update_actor:0.0881531795032388 - perf/total_num_tokens:169008 - perf/time_per_step:28.927802968770266 - perf/throughput:730.300881225135 (TaskRunner pid=2167046) Training Progress: 90%|█████████ | 90/100 [53:18<07:09, 42.92s/it] (TaskRunner pid=2167046) step:91 - global_seqlen/min:18107 - global_seqlen/max:23168 - global_seqlen/minmax_diff:5061 - global_seqlen/balanced_min:21250 - global_seqlen/balanced_max:21254 - global_seqlen/mean:21251.875 - actor/entropy:0.04534173756837845 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4046512842178345 - training/rollout_probs_diff_mean:0.0016173782059922814 - training/rollout_probs_diff_std:0.010041984729468822 - training/rollout_actor_probs_pearson_corr:0.9960680603981018 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999968409538269 - rollout_corr/rollout_is_ratio_fraction_high:2.5360435756738298e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5360435756738298e-05 - rollout_corr/rollout_is_max:5.243876934051514 - rollout_corr/rollout_is_min:0.37942102551460266 - rollout_corr/rollout_is_std:0.025519412010908127 - rollout_corr/rollout_is_eff_sample_size:0.9993493025433431 - rollout_corr/rollout_is_seq_mean:0.9999523162841797 - rollout_corr/rollout_is_seq_std:0.002020125510171056 - rollout_corr/rollout_is_seq_max:1.012386441230774 - rollout_corr/rollout_is_seq_min:0.9923055171966553 - rollout_corr/rollout_is_seq_max_deviation:0.012386441230773926 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0488830306567252) - rollout_corr/training_log_ppl:np.float64(0.0465941856673453) - rollout_corr/kl:np.float64(0.0016078722916241794) - rollout_corr/k3_kl:np.float64(0.0012860047578371336) - rollout_corr/rollout_ppl:np.float64(1.047186788637191) - rollout_corr/rollout_log_ppl:np.float64(0.04498631308433687) - rollout_corr/log_ppl_diff:np.float64(0.0016078725830084295) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026501509328227257) - rollout_corr/log_ppl_diff_max:np.float64(0.04505942016839981) - rollout_corr/log_ppl_diff_min:np.float64(-0.015204224735498428) - rollout_corr/ppl_ratio:np.float64(1.0016248126048595) - rollout_corr/chi2_token:np.float64(0.0017210114747285843) - rollout_corr/chi2_seq:np.float64(2.4243051193188876) - actor/pg_loss:np.float64(0.0012130095856264234) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044657633543465636) - actor/ppo_kl:np.float64(0.0011831951068810298) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(150807.6977907042) - self_distillation/token_reweight_w_std:np.float64(1827940.7117348753) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000478033907712) - self_distillation/token_reweight_w_clip_frac:np.float64(0.012809829524485394) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.2270626574754715) - perf/mfu/actor:np.float64(0.038449014366350065) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.40208053588867) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:3 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00025835944688878953 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00025835944688878953 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:308.05859375 - response_length/max:777.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.05859375 - response_length_non_aborted/max:777.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:356.0625 - prompt_length/max:477.0 - prompt_length/min:196.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.956288754940033e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3745724577456713) - timing_s/agent_loop/generate_sequences/max:np.float64(6.176387269049883) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.267919529833307) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.176387269049883) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:777 - timing_s/gen:11.775354152545333 - timing_s/reward:0.05318715050816536 - timing_s/old_log_prob:2.3645179886370897 - timing_s/adv:0.549498001113534 - timing_s/update_actor:15.437682662159204 - timing_s/step:30.2809515055269 - timing_s/stop_profile:0.00013466738164424896 - timing_per_token_ms/gen:0.1493140528834223 - timing_per_token_ms/adv:0.0032320560016088815 - timing_per_token_ms/update_actor:0.09080188608157636 - perf/total_num_tokens:170015 - perf/time_per_step:30.2809515055269 - perf/throughput:701.8232236236398 (TaskRunner pid=2167046) Training Progress: 91%|█████████ | 91/100 [53:48<05:52, 39.14s/it] (TaskRunner pid=2167046) step:92 - global_seqlen/min:21040 - global_seqlen/max:26095 - global_seqlen/minmax_diff:5055 - global_seqlen/balanced_min:23108 - global_seqlen/balanced_max:23110 - global_seqlen/mean:23109.375 - actor/entropy:0.049469124525785446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38041767477989197 - training/rollout_probs_diff_mean:0.001835858216509223 - training/rollout_probs_diff_std:0.010576015338301659 - training/rollout_actor_probs_pearson_corr:0.9960457682609558 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.69140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.999874472618103 - rollout_corr/rollout_is_ratio_fraction_high:3.575727896532044e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011919093230972067 - rollout_corr/rollout_is_max:2.2295782566070557 - rollout_corr/rollout_is_min:0.2960941791534424 - rollout_corr/rollout_is_std:0.02729356847703457 - rollout_corr/rollout_is_eff_sample_size:0.9992554966385375 - rollout_corr/rollout_is_seq_mean:0.9999048709869385 - rollout_corr/rollout_is_seq_std:0.0016976064071059227 - rollout_corr/rollout_is_seq_max:1.0060241222381592 - rollout_corr/rollout_is_seq_min:0.9939712285995483 - rollout_corr/rollout_is_seq_max_deviation:0.00602877140045166 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0476838406175375) - rollout_corr/training_log_ppl:np.float64(0.045997846729278535) - rollout_corr/kl:np.float64(0.000593296980607505) - rollout_corr/k3_kl:np.float64(0.0005309369002016728) - rollout_corr/rollout_ppl:np.float64(1.0470488839782774) - rollout_corr/rollout_log_ppl:np.float64(0.04540454928155668) - rollout_corr/log_ppl_diff:np.float64(0.0005932974477218522) - rollout_corr/log_ppl_abs_diff:np.float64(0.001585291376159148) - rollout_corr/log_ppl_diff_max:np.float64(0.011447396129369736) - rollout_corr/log_ppl_diff_min:np.float64(-0.007249504327774048) - rollout_corr/ppl_ratio:np.float64(1.0005958129186183) - rollout_corr/chi2_token:np.float64(0.001005234895274043) - rollout_corr/chi2_seq:np.float64(0.339298699516803) - actor/pg_loss:np.float64(0.00011454732157289982) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.591906640096568e-05) - actor/ppo_kl:np.float64(0.00011144039859800614) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69140625) - self_distillation/token_reweight_w_mean:np.float64(27679.604563228786) - self_distillation/token_reweight_w_std:np.float64(400284.27851104736) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002795667387545) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0033208947570528835) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.04137406311929226) - perf/mfu/actor:np.float64(0.042280402409404685) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.34305953979492) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:3 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007419635658152401 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:0.0007419635658152401 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:327.73046875 - response_length/max:601.0 - response_length/min:90.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.73046875 - response_length_non_aborted/max:601.0 - response_length_non_aborted/min:90.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.4375 - prompt_length/max:476.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.195638656616211e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.066079268231988) - timing_s/agent_loop/generate_sequences/max:np.float64(5.343240167945623) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6404598403241835) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.343240167945623) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:569 - timing_s/gen:11.119418624788523 - timing_s/reward:0.06010722741484642 - timing_s/old_log_prob:2.3270440250635147 - timing_s/adv:0.520053131505847 - timing_s/update_actor:15.255080388858914 - timing_s/step:29.369669487699866 - timing_s/stop_profile:0.00011682882905006409 - timing_per_token_ms/gen:0.13253338686740632 - timing_per_token_ms/adv:0.0028129986829254737 - timing_per_token_ms/update_actor:0.08251564780991975 - perf/total_num_tokens:184875 - perf/time_per_step:29.369669487699866 - perf/throughput:786.8449118802068 (TaskRunner pid=2167046) Training Progress: 92%|█████████▏| 92/100 [54:17<04:49, 36.23s/it] (TaskRunner pid=2167046) step:93 - global_seqlen/min:16732 - global_seqlen/max:24478 - global_seqlen/minmax_diff:7746 - global_seqlen/balanced_min:21236 - global_seqlen/balanced_max:21239 - global_seqlen/mean:21237.875 - actor/entropy:0.037446752190589905 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7017467021942139 - training/rollout_probs_diff_mean:0.0014789922861382365 - training/rollout_probs_diff_std:0.010445463471114635 - training/rollout_actor_probs_pearson_corr:0.9950976371765137 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:1.000028133392334 - rollout_corr/rollout_is_ratio_fraction_high:6.884113099658862e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001101458037737757 - rollout_corr/rollout_is_max:2.2795581817626953 - rollout_corr/rollout_is_min:0.27706241607666016 - rollout_corr/rollout_is_std:0.02562429942190647 - rollout_corr/rollout_is_eff_sample_size:0.999343826080078 - rollout_corr/rollout_is_seq_mean:0.9999866485595703 - rollout_corr/rollout_is_seq_std:0.0016226578736677766 - rollout_corr/rollout_is_seq_max:1.0060209035873413 - rollout_corr/rollout_is_seq_min:0.9960787892341614 - rollout_corr/rollout_is_seq_max_deviation:0.006020903587341309 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0422259448096156) - rollout_corr/training_log_ppl:np.float64(0.040301359121713176) - rollout_corr/kl:np.float64(0.0005277691595435563) - rollout_corr/k3_kl:np.float64(0.000485558227204419) - rollout_corr/rollout_ppl:np.float64(1.0416553649120033) - rollout_corr/rollout_log_ppl:np.float64(0.03977358995780378) - rollout_corr/log_ppl_diff:np.float64(0.0005277691602714185) - rollout_corr/log_ppl_abs_diff:np.float64(0.001454660314720968) - rollout_corr/log_ppl_diff_max:np.float64(0.008309893310070038) - rollout_corr/log_ppl_diff_min:np.float64(-0.005868792533874512) - rollout_corr/ppl_ratio:np.float64(1.0005299146287143) - rollout_corr/chi2_token:np.float64(0.0008362585213035345) - rollout_corr/chi2_seq:np.float64(0.17302076471969485) - actor/pg_loss:np.float64(0.0005822493694722652) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018108219865098363) - actor/ppo_kl:np.float64(0.0001428027055716541) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_w_mean:np.float64(143667.05467315274) - self_distillation/token_reweight_w_std:np.float64(1510607.7448990536) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000457826536149) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011807282658992335) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.18011473584920168) - perf/mfu/actor:np.float64(0.0392293668835817) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.27584838867188) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:3 - critic/score/mean:0.7734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0028964905068278313 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0028964905068278313 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:283.71484375 - response_length/max:598.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:283.71484375 - response_length_non_aborted/max:598.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:379.96875 - prompt_length/max:492.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.985532283782959e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7112865522503853) - timing_s/agent_loop/generate_sequences/max:np.float64(4.990263374522328) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0313110155111644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.990263374522328) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:543 - timing_s/gen:10.725034764036536 - timing_s/reward:0.05722231976687908 - timing_s/old_log_prob:2.32738777436316 - timing_s/adv:0.5332276038825512 - timing_s/update_actor:15.073657972738147 - timing_s/step:28.806073553860188 - timing_s/stop_profile:0.0001096818596124649 - timing_per_token_ms/gen:0.14766469915100353 - timing_per_token_ms/adv:0.0031384237116622495 - timing_per_token_ms/update_actor:0.08871919844109961 - perf/total_num_tokens:169903 - perf/time_per_step:28.806073553860188 - perf/throughput:737.2707342529852 (TaskRunner pid=2167046) Training Progress: 93%|█████████▎| 93/100 [54:46<03:58, 34.01s/it] (TaskRunner pid=2167046) step:94 - global_seqlen/min:18330 - global_seqlen/max:25428 - global_seqlen/minmax_diff:7098 - global_seqlen/balanced_min:22781 - global_seqlen/balanced_max:22786 - global_seqlen/mean:22783.75 - actor/entropy:0.041148312389850616 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4031469225883484 - training/rollout_probs_diff_mean:0.0015067117055878043 - training/rollout_probs_diff_std:0.00942541379481554 - training/rollout_actor_probs_pearson_corr:0.9961856603622437 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.75 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.999973714351654 - rollout_corr/rollout_is_ratio_fraction_high:2.427950494165998e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.069876326364465e-05 - rollout_corr/rollout_is_max:3.678283214569092 - rollout_corr/rollout_is_min:0.35399362444877625 - rollout_corr/rollout_is_std:0.024225344881415367 - rollout_corr/rollout_is_eff_sample_size:0.9994133578093227 - rollout_corr/rollout_is_seq_mean:1.0000309944152832 - rollout_corr/rollout_is_seq_std:0.0017812533769756556 - rollout_corr/rollout_is_seq_max:1.0156874656677246 - rollout_corr/rollout_is_seq_min:0.9939485788345337 - rollout_corr/rollout_is_seq_max_deviation:0.01568746566772461 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0413265423849225) - rollout_corr/training_log_ppl:np.float64(0.039565609684359515) - rollout_corr/kl:np.float64(0.00037305394429409944) - rollout_corr/k3_kl:np.float64(0.00047105288938809053) - rollout_corr/rollout_ppl:np.float64(1.040933893993497) - rollout_corr/rollout_log_ppl:np.float64(0.03919255502250962) - rollout_corr/log_ppl_diff:np.float64(0.000373054661849892) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014055275255486777) - rollout_corr/log_ppl_diff_max:np.float64(0.009166814386844635) - rollout_corr/log_ppl_diff_min:np.float64(-0.007318452000617981) - rollout_corr/ppl_ratio:np.float64(1.00037496490404) - rollout_corr/chi2_token:np.float64(0.0012893092352896929) - rollout_corr/chi2_seq:np.float64(0.37059705168940127) - actor/pg_loss:np.float64(0.0002516477834433317) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.616753176378552e-05) - actor/ppo_kl:np.float64(7.975473046561987e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_w_mean:np.float64(70703.44456977467) - self_distillation/token_reweight_w_std:np.float64(886742.9536180737) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004530518781394) - self_distillation/token_reweight_w_clip_frac:np.float64(0.008093894488411024) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.08848729822784662) - perf/mfu/actor:np.float64(0.042086406035394576) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.20873260498047) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:3 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001916563487611711 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001916563487611711 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.7734375 - response_length/max:786.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:321.7734375 - response_length_non_aborted/max:786.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:390.21875 - prompt_length/max:483.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013638846576213837 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3756834175437689) - timing_s/agent_loop/generate_sequences/max:np.float64(6.131799399852753) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.531095000435016) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.131799399852753) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:269 - timing_s/agent_loop/slowest/response_length:786 - timing_s/gen:12.095772627741098 - timing_s/reward:0.05289060063660145 - timing_s/old_log_prob:2.267318308353424 - timing_s/adv:0.5463232770562172 - timing_s/update_actor:14.966452691704035 - timing_s/step:30.0420327372849 - timing_s/stop_profile:4.320219159126282e-05 - timing_per_token_ms/gen:0.14683969004468764 - timing_per_token_ms/adv:0.002997329659605076 - timing_per_token_ms/update_actor:0.08211144286884312 - perf/total_num_tokens:182270 - perf/time_per_step:30.0420327372849 - perf/throughput:758.3957516870452 (TaskRunner pid=2167046) Training Progress: 94%|█████████▍| 94/100 [55:16<03:16, 32.83s/it] (TaskRunner pid=2167046) step:95 - global_seqlen/min:19988 - global_seqlen/max:25783 - global_seqlen/minmax_diff:5795 - global_seqlen/balanced_min:22801 - global_seqlen/balanced_max:22805 - global_seqlen/mean:22803.375 - actor/entropy:0.03662053495645523 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3295740783214569 - training/rollout_probs_diff_mean:0.0016202006954699755 - training/rollout_probs_diff_std:0.01027343887835741 - training/rollout_actor_probs_pearson_corr:0.9951478242874146 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.71875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0001161098480225 - rollout_corr/rollout_is_ratio_fraction_high:8.639948646305129e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001604561839485541 - rollout_corr/rollout_is_max:3.8464818000793457 - rollout_corr/rollout_is_min:0.3074188232421875 - rollout_corr/rollout_is_std:0.027437321841716766 - rollout_corr/rollout_is_eff_sample_size:0.999247997688126 - rollout_corr/rollout_is_seq_mean:1.0001269578933716 - rollout_corr/rollout_is_seq_std:0.0018558091251179576 - rollout_corr/rollout_is_seq_max:1.0104142427444458 - rollout_corr/rollout_is_seq_min:0.9959591627120972 - rollout_corr/rollout_is_seq_max_deviation:0.0104142427444458 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.037611531559378) - rollout_corr/training_log_ppl:np.float64(0.0367084229574175) - rollout_corr/kl:np.float64(0.00037698546704056923) - rollout_corr/k3_kl:np.float64(0.00048532903274178807) - rollout_corr/rollout_ppl:np.float64(1.0372191835194826) - rollout_corr/rollout_log_ppl:np.float64(0.036331437019725854) - rollout_corr/log_ppl_diff:np.float64(0.0003769859376916429) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013970075160614215) - rollout_corr/log_ppl_diff_max:np.float64(0.007364674936980009) - rollout_corr/log_ppl_diff_min:np.float64(-0.006315544247627258) - rollout_corr/ppl_ratio:np.float64(1.0003787786699831) - rollout_corr/chi2_token:np.float64(0.001227242173627019) - rollout_corr/chi2_seq:np.float64(0.30870523513294756) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(6.90859128940069e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_w_mean:np.float64(0.9999999955762178) - self_distillation/token_reweight_w_std:np.float64(0.0) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999999955762178) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.04232899000757017) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.19301223754883) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:3 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:316.48046875 - response_length/max:757.0 - response_length/min:145.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:316.48046875 - response_length_non_aborted/max:757.0 - response_length_non_aborted/min:145.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:396.125 - prompt_length/max:493.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010585971176624298 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7680115904659033) - timing_s/agent_loop/generate_sequences/max:np.float64(6.07424671202898) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.536307650647359) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.07424671202898) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:266 - timing_s/agent_loop/slowest/response_length:757 - timing_s/gen:11.979613646864891 - timing_s/reward:0.05839447304606438 - timing_s/old_log_prob:2.2557638846337795 - timing_s/adv:0.5291502270847559 - timing_s/update_actor:15.024496980011463 - timing_s/step:29.933929996564984 - timing_s/stop_profile:2.6324763894081116e-05 - timing_per_token_ms/gen:0.14786178114843299 - timing_per_token_ms/adv:0.0029006135445123577 - timing_per_token_ms/update_actor:0.08235895443115034 - perf/total_num_tokens:182427 - perf/time_per_step:29.933929996564984 - perf/throughput:761.7902160730904 (TaskRunner pid=2167046) Training Progress: 95%|█████████▌| 95/100 [55:46<02:39, 31.97s/it] (TaskRunner pid=2167046) step:96 - global_seqlen/min:19812 - global_seqlen/max:25392 - global_seqlen/minmax_diff:5580 - global_seqlen/balanced_min:22738 - global_seqlen/balanced_max:22741 - global_seqlen/mean:22739.75 - actor/entropy:0.0352025143802166 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.722224771976471 - training/rollout_probs_diff_mean:0.001679497305303812 - training/rollout_probs_diff_std:0.011236628517508507 - training/rollout_actor_probs_pearson_corr:0.993776261806488 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:1.0001745223999023 - rollout_corr/rollout_is_ratio_fraction_high:6.445458711823449e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011601825099205598 - rollout_corr/rollout_is_max:4.976936340332031 - rollout_corr/rollout_is_min:0.20165325701236725 - rollout_corr/rollout_is_std:0.02686212584376335 - rollout_corr/rollout_is_eff_sample_size:0.9992793035761552 - rollout_corr/rollout_is_seq_mean:1.0002118349075317 - rollout_corr/rollout_is_seq_std:0.0018694534664973617 - rollout_corr/rollout_is_seq_max:1.0106958150863647 - rollout_corr/rollout_is_seq_min:0.9955504536628723 - rollout_corr/rollout_is_seq_max_deviation:0.010695815086364746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0348785421811044) - rollout_corr/training_log_ppl:np.float64(0.03406564036595228) - rollout_corr/kl:np.float64(0.00037139787588214013) - rollout_corr/k3_kl:np.float64(0.0005362462779121557) - rollout_corr/rollout_ppl:np.float64(1.03448999626562) - rollout_corr/rollout_log_ppl:np.float64(0.03369424184438685) - rollout_corr/log_ppl_diff:np.float64(0.00037139852156542474) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014863652131680283) - rollout_corr/log_ppl_diff_max:np.float64(0.00833505392074585) - rollout_corr/log_ppl_diff_min:np.float64(-0.007462680339813232) - rollout_corr/ppl_ratio:np.float64(1.0003736605867743) - rollout_corr/chi2_token:np.float64(0.002083134837448597) - rollout_corr/chi2_seq:np.float64(1.332067912677303) - actor/pg_loss:np.float64(-0.00010274117812514305) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.768613795633428e-05) - actor/ppo_kl:np.float64(0.00016692980348764763) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(36289.31735826656) - self_distillation/token_reweight_w_std:np.float64(368548.66153580655) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000207382254303) - self_distillation/token_reweight_w_clip_frac:np.float64(0.001245500759978313) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.05526710581034422) - perf/mfu/actor:np.float64(0.041736199174072765) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.15214920043945) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:3 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0001611364568816498 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0001611364568816498 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:303.0234375 - response_length/max:593.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:303.0234375 - response_length_non_aborted/max:593.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:407.59375 - prompt_length/max:505.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.396299719810486e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.646346740424633) - timing_s/agent_loop/generate_sequences/max:np.float64(5.124028909951448) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0053514242972597) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.124028909951448) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:593 - timing_s/gen:10.97561901062727 - timing_s/reward:0.05890806205570698 - timing_s/old_log_prob:2.2199188880622387 - timing_s/adv:0.5234331879764795 - timing_s/update_actor:15.102608820423484 - timing_s/step:28.966348888352513 - timing_s/stop_profile:0.00011355988681316376 - timing_per_token_ms/gen:0.1414857943463953 - timing_per_token_ms/adv:0.002877302894581512 - timing_per_token_ms/update_actor:0.08301877120693656 - perf/total_num_tokens:181918 - perf/time_per_step:28.966348888352513 - perf/throughput:785.04025784015 (TaskRunner pid=2167046) Training Progress: 96%|█████████▌| 96/100 [56:15<02:04, 31.08s/it] (TaskRunner pid=2167046) step:97 - global_seqlen/min:18675 - global_seqlen/max:24360 - global_seqlen/minmax_diff:5685 - global_seqlen/balanced_min:21800 - global_seqlen/balanced_max:21804 - global_seqlen/mean:21803.5 - actor/entropy:0.041224244982004166 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2449004352092743 - training/rollout_probs_diff_mean:0.0014201895100995898 - training/rollout_probs_diff_std:0.0084342360496521 - training/rollout_actor_probs_pearson_corr:0.9970186352729797 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90625 - self_distillation/correct_sample_fraction:0.87890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90625 - rollout_corr/rollout_is_mean:1.000179409980774 - rollout_corr/rollout_is_ratio_fraction_high:3.7255973438732326e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.9674632464302704e-05 - rollout_corr/rollout_is_max:2.874107599258423 - rollout_corr/rollout_is_min:0.41924646496772766 - rollout_corr/rollout_is_std:0.02389836125075817 - rollout_corr/rollout_is_eff_sample_size:0.999429551518837 - rollout_corr/rollout_is_seq_mean:1.0001500844955444 - rollout_corr/rollout_is_seq_std:0.0014063200214877725 - rollout_corr/rollout_is_seq_max:1.004724144935608 - rollout_corr/rollout_is_seq_min:0.9953712821006775 - rollout_corr/rollout_is_seq_max_deviation:0.00472414493560791 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0406718435697258) - rollout_corr/training_log_ppl:np.float64(0.039368912926875055) - rollout_corr/kl:np.float64(0.00030021515657274733) - rollout_corr/k3_kl:np.float64(0.0004091687973215574) - rollout_corr/rollout_ppl:np.float64(1.040352996904403) - rollout_corr/rollout_log_ppl:np.float64(0.03906869714410277) - rollout_corr/log_ppl_diff:np.float64(0.000300215782772284) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011868714937008917) - rollout_corr/log_ppl_diff_max:np.float64(0.010238464921712875) - rollout_corr/log_ppl_diff_min:np.float64(-0.00530620664358139) - rollout_corr/ppl_ratio:np.float64(1.0003018721472472) - rollout_corr/chi2_token:np.float64(0.0010263379663228989) - rollout_corr/chi2_seq:np.float64(0.36499369889497757) - actor/pg_loss:np.float64(8.685840293765068e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010191214551014127) - actor/ppo_kl:np.float64(0.0001593927016365626) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.87890625) - self_distillation/token_reweight_w_mean:np.float64(37507.985115043586) - self_distillation/token_reweight_w_std:np.float64(574614.4153616531) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999924194999039) - self_distillation/token_reweight_w_clip_frac:np.float64(0.005771750904386863) - self_distillation/empty_target_batch:np.float64(0.09375) - actor/grad_norm:np.float64(0.10139033989980817) - perf/mfu/actor:np.float64(0.040042574424636254) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.15067863464355) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:3 - critic/score/mean:0.87890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.87890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00021422184363473207 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.00021422184363473207 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:314.546875 - response_length/max:600.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.546875 - response_length_non_aborted/max:600.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:366.8125 - prompt_length/max:490.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013898685574531555 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9727743286639452) - timing_s/agent_loop/generate_sequences/max:np.float64(5.126253275200725) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0250568900373764) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.126253275200725) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:579 - timing_s/gen:10.892221527174115 - timing_s/reward:0.05761564336717129 - timing_s/old_log_prob:2.2481269892305136 - timing_s/adv:0.5208297949284315 - timing_s/update_actor:15.106158940121531 - timing_s/step:28.910267505794764 - timing_s/stop_profile:0.0001242309808731079 - timing_per_token_ms/gen:0.1352667717348134 - timing_per_token_ms/adv:0.0029859299821613015 - timing_per_token_ms/update_actor:0.08660397952233317 - perf/total_num_tokens:174428 - perf/time_per_step:28.910267505794764 - perf/throughput:754.1784245209669 (TaskRunner pid=2167046) Training Progress: 97%|█████████▋| 97/100 [56:44<01:31, 30.45s/it] (TaskRunner pid=2167046) step:98 - global_seqlen/min:20744 - global_seqlen/max:24464 - global_seqlen/minmax_diff:3720 - global_seqlen/balanced_min:22195 - global_seqlen/balanced_max:22198 - global_seqlen/mean:22197.25 - actor/entropy:0.044722869992256165 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37975114583969116 - training/rollout_probs_diff_mean:0.0017690645763650537 - training/rollout_probs_diff_std:0.010033019818365574 - training/rollout_actor_probs_pearson_corr:0.9960561990737915 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.9375 - self_distillation/correct_sample_fraction:0.921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.9375 - rollout_corr/rollout_is_mean:1.0002034902572632 - rollout_corr/rollout_is_ratio_fraction_high:5.368695201468654e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.368695201468654e-05 - rollout_corr/rollout_is_max:4.066667556762695 - rollout_corr/rollout_is_min:0.3545277714729309 - rollout_corr/rollout_is_std:0.025945613160729408 - rollout_corr/rollout_is_eff_sample_size:0.9993276351484325 - rollout_corr/rollout_is_seq_mean:1.0001791715621948 - rollout_corr/rollout_is_seq_std:0.001851679291576147 - rollout_corr/rollout_is_seq_max:1.0125370025634766 - rollout_corr/rollout_is_seq_min:0.9958828091621399 - rollout_corr/rollout_is_seq_max_deviation:0.012537002563476562 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0432171327993274) - rollout_corr/training_log_ppl:np.float64(0.04202124249331973) - rollout_corr/kl:np.float64(0.00027291100581017247) - rollout_corr/k3_kl:np.float64(0.000395780640820409) - rollout_corr/rollout_ppl:np.float64(1.0429281839169562) - rollout_corr/rollout_log_ppl:np.float64(0.04174833083379781) - rollout_corr/log_ppl_diff:np.float64(0.00027291165952192387) - rollout_corr/log_ppl_abs_diff:np.float64(0.001402678571139404) - rollout_corr/log_ppl_diff_max:np.float64(0.005851060152053833) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076254382729530334) - rollout_corr/ppl_ratio:np.float64(1.0002746109385043) - rollout_corr/chi2_token:np.float64(0.0011082068085670471) - rollout_corr/chi2_seq:np.float64(0.39986567525193095) - actor/pg_loss:np.float64(0.00015399395488202572) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.233245085240924e-05) - actor/ppo_kl:np.float64(8.391451634537361e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.921875) - self_distillation/token_reweight_w_mean:np.float64(28126.951660413528) - self_distillation/token_reweight_w_std:np.float64(436973.87273890217) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999532324727625) - self_distillation/token_reweight_w_clip_frac:np.float64(0.004359587779617868) - self_distillation/empty_target_batch:np.float64(0.0625) - actor/grad_norm:np.float64(0.07238347781822085) - perf/mfu/actor:np.float64(0.04097528863884095) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.1676902770996) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:3 - critic/score/mean:0.921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017314042197540402 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017314042197540402 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:291.0390625 - response_length/max:678.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.0390625 - response_length_non_aborted/max:678.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:402.625 - prompt_length/max:486.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.483657777309418e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.659082705155015) - timing_s/agent_loop/generate_sequences/max:np.float64(5.483136726543307) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.182413850197918) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.483136726543307) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:678 - timing_s/gen:11.112114802002907 - timing_s/reward:0.05823834612965584 - timing_s/old_log_prob:2.2256915345788 - timing_s/adv:0.5277259200811386 - timing_s/update_actor:15.02241063863039 - timing_s/step:29.03360358811915 - timing_s/stop_profile:0.00011938810348510742 - timing_per_token_ms/gen:0.1491438917939885 - timing_per_token_ms/adv:0.0029717978582996687 - timing_per_token_ms/update_actor:0.08459612473746968 - perf/total_num_tokens:177578 - perf/time_per_step:29.03360358811915 - perf/throughput:764.5365113782618 (TaskRunner pid=2167046) Training Progress: 98%|█████████▊| 98/100 [57:13<01:00, 30.04s/it] (TaskRunner pid=2167046) step:99 - global_seqlen/min:18553 - global_seqlen/max:24227 - global_seqlen/minmax_diff:5674 - global_seqlen/balanced_min:21762 - global_seqlen/balanced_max:21765 - global_seqlen/mean:21764.375 - actor/entropy:0.05101349577307701 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7172134518623352 - training/rollout_probs_diff_mean:0.002091911854222417 - training/rollout_probs_diff_std:0.011637764982879162 - training/rollout_actor_probs_pearson_corr:0.9952604174613953 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.7578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9999338984489441 - rollout_corr/rollout_is_ratio_fraction_high:4.100433216081001e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012301300012040883 - rollout_corr/rollout_is_max:2.5501413345336914 - rollout_corr/rollout_is_min:0.23693661391735077 - rollout_corr/rollout_is_std:0.02900037169456482 - rollout_corr/rollout_is_eff_sample_size:0.9991595661760642 - rollout_corr/rollout_is_seq_mean:0.9999666213989258 - rollout_corr/rollout_is_seq_std:0.0020682818721979856 - rollout_corr/rollout_is_seq_max:1.0070600509643555 - rollout_corr/rollout_is_seq_min:0.9947633743286133 - rollout_corr/rollout_is_seq_max_deviation:0.007060050964355469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0553245856426656) - rollout_corr/training_log_ppl:np.float64(0.0528352315795928) - rollout_corr/kl:np.float64(0.0008055590208186914) - rollout_corr/k3_kl:np.float64(0.0006251534139227743) - rollout_corr/rollout_ppl:np.float64(1.054449059534818) - rollout_corr/rollout_log_ppl:np.float64(0.052029672435310204) - rollout_corr/log_ppl_diff:np.float64(0.0008055591442825971) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018310855939489556) - rollout_corr/log_ppl_diff_max:np.float64(0.018006175756454468) - rollout_corr/log_ppl_diff_min:np.float64(-0.006862897425889969) - rollout_corr/ppl_ratio:np.float64(1.000809220597148) - rollout_corr/chi2_token:np.float64(0.0009292459581047297) - rollout_corr/chi2_seq:np.float64(0.2774903466925025) - actor/pg_loss:np.float64(0.0008074860088527203) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002502697416275623) - actor/ppo_kl:np.float64(0.0002938043922213751) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_w_mean:np.float64(147148.14364791103) - self_distillation/token_reweight_w_std:np.float64(1759607.3955868636) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003166827373207) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014095777121838182) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.20968144107609987) - perf/mfu/actor:np.float64(0.0402602080853634) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.147705078125) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:3 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001307013095356524 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.001307013095356524 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:285.79296875 - response_length/max:605.0 - response_length/min:136.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:285.79296875 - response_length_non_aborted/max:605.0 - response_length_non_aborted/min:136.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:394.34375 - prompt_length/max:480.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.2334503531456e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6478356011211872) - timing_s/agent_loop/generate_sequences/max:np.float64(5.1489850506186485) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2426595673314296) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.1489850506186485) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:281 - timing_s/agent_loop/slowest/response_length:605 - timing_s/gen:11.22121479921043 - timing_s/reward:0.05911407247185707 - timing_s/old_log_prob:2.2879777904599905 - timing_s/adv:0.5154732447117567 - timing_s/update_actor:15.057282112538815 - timing_s/step:29.228888789191842 - timing_s/stop_profile:0.00011774897575378418 - timing_per_token_ms/gen:0.15337280864932315 - timing_per_token_ms/adv:0.002960533237870124 - timing_per_token_ms/update_actor:0.0864789484681895 - perf/total_num_tokens:174115 - perf/time_per_step:29.228888789191842 - perf/throughput:744.6186256676291 (TaskRunner pid=2167046) Training Progress: 99%|█████████▉| 99/100 [57:43<00:29, 29.81s/it] (TaskRunner pid=2167046) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2167046) validation generation end (TaskRunner pid=2167046) [prompt] system (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) ... (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2167046) (TaskRunner pid=2167046) A (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) user (TaskRunner pid=2167046) Given the following crystal structure parameters for the material RbLa9(IrO6)4 (Material ID: mp-560657), calculate the volume of the unit cell (in ų). (TaskRunner pid=2167046) (TaskRunner pid=2167046) Lattice Parameters: (TaskRunner pid=2167046) (TaskRunner pid=2167046) a = 7.82 Å (TaskRunner pid=2167046) b = 7.82 Å (TaskRunner pid=2167046) c = 17.88 Å (TaskRunner pid=2167046) α = 90.00º (TaskRunner pid=2167046) β = 90.00º (TaskRunner pid=2167046) γ = 90.00º (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) A: 1025.67 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) B: 1094.31 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) C: 1200.45 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) D: 1150.78 ų (TaskRunner pid=2167046) (TaskRunner pid=2167046) Please reason step by step. (TaskRunner pid=2167046) assistant (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) [response] (TaskRunner pid=2167046) The volume $ V $ of a unit cell in a crystal with given lattice parameters $ a, b, c $ and angles $ \alpha, \beta, \gamma $ is calculated using the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = abc \sqrt{1 + 2\cos\alpha\cos\beta\cos\gamma - \cos^2\alpha - \cos^2\beta - \cos^2\gamma} (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Given: (TaskRunner pid=2167046) - $ a = 7.82 $ Å (TaskRunner pid=2167046) - $ b = 7.82 $ Å (TaskRunner pid=2167046) - $ c = 17.88 $ Å (TaskRunner pid=2167046) - $ \alpha = 90.00^\circ $ (TaskRunner pid=2167046) - $ \beta = 90.00^\circ $ (TaskRunner pid=2167046) - $ \gamma = 90.00^\circ $ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Since all angles are 90°, their cosines are 0. Substituting into the formula: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = abc \sqrt{1 + 2(0)(0)(0) - (0)^2 - (0)^2 - (0)^2} = abc \sqrt{1} = abc (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Now calculate: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) V = 7.82 \times 7.82 \times 17.88 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) First, calculate $ 7.82 \times 7.82 $: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 7.82^2 = 61.1524 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Then multiply by 17.88: (TaskRunner pid=2167046) (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) 61.1524 \times 17.88 \approx 1094.31 (TaskRunner pid=2167046) $$ (TaskRunner pid=2167046) (TaskRunner pid=2167046) Thus, the volume is approximately 1094.31 ų. (TaskRunner pid=2167046) (TaskRunner pid=2167046) (TaskRunner pid=2167046) B (TaskRunner pid=2167046) (TaskRunner pid=2167046) [ground_truth] B (TaskRunner pid=2167046) [score] 1.0 (TaskRunner pid=2167046) [acc] 1.0 (TaskRunner pid=2167046) [pred] B (TaskRunner pid=2167046) [incorrect_format] 1 (TaskRunner pid=2167046) [feedback] (TaskRunner pid=2167046) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2167440) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/material/qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2167046) step:100 - global_seqlen/min:18525 - global_seqlen/max:24595 - global_seqlen/minmax_diff:6070 - global_seqlen/balanced_min:21455 - global_seqlen/balanced_max:21459 - global_seqlen/mean:21457.25 - actor/entropy:0.052000705152750015 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163344740867615 - training/rollout_probs_diff_mean:0.00194946164265275 - training/rollout_probs_diff_std:0.010225724428892136 - training/rollout_actor_probs_pearson_corr:0.9965116381645203 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:0.9999290108680725 - rollout_corr/rollout_is_ratio_fraction_high:1.3980147741676774e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.990074325585738e-05 - rollout_corr/rollout_is_max:2.706778049468994 - rollout_corr/rollout_is_min:0.4680381119251251 - rollout_corr/rollout_is_std:0.027313215658068657 - rollout_corr/rollout_is_eff_sample_size:0.9992544253530163 - rollout_corr/rollout_is_seq_mean:0.9999463558197021 - rollout_corr/rollout_is_seq_std:0.0017657832941040397 - rollout_corr/rollout_is_seq_max:1.0095514059066772 - rollout_corr/rollout_is_seq_min:0.996039092540741 - rollout_corr/rollout_is_seq_max_deviation:0.009551405906677246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.056842735502869) - rollout_corr/training_log_ppl:np.float64(0.053712156926849275) - rollout_corr/kl:np.float64(0.0006709553122909995) - rollout_corr/k3_kl:np.float64(0.0005436672919252317) - rollout_corr/rollout_ppl:np.float64(1.0561046437360346) - rollout_corr/rollout_log_ppl:np.float64(0.05304120097935083) - rollout_corr/log_ppl_diff:np.float64(0.0006709559474984417) - rollout_corr/log_ppl_abs_diff:np.float64(0.001628608099053963) - rollout_corr/log_ppl_diff_max:np.float64(0.01552504301071167) - rollout_corr/log_ppl_diff_min:np.float64(-0.005156993865966797) - rollout_corr/ppl_ratio:np.float64(1.0006739916279912) - rollout_corr/chi2_token:np.float64(0.0007216199301183224) - rollout_corr/chi2_seq:np.float64(0.15093162842094898) - actor/pg_loss:np.float64(0.0006925989873707294) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00034995409805560485) - actor/ppo_kl:np.float64(0.00022765648916411863) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(32783.957445921376) - self_distillation/token_reweight_w_std:np.float64(431654.97193862277) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002633819822222) - self_distillation/token_reweight_w_clip_frac:np.float64(0.012708691239822656) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.21034935116767883) - perf/mfu/actor:np.float64(0.03965654144334235) - perf/max_memory_allocated_gb:np.float64(117.12247371673584) - perf/max_memory_reserved_gb:np.float64(121.533203125) - perf/cpu_memory_used_gb:np.float64(147.16178703308105) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.7686170212765957) - val-aux/sciknoweval/reward/std@16:np.float64(0.04264975776630575) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7854148936170213) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.02918137925697454) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.7512340425531915) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.042424831149873776) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.7684468085106384) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.04273575303351148) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7959787234042554) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.01639341752615574) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.7343617021276596) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.03784947100976082) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7722765957446808) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.031426065862592674) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.801563829787234) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.008774628971156027) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.7189574468085107) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.03147132698227977) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7746063829787234) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.021830599810403712) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8047021276595744) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.005902094678345516) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.705404255319149) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.02298913016887447) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7758829787234042) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.014657220600786369) - val-aux/sciknoweval/score/mean@16:np.float64(0.7686170212765957) - val-aux/sciknoweval/score/std@16:np.float64(0.04264975776630575) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7854148936170213) - val-aux/sciknoweval/score/best@2/std:np.float64(0.02918137925697454) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.7512340425531915) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.042424831149873776) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.7684468085106384) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.04273575303351148) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7959787234042554) - val-aux/sciknoweval/score/best@4/std:np.float64(0.01639341752615574) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.7343617021276596) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.03784947100976082) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7722765957446808) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.031426065862592674) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.801563829787234) - val-aux/sciknoweval/score/best@8/std:np.float64(0.008774628971156027) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.7189574468085107) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.03147132698227977) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7746063829787234) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.021830599810403712) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8047021276595744) - val-aux/sciknoweval/score/best@16/std:np.float64(0.005902094678345516) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.705404255319149) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.02298913016887447) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7758829787234042) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.014657220600786369) - val-core/sciknoweval/acc/mean@16:np.float64(0.7686170212765957) - val-aux/sciknoweval/acc/std@16:np.float64(0.04264975776630575) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7854148936170213) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.02918137925697454) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.7512340425531915) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.042424831149873776) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.7684468085106384) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.04273575303351148) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7959787234042554) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.01639341752615574) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.7343617021276596) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.03784947100976082) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7722765957446808) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.031426065862592674) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.801563829787234) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.008774628971156027) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.7189574468085107) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.03147132698227977) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7746063829787234) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.021830599810403712) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8047021276595744) - val-core/sciknoweval/acc/best@16/std:np.float64(0.005902094678345516) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.705404255319149) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.02298913016887447) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7758829787234042) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.014657220600786369) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:3 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0022892493288964033 - critic/advantages/max:0.75 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0022892493288964033 - critic/returns/max:0.75 - critic/returns/min:-0.625 - response_length/mean:279.4140625 - response_length/max:589.0 - response_length/min:91.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:279.4140625 - response_length_non_aborted/max:589.0 - response_length_non_aborted/min:91.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:391.125 - prompt_length/max:479.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.36830586194992e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.194736858829856) - timing_s/agent_loop/generate_sequences/max:np.float64(4.990295339375734) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1610305702342885) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.990295339375734) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:589 - timing_s/gen:10.843061864376068 - timing_s/reward:0.05821913294494152 - timing_s/old_log_prob:2.2547264005988836 - timing_s/adv:0.5003067050129175 - timing_s/update_actor:14.95605488307774 - timing_s/step:28.696759099140763 - timing_s/testing:36.83901543915272 - timing_s/save_checkpoint:6.8152069840580225 - timing_s/stop_profile:9.810179471969604e-05 - timing_per_token_ms/gen:0.15158761169266138 - timing_per_token_ms/adv:0.0029145551329557463 - timing_per_token_ms/update_actor:0.08712704845144265 - perf/total_num_tokens:171658 - perf/time_per_step:28.696759099140763 - perf/throughput:747.7238083182178 (TaskRunner pid=2167046) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2167046) "np.float64(0.7686170212765957), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2167046) "np.float64(0.04264975776630575), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2167046) "np.float64(0.7854148936170213), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2167046) "np.float64(0.02918137925697454), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2167046) "np.float64(0.7512340425531915), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2167046) "np.float64(0.042424831149873776), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2167046) "np.float64(0.7684468085106384), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2167046) "np.float64(0.04273575303351148), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2167046) "np.float64(0.7959787234042554), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2167046) "np.float64(0.01639341752615574), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2167046) "np.float64(0.7343617021276596), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2167046) "np.float64(0.03784947100976082), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2167046) "np.float64(0.7722765957446808), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2167046) "np.float64(0.031426065862592674), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2167046) "np.float64(0.801563829787234), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2167046) "np.float64(0.008774628971156027), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2167046) "np.float64(0.7189574468085107), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2167046) "np.float64(0.03147132698227977), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2167046) "np.float64(0.7746063829787234), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2167046) "np.float64(0.021830599810403712), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2167046) "np.float64(0.8047021276595744), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2167046) 'np.float64(0.005902094678345516), ' (TaskRunner pid=2167046) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.705404255319149), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.02298913016887447), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7758829787234042), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.014657220600786369), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/mean@16': np.float64(0.7686170212765957), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/std@16': np.float64(0.04264975776630575), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7854148936170213), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.02918137925697454), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.7512340425531915), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.042424831149873776), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.7684468085106384), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.04273575303351148), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.7959787234042554), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.01639341752615574), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.7343617021276596), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.03784947100976082), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7722765957446808), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.031426065862592674), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.801563829787234), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.008774628971156027), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.7189574468085107), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.03147132698227977), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.7746063829787234), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.021830599810403712), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8047021276595744), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.005902094678345516), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.705404255319149), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.02298913016887447), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7758829787234042), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.014657220600786369), " (TaskRunner pid=2167046) "'val-core/sciknoweval/acc/mean@16': np.float64(0.7686170212765957), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/std@16': np.float64(0.04264975776630575), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7854148936170213), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.02918137925697454), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.7512340425531915), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.042424831149873776), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.7684468085106384), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.04273575303351148), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.7959787234042554), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.01639341752615574), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.7343617021276596), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.03784947100976082), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7722765957446808), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.031426065862592674), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.801563829787234), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.008774628971156027), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.7189574468085107), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.03147132698227977), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.7746063829787234), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.021830599810403712), " (TaskRunner pid=2167046) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8047021276595744), " (TaskRunner pid=2167046) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.005902094678345516), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.705404255319149), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.02298913016887447), " (TaskRunner pid=2167046) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7758829787234042), " (TaskRunner pid=2167046) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.014657220600786369), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/std@16': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/best@2/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/worst@2/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/maj@2/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/worst@4/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/maj@4/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/worst@8/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/worst@16/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2167046) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2167046) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2167046) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2167046) Training Progress: 100%|██████████| 100/100 [58:55<00:00, 42.59s/it] Training Progress: 100%|██████████| 100/100 [58:55<00:00, 35.36s/it] (TaskRunner pid=2167046) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2167046) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2167046) wandb: uploading history steps 98-99, summary, console lines 989-1097 (TaskRunner pid=2167046) wandb: (TaskRunner pid=2167046) wandb: Run history: (TaskRunner pid=2167046) wandb: actor/entropy ▅▆▄▄▃▅█▅▇▆▆▇▆▆▆▇█▆█▆█▇▆▆▅▆▆▄▅▅▅▄▃▃▁▃▃▃▅▇ (TaskRunner pid=2167046) wandb: actor/grad_norm ▇▅▅█▇▇▃▅▄▄▄▄▄▃▃▂▄▄▂▂▃▃▄▃▂▂▂▂▄▂▅▁▂▁▂▄▁▃▁▃ (TaskRunner pid=2167046) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2167046) wandb: actor/lr ▁▂▅▆████████████████████████████████████ (TaskRunner pid=2167046) wandb: actor/pg_clipfrac ▁█▄▄▆▄▆▄▂▄▃▄▆▂▂▄▃▅▃▃▃▅▆▁▃▄▂▅▂▁▃▂▂▂▂▃▃▁▂▂ (TaskRunner pid=2167046) wandb: actor/pg_clipfrac_lower ▁▁▁▇▁▁▁▁▄▁▁▁▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2167046) wandb: actor/pg_loss ▃▃█▆▄▅▃▅▄▅▃▄▃▂▂▅▄▂▂▃▂▃▃▂▂▃▃▄▄▃▁▃▁▂▃▄▁▁▁▃ (TaskRunner pid=2167046) wandb: actor/ppo_kl ▂▂▂▂▂▂▃▃▃▂▁▂▂▂▂▃▄▂▃▁▄▃▂▃▂▂▂▂▂▄▂▃▁▃▂▄▁█▂▃ (TaskRunner pid=2167046) wandb: critic/advantages/max ██▇██▇██▅▇█▅█▅██▄▅██▆█▇█▇▄▇▇████▃▇█▆██▁▇ (TaskRunner pid=2167046) wandb: critic/advantages/mean ▃▄█▄▃▆▄▄▄▅▁▃▄▅▄▄▃▂▃▅▄▄▅▃▄▃▄▅▄▄▄▄▃▄▄▄▄▃▄▃ (TaskRunner pid=2167046) wandb: +220 ... (TaskRunner pid=2167046) wandb: (TaskRunner pid=2167046) wandb: Run summary: (TaskRunner pid=2167046) wandb: actor/entropy 0.052 (TaskRunner pid=2167046) wandb: actor/grad_norm 0.21035 (TaskRunner pid=2167046) wandb: actor/kl_loss 0 (TaskRunner pid=2167046) wandb: actor/lr 0.0 (TaskRunner pid=2167046) wandb: actor/pg_clipfrac 0.00035 (TaskRunner pid=2167046) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2167046) wandb: actor/pg_loss 0.00069 (TaskRunner pid=2167046) wandb: actor/ppo_kl 0.00023 (TaskRunner pid=2167046) wandb: critic/advantages/max 0.75 (TaskRunner pid=2167046) wandb: critic/advantages/mean -0.00229 (TaskRunner pid=2167046) wandb: +220 ... (TaskRunner pid=2167046) wandb: (TaskRunner pid=2167046) wandb: 🚀 View run qwen3gen-material-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/jd6eugui (TaskRunner pid=2167046) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2167046) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2167046) wandb: Find logs at: ./wandb/run-20260702_135403-jd6eugui/logs main_ppo exit code: 0 [2026-07-02 14:53:06] Finished material rlsd_tr [2026-07-02 14:53:06] Starting tooluse grpo Experiment: qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: tooluse Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/tooluse/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/tooluse/test.parquet Ray tmp: /tmp/ray_q3g_tooluse_grpo_Qwen3_4B 2026-07-02 14:53:08,626 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/tooluse Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/tooluse/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/tooluse/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_tooluse_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/tooluse +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_tooluse_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/tooluse', 'EXPERIMENT': 'qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 14:53:19,372 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2186821) TaskRunner hostname: mole-workspace-rw, PID: 2186821 (TaskRunner pid=2186821) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2186821) 'calculate_entropy': False, (TaskRunner pid=2186821) 'calculate_sum_pi_squared': False, (TaskRunner pid=2186821) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2186821) 'async_save': False, (TaskRunner pid=2186821) 'load_contents': ['model', (TaskRunner pid=2186821) 'optimizer', (TaskRunner pid=2186821) 'extra'], (TaskRunner pid=2186821) 'save_contents': ['model', (TaskRunner pid=2186821) 'optimizer', (TaskRunner pid=2186821) 'extra']}, (TaskRunner pid=2186821) 'clip_ratio': 0.2, (TaskRunner pid=2186821) 'clip_ratio_c': 3.0, (TaskRunner pid=2186821) 'clip_ratio_high': 0.28, (TaskRunner pid=2186821) 'clip_ratio_low': 0.2, (TaskRunner pid=2186821) 'data_loader_seed': 42, (TaskRunner pid=2186821) 'entropy_checkpointing': False, (TaskRunner pid=2186821) 'entropy_coeff': 0, (TaskRunner pid=2186821) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2186821) 'freeze_vision_tower': False, (TaskRunner pid=2186821) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2186821) 'dtype': 'bfloat16', (TaskRunner pid=2186821) 'entropy_checkpointing': False, (TaskRunner pid=2186821) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2186821) 'forward_only': False, (TaskRunner pid=2186821) 'forward_prefetch': False, (TaskRunner pid=2186821) 'fsdp_size': -1, (TaskRunner pid=2186821) 'full_determinism': False, (TaskRunner pid=2186821) 'model_dtype': 'fp32', (TaskRunner pid=2186821) 'offload_policy': False, (TaskRunner pid=2186821) 'optimizer_offload': False, (TaskRunner pid=2186821) 'param_offload': False, (TaskRunner pid=2186821) 'reshard_after_forward': True, (TaskRunner pid=2186821) 'seed': 42, (TaskRunner pid=2186821) 'strategy': 'fsdp', (TaskRunner pid=2186821) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2186821) 'use_orig_params': False, (TaskRunner pid=2186821) 'use_torch_compile': True, (TaskRunner pid=2186821) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2186821) 'grad_clip': 1.0, (TaskRunner pid=2186821) 'kl_loss_coef': 0.0, (TaskRunner pid=2186821) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2186821) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2186821) 'loss_scale_factor': None, (TaskRunner pid=2186821) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2186821) 'betas': [0.9, 0.999], (TaskRunner pid=2186821) 'clip_grad': 1.0, (TaskRunner pid=2186821) 'lr': 1e-06, (TaskRunner pid=2186821) 'lr_scheduler_type': 'constant', (TaskRunner pid=2186821) 'lr_warmup_steps': 10, (TaskRunner pid=2186821) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2186821) 'min_lr_ratio': 0.0, (TaskRunner pid=2186821) 'num_cycles': 0.5, (TaskRunner pid=2186821) 'optimizer': 'AdamW', (TaskRunner pid=2186821) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2186821) 'override_optimizer_config': None, (TaskRunner pid=2186821) 'total_training_steps': -1, (TaskRunner pid=2186821) 'warmup_style': None, (TaskRunner pid=2186821) 'weight_decay': 0.01}, (TaskRunner pid=2186821) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2186821) 'clip_cov_lb': 1.0, (TaskRunner pid=2186821) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2186821) 'clip_cov_ub': 5.0, (TaskRunner pid=2186821) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2186821) 'loss_mode': 'vanilla', (TaskRunner pid=2186821) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2186821) 'ppo_epochs': 1, (TaskRunner pid=2186821) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2186821) 'ppo_micro_batch_size': None, (TaskRunner pid=2186821) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2186821) 'ppo_mini_batch_size': 8, (TaskRunner pid=2186821) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2186821) 'all_ranks': False, (TaskRunner pid=2186821) 'enable': False, (TaskRunner pid=2186821) 'ranks': [], (TaskRunner pid=2186821) 'save_path': 'outputs/profile', (TaskRunner pid=2186821) 'tool': None, (TaskRunner pid=2186821) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2186821) 'analysis': True, (TaskRunner pid=2186821) 'contents': [], (TaskRunner pid=2186821) 'discrete': False, (TaskRunner pid=2186821) 'level': 'level0'}, (TaskRunner pid=2186821) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2186821) 'discrete': False}, (TaskRunner pid=2186821) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2186821) 'step_end': None, (TaskRunner pid=2186821) 'step_start': 0}, (TaskRunner pid=2186821) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2186821) 'stack_depth': 32, (TaskRunner pid=2186821) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2186821) 'rollout_n': 8, (TaskRunner pid=2186821) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2186821) 'mode': 'disabled', (TaskRunner pid=2186821) 'record_file': None, (TaskRunner pid=2186821) 'replay_file': None}, (TaskRunner pid=2186821) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2186821) 'alpha': 0.5, (TaskRunner pid=2186821) 'distillation_add_tail': True, (TaskRunner pid=2186821) 'distillation_topk': 100, (TaskRunner pid=2186821) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2186821) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2186821) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2186821) 'enable': False, (TaskRunner pid=2186821) 'loss_weight': 0.0, (TaskRunner pid=2186821) 'mask': 'all'}, (TaskRunner pid=2186821) 'feedback_template': '\n' (TaskRunner pid=2186821) 'The ' (TaskRunner pid=2186821) 'following ' (TaskRunner pid=2186821) 'is ' (TaskRunner pid=2186821) 'feedback ' (TaskRunner pid=2186821) 'from ' (TaskRunner pid=2186821) 'your ' (TaskRunner pid=2186821) 'unsuccessful ' (TaskRunner pid=2186821) 'earlier ' (TaskRunner pid=2186821) 'attempt:\n' (TaskRunner pid=2186821) '\n' (TaskRunner pid=2186821) '{feedback_raw}', (TaskRunner pid=2186821) 'full_logit_distillation': True, (TaskRunner pid=2186821) 'include_environment_feedback': True, (TaskRunner pid=2186821) 'is_clip': 2, (TaskRunner pid=2186821) 'max_reprompt_len': 22528, (TaskRunner pid=2186821) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2186821) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2186821) '\n' (TaskRunner pid=2186821) 'Correctly ' (TaskRunner pid=2186821) 'solve ' (TaskRunner pid=2186821) 'the ' (TaskRunner pid=2186821) 'original ' (TaskRunner pid=2186821) 'question.', (TaskRunner pid=2186821) 'reprompt_truncation': 'right', (TaskRunner pid=2186821) 'solution_template': '\n' (TaskRunner pid=2186821) 'Correct ' (TaskRunner pid=2186821) 'solution:\n' (TaskRunner pid=2186821) '\n' (TaskRunner pid=2186821) '{successful_previous_attempt}', (TaskRunner pid=2186821) 'srpo_dynamic_weighting': False, (TaskRunner pid=2186821) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2186821) 'success_reward_threshold': 0.5, (TaskRunner pid=2186821) 'teacher_regularization': 'ema', (TaskRunner pid=2186821) 'teacher_update_rate': 0.0, (TaskRunner pid=2186821) 'token_reweight_decay_steps': None, (TaskRunner pid=2186821) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2186821) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2186821) 'shuffle': False, (TaskRunner pid=2186821) 'strategy': 'fsdp', (TaskRunner pid=2186821) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2186821) 'tau_neg': 1.05, (TaskRunner pid=2186821) 'tau_pos': 1.0, (TaskRunner pid=2186821) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2186821) 'use_dynamic_bsz': False, (TaskRunner pid=2186821) 'use_fused_kernels': False, (TaskRunner pid=2186821) 'use_kl_loss': False, (TaskRunner pid=2186821) 'use_prefix_grouper': False, (TaskRunner pid=2186821) 'use_remove_padding': True, (TaskRunner pid=2186821) 'use_torch_compile': True}, (TaskRunner pid=2186821) 'hybrid_engine': True, (TaskRunner pid=2186821) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2186821) 'custom_chat_template': None, (TaskRunner pid=2186821) 'enable_activation_offload': False, (TaskRunner pid=2186821) 'enable_gradient_checkpointing': True, (TaskRunner pid=2186821) 'exclude_modules': None, (TaskRunner pid=2186821) 'external_lib': None, (TaskRunner pid=2186821) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2186821) 'hf_config_path': None, (TaskRunner pid=2186821) 'lora_adapter_path': None, (TaskRunner pid=2186821) 'lora_alpha': 16, (TaskRunner pid=2186821) 'lora_rank': 0, (TaskRunner pid=2186821) 'override_config': {}, (TaskRunner pid=2186821) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2186821) 'target_modules': 'all-linear', (TaskRunner pid=2186821) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2186821) 'num_shards': 4}, (TaskRunner pid=2186821) 'tokenizer_path': None, (TaskRunner pid=2186821) 'trust_remote_code': True, (TaskRunner pid=2186821) 'use_fused_kernels': False, (TaskRunner pid=2186821) 'use_liger': False, (TaskRunner pid=2186821) 'use_remove_padding': True, (TaskRunner pid=2186821) 'use_shm': False}, (TaskRunner pid=2186821) 'nccl_timeout': 600, (TaskRunner pid=2186821) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2186821) 'entropy_checkpointing': False, (TaskRunner pid=2186821) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2186821) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2186821) 'dtype': 'bfloat16', (TaskRunner pid=2186821) 'entropy_checkpointing': False, (TaskRunner pid=2186821) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2186821) 'forward_only': True, (TaskRunner pid=2186821) 'forward_prefetch': False, (TaskRunner pid=2186821) 'fsdp_size': -1, (TaskRunner pid=2186821) 'full_determinism': False, (TaskRunner pid=2186821) 'model_dtype': 'fp32', (TaskRunner pid=2186821) 'offload_policy': False, (TaskRunner pid=2186821) 'optimizer_offload': False, (TaskRunner pid=2186821) 'param_offload': False, (TaskRunner pid=2186821) 'reshard_after_forward': True, (TaskRunner pid=2186821) 'seed': 42, (TaskRunner pid=2186821) 'strategy': 'fsdp', (TaskRunner pid=2186821) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2186821) 'use_orig_params': False, (TaskRunner pid=2186821) 'use_torch_compile': True, (TaskRunner pid=2186821) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2186821) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2186821) 'log_prob_micro_batch_size': None, (TaskRunner pid=2186821) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2186821) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2186821) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2186821) 'all_ranks': False, (TaskRunner pid=2186821) 'enable': False, (TaskRunner pid=2186821) 'ranks': [], (TaskRunner pid=2186821) 'save_path': 'outputs/profile', (TaskRunner pid=2186821) 'tool': None, (TaskRunner pid=2186821) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2186821) 'analysis': True, (TaskRunner pid=2186821) 'contents': [], (TaskRunner pid=2186821) 'discrete': False, (TaskRunner pid=2186821) 'level': 'level0'}, (TaskRunner pid=2186821) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2186821) 'discrete': False}, (TaskRunner pid=2186821) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2186821) 'step_end': None, (TaskRunner pid=2186821) 'step_start': 0}, (TaskRunner pid=2186821) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2186821) 'stack_depth': 32, (TaskRunner pid=2186821) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2186821) 'rollout_n': 8, (TaskRunner pid=2186821) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2186821) 'mode': 'disabled', (TaskRunner pid=2186821) 'record_file': None, (TaskRunner pid=2186821) 'replay_file': None}, (TaskRunner pid=2186821) 'strategy': 'fsdp', (TaskRunner pid=2186821) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2186821) 'use_torch_compile': True}, (TaskRunner pid=2186821) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2186821) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2186821) 'agent_loop_config_path': None, (TaskRunner pid=2186821) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2186821) 'name': None, (TaskRunner pid=2186821) 'path': None}, (TaskRunner pid=2186821) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2186821) 'num_workers': 8}, (TaskRunner pid=2186821) 'calculate_log_probs': True, (TaskRunner pid=2186821) 'cudagraph_capture_sizes': None, (TaskRunner pid=2186821) 'data_parallel_size': 1, (TaskRunner pid=2186821) 'disable_log_stats': True, (TaskRunner pid=2186821) 'do_sample': True, (TaskRunner pid=2186821) 'dtype': 'bfloat16', (TaskRunner pid=2186821) 'enable_chunked_prefill': True, (TaskRunner pid=2186821) 'enable_prefix_caching': True, (TaskRunner pid=2186821) 'enable_rollout_routing_replay': False, (TaskRunner pid=2186821) 'enforce_eager': False, (TaskRunner pid=2186821) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2186821) 'expert_parallel_size': 1, (TaskRunner pid=2186821) 'free_cache_engine': True, (TaskRunner pid=2186821) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2186821) 'ignore_eos': False, (TaskRunner pid=2186821) 'layered_summon': False, (TaskRunner pid=2186821) 'load_format': 'dummy', (TaskRunner pid=2186821) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2186821) 'log_prob_micro_batch_size': None, (TaskRunner pid=2186821) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2186821) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2186821) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2186821) 'max_model_len': 10240, (TaskRunner pid=2186821) 'max_num_batched_tokens': 10240, (TaskRunner pid=2186821) 'max_num_seqs': 1024, (TaskRunner pid=2186821) 'mode': 'async', (TaskRunner pid=2186821) 'multi_stage_wake_up': False, (TaskRunner pid=2186821) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2186821) 'enable': False, (TaskRunner pid=2186821) 'format': 'hermes', (TaskRunner pid=2186821) 'interaction_config_path': None, (TaskRunner pid=2186821) 'max_assistant_turns': None, (TaskRunner pid=2186821) 'max_parallel_calls': 1, (TaskRunner pid=2186821) 'max_tool_response_length': 256, (TaskRunner pid=2186821) 'max_user_turns': None, (TaskRunner pid=2186821) 'num_repeat_rollouts': None, (TaskRunner pid=2186821) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2186821) 'tool_config_path': None, (TaskRunner pid=2186821) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2186821) 'use_inference_chat_template': False}, (TaskRunner pid=2186821) 'n': 8, (TaskRunner pid=2186821) 'name': 'vllm', (TaskRunner pid=2186821) 'over_sample_rate': 0, (TaskRunner pid=2186821) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2186821) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2186821) 'all_ranks': False, (TaskRunner pid=2186821) 'enable': False, (TaskRunner pid=2186821) 'ranks': [], (TaskRunner pid=2186821) 'save_path': 'outputs/profile', (TaskRunner pid=2186821) 'tool': None, (TaskRunner pid=2186821) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2186821) 'analysis': True, (TaskRunner pid=2186821) 'contents': [], (TaskRunner pid=2186821) 'discrete': False, (TaskRunner pid=2186821) 'level': 'level0'}, (TaskRunner pid=2186821) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2186821) 'discrete': False}, (TaskRunner pid=2186821) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2186821) 'step_end': None, (TaskRunner pid=2186821) 'step_start': 0}, (TaskRunner pid=2186821) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2186821) 'stack_depth': 32, (TaskRunner pid=2186821) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2186821) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2186821) 'enable': False, (TaskRunner pid=2186821) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2186821) 'port': 9090, (TaskRunner pid=2186821) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2186821) 'prompt_length': 2048, (TaskRunner pid=2186821) 'quantization': None, (TaskRunner pid=2186821) 'quantization_config_file': None, (TaskRunner pid=2186821) 'response_length': 8192, (TaskRunner pid=2186821) 'scheduling_policy': 'fcfs', (TaskRunner pid=2186821) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2186821) 'skip_rollout': False, (TaskRunner pid=2186821) 'skip_tokenizer_init': True, (TaskRunner pid=2186821) 'temperature': 1.0, (TaskRunner pid=2186821) 'tensor_model_parallel_size': 2, (TaskRunner pid=2186821) 'top_k': -1, (TaskRunner pid=2186821) 'top_p': 1.0, (TaskRunner pid=2186821) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2186821) 'backend': None, (TaskRunner pid=2186821) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2186821) 'token2text': False}, (TaskRunner pid=2186821) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2186821) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2186821) 'do_sample': True, (TaskRunner pid=2186821) 'n': 16, (TaskRunner pid=2186821) 'temperature': 0.6, (TaskRunner pid=2186821) 'top_k': -1, (TaskRunner pid=2186821) 'top_p': 0.95}}}, (TaskRunner pid=2186821) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2186821) 'adv_estimator': 'grpo', (TaskRunner pid=2186821) 'gamma': 1.0, (TaskRunner pid=2186821) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2186821) 'horizon': 10000, (TaskRunner pid=2186821) 'kl_coef': 0.001, (TaskRunner pid=2186821) 'target_kl': 0.1, (TaskRunner pid=2186821) 'type': 'fixed'}, (TaskRunner pid=2186821) 'kl_penalty': 'kl', (TaskRunner pid=2186821) 'lam': 1.0, (TaskRunner pid=2186821) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2186821) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2186821) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2186821) 'loss_type': 'ppo_clip', (TaskRunner pid=2186821) 'rollout_is': 'token', (TaskRunner pid=2186821) 'rollout_is_batch_normalize': False, (TaskRunner pid=2186821) 'rollout_is_threshold': 2.0, (TaskRunner pid=2186821) 'rollout_rs': None, (TaskRunner pid=2186821) 'rollout_rs_threshold': None}, (TaskRunner pid=2186821) 'use_kl_in_reward': False, (TaskRunner pid=2186821) 'use_pf_ppo': False}, (TaskRunner pid=2186821) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2186821) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2186821) 'async_save': False, (TaskRunner pid=2186821) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2186821) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2186821) 'cliprange_value': 0.5, (TaskRunner pid=2186821) 'data_loader_seed': 42, (TaskRunner pid=2186821) 'enable': None, (TaskRunner pid=2186821) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2186821) 'forward_micro_batch_size': None, (TaskRunner pid=2186821) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2186821) 'grad_clip': 1.0, (TaskRunner pid=2186821) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2186821) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2186821) 'enable_activation_offload': False, (TaskRunner pid=2186821) 'enable_gradient_checkpointing': True, (TaskRunner pid=2186821) 'external_lib': None, (TaskRunner pid=2186821) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2186821) 'dtype': 'bfloat16', (TaskRunner pid=2186821) 'entropy_checkpointing': False, (TaskRunner pid=2186821) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2186821) 'forward_only': False, (TaskRunner pid=2186821) 'forward_prefetch': False, (TaskRunner pid=2186821) 'fsdp_size': -1, (TaskRunner pid=2186821) 'full_determinism': False, (TaskRunner pid=2186821) 'model_dtype': 'fp32', (TaskRunner pid=2186821) 'offload_policy': False, (TaskRunner pid=2186821) 'optimizer_offload': False, (TaskRunner pid=2186821) 'param_offload': False, (TaskRunner pid=2186821) 'reshard_after_forward': True, (TaskRunner pid=2186821) 'seed': 42, (TaskRunner pid=2186821) 'strategy': 'fsdp', (TaskRunner pid=2186821) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2186821) 'use_orig_params': False, (TaskRunner pid=2186821) 'use_torch_compile': True, (TaskRunner pid=2186821) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2186821) 'lora_alpha': 16, (TaskRunner pid=2186821) 'lora_rank': 0, (TaskRunner pid=2186821) 'override_config': {}, (TaskRunner pid=2186821) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2186821) 'target_modules': 'all-linear', (TaskRunner pid=2186821) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2186821) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2186821) 'trust_remote_code': True, (TaskRunner pid=2186821) 'use_remove_padding': False, (TaskRunner pid=2186821) 'use_shm': False}, (TaskRunner pid=2186821) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2186821) 'betas': [0.9, 0.999], (TaskRunner pid=2186821) 'clip_grad': 1.0, (TaskRunner pid=2186821) 'lr': 1e-05, (TaskRunner pid=2186821) 'lr_scheduler_type': 'constant', (TaskRunner pid=2186821) 'lr_warmup_steps': -1, (TaskRunner pid=2186821) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2186821) 'min_lr_ratio': 0.0, (TaskRunner pid=2186821) 'num_cycles': 0.5, (TaskRunner pid=2186821) 'optimizer': 'AdamW', (TaskRunner pid=2186821) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2186821) 'override_optimizer_config': None, (TaskRunner pid=2186821) 'total_training_steps': -1, (TaskRunner pid=2186821) 'warmup_style': None, (TaskRunner pid=2186821) 'weight_decay': 0.01}, (TaskRunner pid=2186821) 'ppo_epochs': 1, (TaskRunner pid=2186821) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2186821) 'ppo_micro_batch_size': None, (TaskRunner pid=2186821) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2186821) 'ppo_mini_batch_size': 8, (TaskRunner pid=2186821) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2186821) 'all_ranks': False, (TaskRunner pid=2186821) 'enable': False, (TaskRunner pid=2186821) 'ranks': [], (TaskRunner pid=2186821) 'save_path': 'outputs/profile', (TaskRunner pid=2186821) 'tool': None, (TaskRunner pid=2186821) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2186821) 'analysis': True, (TaskRunner pid=2186821) 'contents': [], (TaskRunner pid=2186821) 'discrete': False, (TaskRunner pid=2186821) 'level': 'level0'}, (TaskRunner pid=2186821) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2186821) 'discrete': False}, (TaskRunner pid=2186821) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2186821) 'step_end': None, (TaskRunner pid=2186821) 'step_start': 0}, (TaskRunner pid=2186821) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2186821) 'stack_depth': 32, (TaskRunner pid=2186821) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2186821) 'rollout_n': 8, (TaskRunner pid=2186821) 'shuffle': False, (TaskRunner pid=2186821) 'strategy': 'fsdp', (TaskRunner pid=2186821) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2186821) 'use_dynamic_bsz': False}, (TaskRunner pid=2186821) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2186821) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2186821) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2186821) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2186821) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2186821) 'dataloader_num_workers': 8, (TaskRunner pid=2186821) 'filter_overlong_prompts': True, (TaskRunner pid=2186821) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2186821) 'image_key': 'images', (TaskRunner pid=2186821) 'image_patch_size': 14, (TaskRunner pid=2186821) 'max_prompt_length': 2048, (TaskRunner pid=2186821) 'max_response_length': 8192, (TaskRunner pid=2186821) 'prompt_key': 'prompt', (TaskRunner pid=2186821) 'return_full_prompt': False, (TaskRunner pid=2186821) 'return_multi_modal_inputs': True, (TaskRunner pid=2186821) 'return_raw_chat': True, (TaskRunner pid=2186821) 'return_raw_input_ids': False, (TaskRunner pid=2186821) 'reward_fn_key': 'data_source', (TaskRunner pid=2186821) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2186821) 'seed': None, (TaskRunner pid=2186821) 'shuffle': True, (TaskRunner pid=2186821) 'tokenizer': None, (TaskRunner pid=2186821) 'tool_config_path': None, (TaskRunner pid=2186821) 'train_batch_size': 32, (TaskRunner pid=2186821) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/tooluse/train.parquet'], (TaskRunner pid=2186821) 'train_max_samples': 3200, (TaskRunner pid=2186821) 'truncation': 'error', (TaskRunner pid=2186821) 'trust_remote_code': True, (TaskRunner pid=2186821) 'use_shm': False, (TaskRunner pid=2186821) 'val_batch_size': None, (TaskRunner pid=2186821) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/tooluse/test.parquet'], (TaskRunner pid=2186821) 'val_max_samples': -1, (TaskRunner pid=2186821) 'validation_shuffle': False, (TaskRunner pid=2186821) 'video_key': 'videos'}, (TaskRunner pid=2186821) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2186821) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2186821) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2186821) 'cuda-memory-usage': 'true', (TaskRunner pid=2186821) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2186821) 'discrete': False, (TaskRunner pid=2186821) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2186821) 'capture-range-end': None, (TaskRunner pid=2186821) 'cuda-graph-trace': 'graph', (TaskRunner pid=2186821) 'cuda-memory-usage': 'true', (TaskRunner pid=2186821) 'kill': 'none', (TaskRunner pid=2186821) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2186821) 'torch_memory': {'context': 'all', (TaskRunner pid=2186821) 'kw_args': {}, (TaskRunner pid=2186821) 'stack_depth': 32, (TaskRunner pid=2186821) 'stacks': 'all', (TaskRunner pid=2186821) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2186821) 'profile_continuous_steps': False, (TaskRunner pid=2186821) 'save_path': 'outputs/profile', (TaskRunner pid=2186821) 'steps': None, (TaskRunner pid=2186821) 'tool': None}, (TaskRunner pid=2186821) 'max_model_len': 10240, (TaskRunner pid=2186821) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_tooluse_grpo_Qwen3_4B', (TaskRunner pid=2186821) 'include_dashboard': False, (TaskRunner pid=2186821) 'num_cpus': None, (TaskRunner pid=2186821) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2186821) 'TASK': 'datasets/tooluse', (TaskRunner pid=2186821) 'USER': 'root'}}}, (TaskRunner pid=2186821) 'timeline_json_file': None}, (TaskRunner pid=2186821) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2186821) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2186821) 'name': 'custom_reward_manager', (TaskRunner pid=2186821) 'path': None}, (TaskRunner pid=2186821) 'name': 'naive', (TaskRunner pid=2186821) 'source': 'register'}, (TaskRunner pid=2186821) 'reward_model': {'enable': False, (TaskRunner pid=2186821) 'enable_resource_pool': False, (TaskRunner pid=2186821) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2186821) 'launch_reward_fn_async': False, (TaskRunner pid=2186821) 'max_length': None, (TaskRunner pid=2186821) 'micro_batch_size': None, (TaskRunner pid=2186821) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2186821) 'model': {'external_lib': None, (TaskRunner pid=2186821) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2186821) 'forward_prefetch': False, (TaskRunner pid=2186821) 'fsdp_size': -1, (TaskRunner pid=2186821) 'param_offload': False, (TaskRunner pid=2186821) 'reshard_after_forward': True, (TaskRunner pid=2186821) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2186821) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2186821) 'override_config': {}, (TaskRunner pid=2186821) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2186821) 'trust_remote_code': False, (TaskRunner pid=2186821) 'use_fused_kernels': False, (TaskRunner pid=2186821) 'use_remove_padding': False, (TaskRunner pid=2186821) 'use_shm': False}, (TaskRunner pid=2186821) 'n_gpus_per_node': 8, (TaskRunner pid=2186821) 'nnodes': 0, (TaskRunner pid=2186821) 'num_workers': 1, (TaskRunner pid=2186821) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2186821) 'all_ranks': False, (TaskRunner pid=2186821) 'enable': False, (TaskRunner pid=2186821) 'ranks': [], (TaskRunner pid=2186821) 'save_path': 'outputs/profile', (TaskRunner pid=2186821) 'tool': None, (TaskRunner pid=2186821) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2186821) 'analysis': True, (TaskRunner pid=2186821) 'contents': [], (TaskRunner pid=2186821) 'discrete': False, (TaskRunner pid=2186821) 'level': 'level0'}, (TaskRunner pid=2186821) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2186821) 'discrete': False}, (TaskRunner pid=2186821) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2186821) 'step_end': None, (TaskRunner pid=2186821) 'step_start': 0}, (TaskRunner pid=2186821) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2186821) 'stack_depth': 32, (TaskRunner pid=2186821) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2186821) 'reward_loop_class_name': None, (TaskRunner pid=2186821) 'reward_loop_module_path': None, (TaskRunner pid=2186821) 'reward_loop_source': 'register', (TaskRunner pid=2186821) 'reward_manager': 'naive', (TaskRunner pid=2186821) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2186821) 'cudagraph_capture_sizes': None, (TaskRunner pid=2186821) 'data_parallel_size': 1, (TaskRunner pid=2186821) 'disable_log_stats': True, (TaskRunner pid=2186821) 'dtype': 'bfloat16', (TaskRunner pid=2186821) 'enable_chunked_prefill': True, (TaskRunner pid=2186821) 'enable_prefix_caching': True, (TaskRunner pid=2186821) 'enforce_eager': True, (TaskRunner pid=2186821) 'engine_kwargs': {}, (TaskRunner pid=2186821) 'expert_parallel_size': 1, (TaskRunner pid=2186821) 'free_cache_engine': True, (TaskRunner pid=2186821) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2186821) 'limit_images': None, (TaskRunner pid=2186821) 'load_format': 'auto', (TaskRunner pid=2186821) 'max_model_len': None, (TaskRunner pid=2186821) 'max_num_batched_tokens': 8192, (TaskRunner pid=2186821) 'max_num_seqs': 1024, (TaskRunner pid=2186821) 'name': '???', (TaskRunner pid=2186821) 'prompt_length': 2048, (TaskRunner pid=2186821) 'response_length': 2048, (TaskRunner pid=2186821) 'skip_tokenizer_init': False, (TaskRunner pid=2186821) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2186821) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2186821) 'memory_limit_mb': 1024, (TaskRunner pid=2186821) 'url': None}, (TaskRunner pid=2186821) 'strategy': 'fsdp', (TaskRunner pid=2186821) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2186821) 'use_dynamic_bsz': False, (TaskRunner pid=2186821) 'use_reward_loop': False}, (TaskRunner pid=2186821) 'trainer': {'balance_batch': True, (TaskRunner pid=2186821) 'critic_warmup': 0, (TaskRunner pid=2186821) 'default_hdfs_dir': None, (TaskRunner pid=2186821) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2186821) 'del_local_ckpt_after_load': False, (TaskRunner pid=2186821) 'device': 'cuda', (TaskRunner pid=2186821) 'esi_redundant_time': 0, (TaskRunner pid=2186821) 'experiment_name': 'qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2186821) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2186821) 'log_val_generations': 0, (TaskRunner pid=2186821) 'logger': ['console', 'wandb'], (TaskRunner pid=2186821) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2186821) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2186821) 'n_gpus_per_node': 8, (TaskRunner pid=2186821) 'nnodes': 1, (TaskRunner pid=2186821) 'project_name': 'SDPO-root', (TaskRunner pid=2186821) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2186821) 'resume_from_path': None, (TaskRunner pid=2186821) 'resume_mode': 'auto', (TaskRunner pid=2186821) 'rollout_data_dir': None, (TaskRunner pid=2186821) 'save_freq': 10, (TaskRunner pid=2186821) 'test_freq': 10, (TaskRunner pid=2186821) 'total_epochs': 30, (TaskRunner pid=2186821) 'total_training_steps': 100, (TaskRunner pid=2186821) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2186821) 'val_before_train': False, (TaskRunner pid=2186821) 'val_only': False, (TaskRunner pid=2186821) 'validation_data_dir': None}, (TaskRunner pid=2186821) 'transfer_queue': {'enable': False}, (TaskRunner pid=2186821) 'vars': (TaskRunner pid=2186821) {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/tooluse', (TaskRunner pid=2186821) 'dir': '/users/root/SDPO', (TaskRunner pid=2186821) 'log_dir': '/users/root/output', (TaskRunner pid=2186821) 'task': 'datasets/tooluse'}} (TaskRunner pid=2186821) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2186821) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2186821) use_critic=need_critic(config), (TaskRunner pid=2186821) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2186821) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2186821) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2186821) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2186821) Using dataset class: RLHFDataset (TaskRunner pid=2186821) dataset len: 4046 (TaskRunner pid=2186821) selected 3200 random samples out of 4046 (TaskRunner pid=2186821) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2186821) WARNING:2026-07-02 14:53:28,280:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2186821) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/3200 [00:00 (TaskRunner pid=2186821) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2186821) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2186821) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2186821) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2186821) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2186821) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 68/68 [00:00<00:00, 101.96 examples/s] (TaskRunner pid=2186821) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2186821) self.use_critic = need_critic(self.config) (WorkerDict pid=2187231) [W702 14:53:40.472294306 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:40825 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2187230) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : (WorkerDict pid=2187230) 7 (WorkerDict pid=2187232) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2187232) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2187235) [W702 14:53:40.916361904 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:40825 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2187230) Model config after override: Qwen3Config { (WorkerDict pid=2187230) "architectures": [ (WorkerDict pid=2187230) "Qwen3ForCausalLM" (WorkerDict pid=2187230) ], (WorkerDict pid=2187230) "attention_bias": false, (WorkerDict pid=2187230) "attention_dropout": 0.0, (WorkerDict pid=2187230) "dtype": "bfloat16", (WorkerDict pid=2187230) "eos_token_id": 151645, (WorkerDict pid=2187230) "head_dim": 128, (WorkerDict pid=2187230) "hidden_act": "silu", (WorkerDict pid=2187230) "hidden_size": 2560, (WorkerDict pid=2187230) "initializer_range": 0.02, (WorkerDict pid=2187230) "intermediate_size": 9728, (WorkerDict pid=2187230) "layer_types": [ (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention", (WorkerDict pid=2187230) "full_attention" (WorkerDict pid=2187230) ], (WorkerDict pid=2187230) "max_position_embeddings": 40960, (WorkerDict pid=2187230) "max_window_layers": 36, (WorkerDict pid=2187230) "model_type": "qwen3", (WorkerDict pid=2187230) "num_attention_heads": 32, (WorkerDict pid=2187230) "num_hidden_layers": 36, (WorkerDict pid=2187230) "num_key_value_heads": 8, (WorkerDict pid=2187230) "pad_token_id": 151643, (WorkerDict pid=2187230) "rms_norm_eps": 1e-06, (WorkerDict pid=2187230) "rope_scaling": null, (WorkerDict pid=2187230) "rope_theta": 1000000, (WorkerDict pid=2187230) "sliding_window": null, (WorkerDict pid=2187230) "tie_word_embeddings": true, (WorkerDict pid=2187230) "transformers_version": "4.57.1", (WorkerDict pid=2187230) "use_cache": true, (WorkerDict pid=2187230) "use_sliding_window": false, (WorkerDict pid=2187230) "vocab_size": 151936 (WorkerDict pid=2187230) } (WorkerDict pid=2187230) (WorkerDict pid=2187232) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2187232) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2187232) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2187230) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2187230) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2187230) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2187235) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2187235) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2187235) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2187235) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2187235) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.91s/it] [repeated 13x across cluster] (WorkerDict pid=2187232) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.95s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.98s/it] (WorkerDict pid=2187235) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it] [repeated 5x across cluster] (WorkerDict pid=2187230) Actor use_remove_padding=True (WorkerDict pid=2187230) Actor use_fused_kernels=False (WorkerDict pid=2187230) Actor use_prefix_grouper=False (WorkerDict pid=2187235) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2187235) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2187230) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2187235) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2187235) warnings.warn( (WorkerDict pid=2187231) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2187231) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2187231) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2187231) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2187234) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2187234) warnings.warn( [repeated 7x across cluster] (TaskRunner pid=2186821) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2186821) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (vLLMHttpServer pid=2188089) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2188089) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2188089) ['serve', (vLLMHttpServer pid=2188089) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2188089) '--dtype', (vLLMHttpServer pid=2188089) 'bfloat16', (vLLMHttpServer pid=2188089) '--load_format', (vLLMHttpServer pid=2188089) 'dummy', (vLLMHttpServer pid=2188089) '--trust_remote_code', (vLLMHttpServer pid=2188089) '--max_model_len', (vLLMHttpServer pid=2188089) '40960', (vLLMHttpServer pid=2188089) '--max_num_seqs', (vLLMHttpServer pid=2188089) '1024', (vLLMHttpServer pid=2188089) '--enable_chunked_prefill', (vLLMHttpServer pid=2188089) '--max_num_batched_tokens', (vLLMHttpServer pid=2188089) '10240', (vLLMHttpServer pid=2188089) '--enable_prefix_caching', (vLLMHttpServer pid=2188089) '--enable_sleep_mode', (vLLMHttpServer pid=2188089) '--logprobs_mode', (vLLMHttpServer pid=2188089) 'processed_logprobs', (vLLMHttpServer pid=2188089) '--disable_custom_all_reduce', (vLLMHttpServer pid=2188089) '--gpu_memory_utilization', (vLLMHttpServer pid=2188089) '0.8', (vLLMHttpServer pid=2188089) '--disable_log_stats', (vLLMHttpServer pid=2188089) '--tensor_parallel_size', (vLLMHttpServer pid=2188089) '2', (vLLMHttpServer pid=2188089) '--seed', (vLLMHttpServer pid=2188089) '0', (vLLMHttpServer pid=2188089) '--override_generation_config', (vLLMHttpServer pid=2188089) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2188089) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2188089) '--hf_overrides', (vLLMHttpServer pid=2188089) '{}', (vLLMHttpServer pid=2188089) '--scheduling_policy', (vLLMHttpServer pid=2188089) 'fcfs'] (WorkerDict pid=2187237) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2188089) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2188089) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2188090) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2188090) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2188089) WARNING 07-02 14:54:22 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2187230) WARNING 07-02 14:54:30 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2188092) WARNING 07-02 14:54:23 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2187232) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2187231) [rank1]:W0702 14:54:40.853000 2187231 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2187231) [rank1]:W0702 14:54:40.853000 2187231 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2187231) [rank1]:W0702 14:54:40.853000 2187231 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2187231) [rank1]:W0702 14:54:40.853000 2187231 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2187231) [rank1]:W0702 14:54:40.853000 2187231 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2187231) [rank1]:W0702 14:54:40.853000 2187231 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/fxgraph/vs/fvscv5dg7y3y3dcup4ddfpgbmpglg73u5zi54vpf2q4orgekbtq7/.2187235.134989297153728.tmp' (vLLMHttpServer pid=2188092) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2188092) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (vLLMHttpServer pid=2188092) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2188092) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (WorkerDict pid=2187236) 2026-07-02 14:54:44,874 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2187236) 2026-07-02 14:54:44,891 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2187230) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: The user is asking for a random picture of an axolotl. The appropriate tool for this is getRandomAxolotlImage. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2186821) step:10 - global_seqlen/min:15336 - global_seqlen/max:41224 - global_seqlen/minmax_diff:25888 - global_seqlen/balanced_min:19566 - global_seqlen/balanced_max:27059 - global_seqlen/mean:22376.125 - actor/entropy:0.06471491605043411 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25514164566993713 - training/rollout_probs_diff_mean:0.0019480646587908268 - training/rollout_probs_diff_std:0.009871283546090126 - training/rollout_actor_probs_pearson_corr:0.9974164962768555 - rollout_corr/rollout_is_mean:0.9997751116752625 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0001353515690425411 - rollout_corr/rollout_is_max:1.7737531661987305 - rollout_corr/rollout_is_min:0.44347867369651794 - rollout_corr/rollout_is_std:0.02663147822022438 - rollout_corr/rollout_is_eff_sample_size:0.9992908503494108 - rollout_corr/rollout_is_seq_mean:0.9994572997093201 - rollout_corr/rollout_is_seq_std:0.0046390630304813385 - rollout_corr/rollout_is_seq_max:1.0134824514389038 - rollout_corr/rollout_is_seq_min:0.9830322265625 - rollout_corr/rollout_is_seq_max_deviation:0.0169677734375 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1388853043317795) - rollout_corr/training_log_ppl:np.float64(0.1254820067433684) - rollout_corr/kl:np.float64(0.0009716963424644121) - rollout_corr/k3_kl:np.float64(0.0009642050270038949) - rollout_corr/rollout_ppl:np.float64(1.1376882079057395) - rollout_corr/rollout_log_ppl:np.float64(0.12451030904594518) - rollout_corr/log_ppl_diff:np.float64(0.0009716976974232239) - rollout_corr/log_ppl_abs_diff:np.float64(0.003936542275368993) - rollout_corr/log_ppl_diff_max:np.float64(0.02153278887271881) - rollout_corr/log_ppl_diff_min:np.float64(-0.020883288234472275) - rollout_corr/ppl_ratio:np.float64(1.000986902276054) - rollout_corr/chi2_token:np.float64(0.0018790557514876127) - rollout_corr/chi2_seq:np.float64(0.01500549796037376) - actor/pg_loss:np.float64(3.802124410867691e-07) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047881080718070734) - actor/ppo_kl:np.float64(-0.0003828074489522493) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4972062036395073) - perf/mfu/actor:np.float64(0.06153828124526606) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(99.23387908935547) - actor/lr:np.float64(9e-07) - val-aux/tooluse/reward/mean@16:np.float64(0.5965073529411765) - val-aux/tooluse/reward/std@16:np.float64(0.02526972233916341) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6038382352941176) - val-aux/tooluse/reward/best@2/std:np.float64(0.010997622736856439) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5896176470588235) - val-aux/tooluse/reward/worst@2/std:np.float64(0.02883197168984434) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5966176470588236) - val-aux/tooluse/reward/maj@2/std:np.float64(0.024733745354939912) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6060441176470588) - val-aux/tooluse/reward/best@4/std:np.float64(0.006000270322630196) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5781176470588235) - val-aux/tooluse/reward/worst@4/std:np.float64(0.033952338910140115) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.6006176470588235) - val-aux/tooluse/reward/maj@4/std:np.float64(0.014513185041700504) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6086764705882353) - val-aux/tooluse/reward/best@8/std:np.float64(0.007172793175445411) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5626617647058823) - val-aux/tooluse/reward/worst@8/std:np.float64(0.03534818492066255) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.6026470588235293) - val-aux/tooluse/reward/maj@8/std:np.float64(0.0035617805240550572) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6124117647058823) - val-aux/tooluse/reward/best@16/std:np.float64(0.007041400065249293) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5455882352941176) - val-aux/tooluse/reward/worst@16/std:np.float64(0.02883751990660591) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.6029264705882353) - val-aux/tooluse/reward/maj@16/std:np.float64(0.00046480825380232674) - val-aux/tooluse/score/mean@16:np.float64(0.5965073529411765) - val-aux/tooluse/score/std@16:np.float64(0.02526972233916341) - val-aux/tooluse/score/best@2/mean:np.float64(0.6038382352941176) - val-aux/tooluse/score/best@2/std:np.float64(0.010997622736856439) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5896176470588235) - val-aux/tooluse/score/worst@2/std:np.float64(0.02883197168984434) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5966176470588236) - val-aux/tooluse/score/maj@2/std:np.float64(0.024733745354939912) - val-aux/tooluse/score/best@4/mean:np.float64(0.6060441176470588) - val-aux/tooluse/score/best@4/std:np.float64(0.006000270322630196) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5781176470588235) - val-aux/tooluse/score/worst@4/std:np.float64(0.033952338910140115) - val-aux/tooluse/score/maj@4/mean:np.float64(0.6006176470588235) - val-aux/tooluse/score/maj@4/std:np.float64(0.014513185041700504) - val-aux/tooluse/score/best@8/mean:np.float64(0.6086764705882353) - val-aux/tooluse/score/best@8/std:np.float64(0.007172793175445411) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5626617647058823) - val-aux/tooluse/score/worst@8/std:np.float64(0.03534818492066255) - val-aux/tooluse/score/maj@8/mean:np.float64(0.6026470588235293) - val-aux/tooluse/score/maj@8/std:np.float64(0.0035617805240550572) - val-aux/tooluse/score/best@16/mean:np.float64(0.6124117647058823) - val-aux/tooluse/score/best@16/std:np.float64(0.007041400065249293) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5455882352941176) - val-aux/tooluse/score/worst@16/std:np.float64(0.02883751990660591) - val-aux/tooluse/score/maj@16/mean:np.float64(0.6029264705882353) - val-aux/tooluse/score/maj@16/std:np.float64(0.00046480825380232674) - val-core/tooluse/acc/mean@16:np.float64(0.5965073529411765) - val-aux/tooluse/acc/std@16:np.float64(0.02526972233916341) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6038382352941176) - val-aux/tooluse/acc/best@2/std:np.float64(0.010997622736856439) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5896176470588235) - val-aux/tooluse/acc/worst@2/std:np.float64(0.02883197168984434) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5966176470588236) - val-aux/tooluse/acc/maj@2/std:np.float64(0.024733745354939912) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6060441176470588) - val-aux/tooluse/acc/best@4/std:np.float64(0.006000270322630196) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5781176470588235) - val-aux/tooluse/acc/worst@4/std:np.float64(0.033952338910140115) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.6006176470588235) - val-aux/tooluse/acc/maj@4/std:np.float64(0.014513185041700504) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6086764705882353) - val-aux/tooluse/acc/best@8/std:np.float64(0.007172793175445411) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5626617647058823) - val-aux/tooluse/acc/worst@8/std:np.float64(0.03534818492066255) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.6026470588235293) - val-aux/tooluse/acc/maj@8/std:np.float64(0.0035617805240550572) - val-core/tooluse/acc/best@16/mean:np.float64(0.6124117647058823) - val-core/tooluse/acc/best@16/std:np.float64(0.007041400065249293) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5455882352941176) - val-aux/tooluse/acc/worst@16/std:np.float64(0.02883751990660591) - val-core/tooluse/acc/maj@16/mean:np.float64(0.6029264705882353) - val-core/tooluse/acc/maj@16/std:np.float64(0.00046480825380232674) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027436893433332443 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0027436893433332443 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:173.16015625 - response_length/max:8192.0 - response_length/min:12.0 - response_length/clip_ratio:0.01171875 - response_length_non_aborted/mean:173.16015625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:12.0 - response_length_non_aborted/clip_ratio:0.01171875 - response/aborted_ratio:0.0 - prompt_length/mean:526.09375 - prompt_length/max:786.0 - prompt_length/min:344.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010194070637226105 - timing_s/agent_loop/generate_sequences/min:np.float64(0.28289108723402023) - timing_s/agent_loop/generate_sequences/max:np.float64(48.11588947288692) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.4584810184824164) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(48.11588947288692) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:621 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:53.95149340853095 - timing_s/reward:0.06835589557886124 - timing_s/old_log_prob:2.591398885473609 - timing_s/adv:0.13262889347970486 - timing_s/update_actor:10.779934303835034 - timing_s/step:67.6131257750094 - timing_s/testing:21.740137917920947 - timing_s/save_checkpoint:6.4859949592500925 - timing_s/stop_profile:0.00010768882930278778 - timing_per_token_ms/update_actor:0.06022006884477894 - timing_per_token_ms/adv:0.0007409062867213651 - timing_per_token_ms/gen:1.217069940863339 - perf/total_num_tokens:179009 - perf/time_per_step:67.6131257750094 - perf/throughput:330.94350754407037 (TaskRunner pid=2186821) Training Progress: 10%|█ | 10/100 [05:17<1:07:27, 44.97s/it] (TaskRunner pid=2186821) step:11 - global_seqlen/min:17578 - global_seqlen/max:27283 - global_seqlen/minmax_diff:9705 - global_seqlen/balanced_min:21652 - global_seqlen/balanced_max:21657 - global_seqlen/mean:21654.25 - actor/entropy:0.11065898835659027 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29282984137535095 - training/rollout_probs_diff_mean:0.004017804283648729 - training/rollout_probs_diff_std:0.015260000713169575 - training/rollout_actor_probs_pearson_corr:0.9959209561347961 - rollout_corr/rollout_is_mean:1.0003602504730225 - rollout_corr/rollout_is_ratio_fraction_high:0.00016127298295032233 - rollout_corr/rollout_is_ratio_fraction_low:0.00037630362203344703 - rollout_corr/rollout_is_max:3.364696979522705 - rollout_corr/rollout_is_min:0.3406287729740143 - rollout_corr/rollout_is_std:0.04031352326273918 - rollout_corr/rollout_is_eff_sample_size:0.9983781696179648 - rollout_corr/rollout_is_seq_mean:1.000364065170288 - rollout_corr/rollout_is_seq_std:0.007739383261650801 - rollout_corr/rollout_is_seq_max:1.0629969835281372 - rollout_corr/rollout_is_seq_min:0.9764702320098877 - rollout_corr/rollout_is_seq_max_deviation:0.06299698352813721 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1182947410270572) - rollout_corr/training_log_ppl:np.float64(0.10826437650393927) - rollout_corr/kl:np.float64(0.001223238331206744) - rollout_corr/k3_kl:np.float64(0.0013988505138087248) - rollout_corr/rollout_ppl:np.float64(1.1168462545610964) - rollout_corr/rollout_log_ppl:np.float64(0.10704113724750641) - rollout_corr/log_ppl_diff:np.float64(0.001223239256432862) - rollout_corr/log_ppl_abs_diff:np.float64(0.005348508309907629) - rollout_corr/log_ppl_diff_max:np.float64(0.044343382120132446) - rollout_corr/log_ppl_diff_min:np.float64(-0.03604096174240112) - rollout_corr/ppl_ratio:np.float64(1.0012552302796394) - rollout_corr/chi2_token:np.float64(0.0033159349113702774) - rollout_corr/chi2_seq:np.float64(0.32321867998689413) - actor/pg_loss:np.float64(0.00012246717233210802) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003686612944875378) - actor/ppo_kl:np.float64(0.0004963153411168264) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5073169767856598) - perf/mfu/actor:np.float64(0.06195358941008023) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(108.78277206420898) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.30078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004891946911811829 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004891946911811829 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:72.6640625 - response_length/max:256.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:72.6640625 - response_length_non_aborted/max:256.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:604.03125 - prompt_length/max:1164.0 - prompt_length/min:268.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001065470278263092 - timing_s/agent_loop/generate_sequences/min:np.float64(0.35498063638806343) - timing_s/agent_loop/generate_sequences/max:np.float64(1.9746794011443853) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8692239028896438) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.9746794011443853) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:657 - timing_s/agent_loop/slowest/response_length:256 - timing_s/gen:7.388710806146264 - timing_s/reward:0.06730573810636997 - timing_s/old_log_prob:2.6162441559135914 - timing_s/adv:0.1398067157715559 - timing_s/update_actor:10.246204193681479 - timing_s/step:20.5187059789896 - timing_s/stop_profile:0.00011201947927474976 - timing_per_token_ms/update_actor:0.05914661206045856 - timing_per_token_ms/adv:0.0008070397022037007 - timing_per_token_ms/gen:0.3971998068028311 - perf/total_num_tokens:173234 - perf/time_per_step:20.5187059789896 - perf/throughput:1055.3418925234932 (TaskRunner pid=2186821) Training Progress: 11%|█ | 11/100 [05:38<55:37, 37.50s/it] (TaskRunner pid=2186821) step:12 - global_seqlen/min:17954 - global_seqlen/max:28837 - global_seqlen/minmax_diff:10883 - global_seqlen/balanced_min:21192 - global_seqlen/balanced_max:21407 - global_seqlen/mean:21220.5 - actor/entropy:0.1336514800786972 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917335033416748 - training/rollout_probs_diff_mean:0.004169702995568514 - training/rollout_probs_diff_std:0.013394581153988838 - training/rollout_actor_probs_pearson_corr:0.9973517060279846 - rollout_corr/rollout_is_mean:0.9994610548019409 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00014772503345739096 - rollout_corr/rollout_is_max:1.5675175189971924 - rollout_corr/rollout_is_min:0.3256097137928009 - rollout_corr/rollout_is_std:0.037291232496500015 - rollout_corr/rollout_is_eff_sample_size:0.9986100469936493 - rollout_corr/rollout_is_seq_mean:0.9994513392448425 - rollout_corr/rollout_is_seq_std:0.0043182666413486 - rollout_corr/rollout_is_seq_max:1.011215090751648 - rollout_corr/rollout_is_seq_min:0.9867015480995178 - rollout_corr/rollout_is_seq_max_deviation:0.013298451900482178 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1365594435483217) - rollout_corr/training_log_ppl:np.float64(0.12453540127899032) - rollout_corr/kl:np.float64(0.0008226861730284973) - rollout_corr/k3_kl:np.float64(0.0010328982278515753) - rollout_corr/rollout_ppl:np.float64(1.1354309413582087) - rollout_corr/rollout_log_ppl:np.float64(0.12371271370284376) - rollout_corr/log_ppl_diff:np.float64(0.0008226875834225211) - rollout_corr/log_ppl_abs_diff:np.float64(0.00435226490299101) - rollout_corr/log_ppl_diff_max:np.float64(0.023256003856658936) - rollout_corr/log_ppl_diff_min:np.float64(-0.03139470890164375) - rollout_corr/ppl_ratio:np.float64(1.0008434124756604) - rollout_corr/chi2_token:np.float64(0.002417704090476036) - rollout_corr/chi2_seq:np.float64(0.08645861991681159) - actor/pg_loss:np.float64(0.0001993263140320778) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001963553368113935) - actor/ppo_kl:np.float64(-0.00044044979841828535) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.32797153294086456) - perf/mfu/actor:np.float64(0.060148255230898375) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.13369750976562) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00795253086835146 - critic/advantages/max:0.75 - critic/advantages/min:-0.25 - critic/returns/mean:-0.00795253086835146 - critic/returns/max:0.75 - critic/returns/min:-0.25 - response_length/mean:79.328125 - response_length/max:606.0 - response_length/min:10.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:79.328125 - response_length_non_aborted/max:606.0 - response_length_non_aborted/min:10.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:583.8125 - prompt_length/max:960.0 - prompt_length/min:294.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015121325850486755 - timing_s/agent_loop/generate_sequences/min:np.float64(0.21685582771897316) - timing_s/agent_loop/generate_sequences/max:np.float64(3.5554216038435698) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8958015917305602) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.5554216038435698) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:960 - timing_s/agent_loop/slowest/response_length:606 - timing_s/gen:9.70180044695735 - timing_s/reward:0.07100607641041279 - timing_s/old_log_prob:2.5051915924996138 - timing_s/adv:0.15295771323144436 - timing_s/update_actor:10.269677044823766 - timing_s/step:22.790751703083515 - timing_s/stop_profile:0.00011807866394519806 - timing_per_token_ms/update_actor:0.06049384465978515 - timing_per_token_ms/adv:0.0009010020571584338 - timing_per_token_ms/gen:0.4777329351466097 - perf/total_num_tokens:169764 - perf/time_per_step:22.790751703083515 - perf/throughput:931.1013641173114 (TaskRunner pid=2186821) Training Progress: 12%|█▏ | 12/100 [06:01<48:28, 33.06s/it] (TaskRunner pid=2186821) step:13 - global_seqlen/min:14620 - global_seqlen/max:28237 - global_seqlen/minmax_diff:13617 - global_seqlen/balanced_min:21004 - global_seqlen/balanced_max:21704 - global_seqlen/mean:21096.875 - actor/entropy:0.10547120124101639 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19509756565093994 - training/rollout_probs_diff_mean:0.003380682785063982 - training/rollout_probs_diff_std:0.012886268086731434 - training/rollout_actor_probs_pearson_corr:0.9970110654830933 - rollout_corr/rollout_is_mean:1.000056266784668 - rollout_corr/rollout_is_ratio_fraction_high:4.570592864183709e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.570592864183709e-05 - rollout_corr/rollout_is_max:2.1585445404052734 - rollout_corr/rollout_is_min:0.26028817892074585 - rollout_corr/rollout_is_std:0.036277685314416885 - rollout_corr/rollout_is_eff_sample_size:0.9986857781047227 - rollout_corr/rollout_is_seq_mean:1.00021493434906 - rollout_corr/rollout_is_seq_std:0.0042898631654679775 - rollout_corr/rollout_is_seq_max:1.0150363445281982 - rollout_corr/rollout_is_seq_min:0.9877874851226807 - rollout_corr/rollout_is_seq_max_deviation:0.015036344528198242 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1031650374643505) - rollout_corr/training_log_ppl:np.float64(0.09494470232675667) - rollout_corr/kl:np.float64(0.000851536146512899) - rollout_corr/k3_kl:np.float64(0.0009472659370093339) - rollout_corr/rollout_ppl:np.float64(1.1022485885769129) - rollout_corr/rollout_log_ppl:np.float64(0.09409316546589253) - rollout_corr/log_ppl_diff:np.float64(0.0008515368608641438) - rollout_corr/log_ppl_abs_diff:np.float64(0.004001086879725335) - rollout_corr/log_ppl_diff_max:np.float64(0.019642561674118042) - rollout_corr/log_ppl_diff_min:np.float64(-0.05255556106567383) - rollout_corr/ppl_ratio:np.float64(1.0008714410942048) - rollout_corr/chi2_token:np.float64(0.002315883059054613) - rollout_corr/chi2_seq:np.float64(0.29878060379996896) - actor/pg_loss:np.float64(0.00033441511914134026) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019135708134854212) - actor/ppo_kl:np.float64(0.00036150807645185523) - actor/pg_clipfrac_lower:np.float64(5.351027357392013e-05) - actor/grad_norm:np.float64(1.1670051291584969) - perf/mfu/actor:np.float64(0.060736521616678724) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.21525192260742) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.26953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.26953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010683761211112142 - critic/advantages/max:0.75 - critic/advantages/min:-0.375 - critic/returns/mean:0.0010683761211112142 - critic/returns/max:0.75 - critic/returns/min:-0.375 - response_length/mean:85.46484375 - response_length/max:1474.0 - response_length/min:12.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:85.46484375 - response_length_non_aborted/max:1474.0 - response_length_non_aborted/min:12.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:573.8125 - prompt_length/max:953.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010475516319274902 - timing_s/agent_loop/generate_sequences/min:np.float64(0.24746857769787312) - timing_s/agent_loop/generate_sequences/max:np.float64(7.537464823573828) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9030697855414473) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.537464823573828) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:646 - timing_s/agent_loop/slowest/response_length:1474 - timing_s/gen:13.254051620140672 - timing_s/reward:0.06365508399903774 - timing_s/old_log_prob:2.580045059323311 - timing_s/adv:0.17224122397601604 - timing_s/update_actor:9.916575159877539 - timing_s/step:26.088937563821673 - timing_s/stop_profile:0.00011115334928035736 - timing_per_token_ms/update_actor:0.058756185216279296 - timing_per_token_ms/adv:0.0010205375439254394 - timing_per_token_ms/gen:0.6057887298386888 - perf/total_num_tokens:168775 - perf/time_per_step:26.088937563821673 - perf/throughput:808.6521326669769 (TaskRunner pid=2186821) Training Progress: 13%|█▎ | 13/100 [06:27<44:53, 30.96s/it] (TaskRunner pid=2186821) step:14 - global_seqlen/min:18585 - global_seqlen/max:27856 - global_seqlen/minmax_diff:9271 - global_seqlen/balanced_min:22463 - global_seqlen/balanced_max:22479 - global_seqlen/mean:22476.5 - actor/entropy:0.13945402204990387 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37773674726486206 - training/rollout_probs_diff_mean:0.003940461669117212 - training/rollout_probs_diff_std:0.014028717763721943 - training/rollout_actor_probs_pearson_corr:0.9971532225608826 - rollout_corr/rollout_is_mean:1.0002578496932983 - rollout_corr/rollout_is_ratio_fraction_high:9.551098628435284e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00019102197256870568 - rollout_corr/rollout_is_max:2.615034341812134 - rollout_corr/rollout_is_min:0.13723966479301453 - rollout_corr/rollout_is_std:0.04035342484712601 - rollout_corr/rollout_is_eff_sample_size:0.9983750802313806 - rollout_corr/rollout_is_seq_mean:0.9999485015869141 - rollout_corr/rollout_is_seq_std:0.004765972029417753 - rollout_corr/rollout_is_seq_max:1.0165014266967773 - rollout_corr/rollout_is_seq_min:0.9816931486129761 - rollout_corr/rollout_is_seq_max_deviation:0.018306851387023926 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1481469324789941) - rollout_corr/training_log_ppl:np.float64(0.12590649889898486) - rollout_corr/kl:np.float64(0.0013059416769980459) - rollout_corr/k3_kl:np.float64(0.0009203811810643003) - rollout_corr/rollout_ppl:np.float64(1.1466691764071584) - rollout_corr/rollout_log_ppl:np.float64(0.12460055729025044) - rollout_corr/log_ppl_diff:np.float64(0.0013059416087344289) - rollout_corr/log_ppl_abs_diff:np.float64(0.0041504556211293675) - rollout_corr/log_ppl_diff_max:np.float64(0.022361963987350464) - rollout_corr/log_ppl_diff_min:np.float64(-0.014641091227531433) - rollout_corr/ppl_ratio:np.float64(1.0013217092491686) - rollout_corr/chi2_token:np.float64(0.0010603347327560186) - rollout_corr/chi2_seq:np.float64(0.11991321481764317) - actor/pg_loss:np.float64(0.000248922617174685) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029277181238285266) - actor/ppo_kl:np.float64(0.00040014971494528595) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.44558000192046165) - perf/mfu/actor:np.float64(0.06543933858980965) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.22722053527832) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.25390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005832139402627945 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.005832139402627945 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:81.796875 - response_length/max:730.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:81.796875 - response_length_non_aborted/max:730.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:620.59375 - prompt_length/max:1271.0 - prompt_length/min:222.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011085905134677887 - timing_s/agent_loop/generate_sequences/min:np.float64(0.2806439455598593) - timing_s/agent_loop/generate_sequences/max:np.float64(3.924503242596984) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7626911898623803) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.924503242596984) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:563 - timing_s/agent_loop/slowest/response_length:730 - timing_s/gen:10.226195132359862 - timing_s/reward:0.07936667464673519 - timing_s/old_log_prob:2.4005846194922924 - timing_s/adv:0.1209832914173603 - timing_s/update_actor:9.730293542146683 - timing_s/step:22.64402087777853 - timing_s/stop_profile:0.00011856108903884888 - timing_per_token_ms/update_actor:0.054113705103923444 - timing_per_token_ms/adv:0.0006728321325459942 - timing_per_token_ms/gen:0.48835697862272504 - perf/total_num_tokens:179812 - perf/time_per_step:22.64402087777853 - perf/throughput:992.6019818351729 (TaskRunner pid=2186821) Training Progress: 14%|█▍ | 14/100 [06:49<40:47, 28.46s/it] (TaskRunner pid=2186821) step:15 - global_seqlen/min:18183 - global_seqlen/max:26649 - global_seqlen/minmax_diff:8466 - global_seqlen/balanced_min:22076 - global_seqlen/balanced_max:22991 - global_seqlen/mean:22234.625 - actor/entropy:0.10255803912878036 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21512344479560852 - training/rollout_probs_diff_mean:0.0031315009109675884 - training/rollout_probs_diff_std:0.01174172293394804 - training/rollout_actor_probs_pearson_corr:0.9973945617675781 - rollout_corr/rollout_is_mean:1.0000900030136108 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.487323167268187e-05 - rollout_corr/rollout_is_max:1.9659780263900757 - rollout_corr/rollout_is_min:0.38385188579559326 - rollout_corr/rollout_is_std:0.03351758420467377 - rollout_corr/rollout_is_eff_sample_size:0.9988780702140859 - rollout_corr/rollout_is_seq_mean:0.9997937679290771 - rollout_corr/rollout_is_seq_std:0.004385844804346561 - rollout_corr/rollout_is_seq_max:1.015427589416504 - rollout_corr/rollout_is_seq_min:0.984419584274292 - rollout_corr/rollout_is_seq_max_deviation:0.015580415725708008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1088435342535377) - rollout_corr/training_log_ppl:np.float64(0.09964647670267368) - rollout_corr/kl:np.float64(0.0006665136890338097) - rollout_corr/k3_kl:np.float64(0.0006070779035987073) - rollout_corr/rollout_ppl:np.float64(1.1081091212108731) - rollout_corr/rollout_log_ppl:np.float64(0.09897996178278845) - rollout_corr/log_ppl_diff:np.float64(0.0006665149198852305) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031428449069608178) - rollout_corr/log_ppl_diff_max:np.float64(0.016401618719100952) - rollout_corr/log_ppl_diff_min:np.float64(-0.014488592743873596) - rollout_corr/ppl_ratio:np.float64(1.0006763231940567) - rollout_corr/chi2_token:np.float64(0.0011608535423874855) - rollout_corr/chi2_seq:np.float64(0.20763885555788875) - actor/pg_loss:np.float64(0.00013550324365496635) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011524505316629075) - actor/ppo_kl:np.float64(-0.00011097606275711769) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.32620856538414955) - perf/mfu/actor:np.float64(0.06362896524765974) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.21925354003906) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.021651335060596466 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.021651335060596466 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:87.05078125 - response_length/max:1798.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:87.05078125 - response_length_non_aborted/max:1798.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:607.78125 - prompt_length/max:1112.0 - prompt_length/min:389.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011326000094413757 - timing_s/agent_loop/generate_sequences/min:np.float64(0.33877107687294483) - timing_s/agent_loop/generate_sequences/max:np.float64(9.166378064081073) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9166920651914552) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.166378064081073) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:669 - timing_s/agent_loop/slowest/response_length:1798 - timing_s/gen:14.91093702800572 - timing_s/reward:0.06791229732334614 - timing_s/old_log_prob:2.3873610757291317 - timing_s/adv:0.14204834587872028 - timing_s/update_actor:9.984724432229996 - timing_s/step:27.58326202072203 - timing_s/stop_profile:0.00011246278882026672 - timing_per_token_ms/update_actor:0.05613274584252037 - timing_per_token_ms/adv:0.0007985762402037379 - timing_per_token_ms/gen:0.6691019532423478 - perf/total_num_tokens:177877 - perf/time_per_step:27.58326202072203 - perf/throughput:806.0912078961564 (TaskRunner pid=2186821) Training Progress: 15%|█▌ | 15/100 [07:17<39:57, 28.21s/it] (TaskRunner pid=2186821) step:16 - global_seqlen/min:16842 - global_seqlen/max:27176 - global_seqlen/minmax_diff:10334 - global_seqlen/balanced_min:21430 - global_seqlen/balanced_max:21434 - global_seqlen/mean:21432.0 - actor/entropy:0.11721711605787277 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7327550649642944 - training/rollout_probs_diff_mean:0.0037980861961841583 - training/rollout_probs_diff_std:0.01433052122592926 - training/rollout_actor_probs_pearson_corr:0.9966628551483154 - rollout_corr/rollout_is_mean:1.0002752542495728 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.925383735913783e-05 - rollout_corr/rollout_is_max:1.883165955543518 - rollout_corr/rollout_is_min:0.16806770861148834 - rollout_corr/rollout_is_std:0.038128167390823364 - rollout_corr/rollout_is_eff_sample_size:0.9985493039646843 - rollout_corr/rollout_is_seq_mean:1.0004366636276245 - rollout_corr/rollout_is_seq_std:0.004696501884609461 - rollout_corr/rollout_is_seq_max:1.0264089107513428 - rollout_corr/rollout_is_seq_min:0.9884065389633179 - rollout_corr/rollout_is_seq_max_deviation:0.026408910751342773 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1303188595920801) - rollout_corr/training_log_ppl:np.float64(0.11847262882110954) - rollout_corr/kl:np.float64(0.0014165267428762895) - rollout_corr/k3_kl:np.float64(0.001299961922031967) - rollout_corr/rollout_ppl:np.float64(1.1286529414355755) - rollout_corr/rollout_log_ppl:np.float64(0.11705610053650162) - rollout_corr/log_ppl_diff:np.float64(0.0014165282846079208) - rollout_corr/log_ppl_abs_diff:np.float64(0.0044320387751213275) - rollout_corr/log_ppl_diff_max:np.float64(0.03061334416270256) - rollout_corr/log_ppl_diff_min:np.float64(-0.018878445029258728) - rollout_corr/ppl_ratio:np.float64(1.0014400919899344) - rollout_corr/chi2_token:np.float64(0.002078073099255562) - rollout_corr/chi2_seq:np.float64(0.14976750616915524) - actor/pg_loss:np.float64(-0.00031990697607398033) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005518490725080483) - actor/ppo_kl:np.float64(0.0010948172209346296) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2516094371676445) - perf/mfu/actor:np.float64(0.06188243601656669) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.22307968139648) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00024544805637560785 - critic/advantages/max:0.75 - critic/advantages/min:-0.75 - critic/returns/mean:0.00024544805637560785 - critic/returns/max:0.75 - critic/returns/min:-0.75 - response_length/mean:87.53125 - response_length/max:306.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:87.53125 - response_length_non_aborted/max:306.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:582.21875 - prompt_length/max:1227.0 - prompt_length/min:268.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011365115642547607 - timing_s/agent_loop/generate_sequences/min:np.float64(0.34439461678266525) - timing_s/agent_loop/generate_sequences/max:np.float64(2.325568325817585) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9509003902130644) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.325568325817585) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:821 - timing_s/agent_loop/slowest/response_length:306 - timing_s/gen:8.146629331633449 - timing_s/reward:0.08607851527631283 - timing_s/old_log_prob:2.420163882896304 - timing_s/adv:0.12450077012181282 - timing_s/update_actor:9.782093726098537 - timing_s/step:20.646614979952574 - timing_s/stop_profile:0.00011237338185310364 - timing_per_token_ms/update_actor:0.057053084908656085 - timing_per_token_ms/adv:0.0007261383102476018 - timing_per_token_ms/gen:0.3635589669597219 - perf/total_num_tokens:171456 - perf/time_per_step:20.646614979952574 - perf/throughput:1038.0394084362022 (TaskRunner pid=2186821) Training Progress: 16%|█▌ | 16/100 [07:38<36:19, 25.95s/it] (TaskRunner pid=2186821) step:17 - global_seqlen/min:18088 - global_seqlen/max:28258 - global_seqlen/minmax_diff:10170 - global_seqlen/balanced_min:21855 - global_seqlen/balanced_max:21863 - global_seqlen/mean:21858.625 - actor/entropy:0.11119091510772705 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1932182013988495 - training/rollout_probs_diff_mean:0.0033948849886655807 - training/rollout_probs_diff_std:0.012288741767406464 - training/rollout_actor_probs_pearson_corr:0.9974102973937988 - rollout_corr/rollout_is_mean:1.0001575946807861 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9823064804077148 - rollout_corr/rollout_is_min:0.5485212206840515 - rollout_corr/rollout_is_std:0.033592186868190765 - rollout_corr/rollout_is_eff_sample_size:0.9988733125587188 - rollout_corr/rollout_is_seq_mean:1.0001633167266846 - rollout_corr/rollout_is_seq_std:0.0041444324888288975 - rollout_corr/rollout_is_seq_max:1.0214910507202148 - rollout_corr/rollout_is_seq_min:0.9865897297859192 - rollout_corr/rollout_is_seq_max_deviation:0.021491050720214844 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1170064881443977) - rollout_corr/training_log_ppl:np.float64(0.10516982415720122) - rollout_corr/kl:np.float64(0.0009147260237867272) - rollout_corr/k3_kl:np.float64(0.0009090476105386713) - rollout_corr/rollout_ppl:np.float64(1.1159183429554105) - rollout_corr/rollout_log_ppl:np.float64(0.1042550980928354) - rollout_corr/log_ppl_diff:np.float64(0.0009147260643658228) - rollout_corr/log_ppl_abs_diff:np.float64(0.003960158232075628) - rollout_corr/log_ppl_diff_max:np.float64(0.04896654933691025) - rollout_corr/log_ppl_diff_min:np.float64(-0.015669062733650208) - rollout_corr/ppl_ratio:np.float64(1.0009330231696367) - rollout_corr/chi2_token:np.float64(0.0018352679908275604) - rollout_corr/chi2_seq:np.float64(0.1100027107167989) - actor/pg_loss:np.float64(7.333746179938316e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005187013630347792) - actor/ppo_kl:np.float64(0.0005165046874537893) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4459506645798683) - perf/mfu/actor:np.float64(0.0629563766170818) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.24784088134766) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016162274405360222 - critic/advantages/max:0.625 - critic/advantages/min:-0.5 - critic/returns/mean:-0.016162274405360222 - critic/returns/max:0.625 - critic/returns/min:-0.5 - response_length/mean:74.86328125 - response_length/max:213.0 - response_length/min:31.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:74.86328125 - response_length_non_aborted/max:213.0 - response_length_non_aborted/min:31.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:608.21875 - prompt_length/max:1504.0 - prompt_length/min:344.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010832399129867554 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5169259607791901) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7946552261710167) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9706476531428052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7946552261710167) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:864 - timing_s/agent_loop/slowest/response_length:213 - timing_s/gen:7.6037781443446875 - timing_s/reward:0.07036185823380947 - timing_s/old_log_prob:2.42200630530715 - timing_s/adv:0.1444731205701828 - timing_s/update_actor:10.026633586734533 - timing_s/step:20.353611171245575 - timing_s/stop_profile:0.00012239627540111542 - timing_per_token_ms/update_actor:0.05733797063364309 - timing_per_token_ms/adv:0.0008261791430738599 - timing_per_token_ms/gen:0.39675335999711386 - perf/total_num_tokens:174869 - perf/time_per_step:20.353611171245575 - perf/throughput:1073.9433320255534 (TaskRunner pid=2186821) Training Progress: 17%|█▋ | 17/100 [07:58<33:35, 24.28s/it] (TaskRunner pid=2186821) step:18 - global_seqlen/min:18813 - global_seqlen/max:23698 - global_seqlen/minmax_diff:4885 - global_seqlen/balanced_min:20818 - global_seqlen/balanced_max:20820 - global_seqlen/mean:20819.0 - actor/entropy:0.12094065546989441 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6551973819732666 - training/rollout_probs_diff_mean:0.003967585042119026 - training/rollout_probs_diff_std:0.01763567328453064 - training/rollout_actor_probs_pearson_corr:0.9953041672706604 - rollout_corr/rollout_is_mean:0.999499499797821 - rollout_corr/rollout_is_ratio_fraction_high:4.929022179567255e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00034503155620768666 - rollout_corr/rollout_is_max:2.096895694732666 - rollout_corr/rollout_is_min:0.2231307476758957 - rollout_corr/rollout_is_std:0.03835495188832283 - rollout_corr/rollout_is_eff_sample_size:0.9985299295627951 - rollout_corr/rollout_is_seq_mean:0.9994632005691528 - rollout_corr/rollout_is_seq_std:0.00480045797303319 - rollout_corr/rollout_is_seq_max:1.0149915218353271 - rollout_corr/rollout_is_seq_min:0.9820056557655334 - rollout_corr/rollout_is_seq_max_deviation:0.017994344234466553 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1333318478427827) - rollout_corr/training_log_ppl:np.float64(0.12167448409672943) - rollout_corr/kl:np.float64(0.0021152384450431327) - rollout_corr/k3_kl:np.float64(0.0012530820485103789) - rollout_corr/rollout_ppl:np.float64(1.1310462038964033) - rollout_corr/rollout_log_ppl:np.float64(0.11955924579160637) - rollout_corr/log_ppl_diff:np.float64(0.002115238305123057) - rollout_corr/log_ppl_abs_diff:np.float64(0.004981779769877903) - rollout_corr/log_ppl_diff_max:np.float64(0.0206269770860672) - rollout_corr/log_ppl_diff_min:np.float64(-0.022997260093688965) - rollout_corr/ppl_ratio:np.float64(1.0021386400330812) - rollout_corr/chi2_token:np.float64(0.0006961924955248833) - rollout_corr/chi2_seq:np.float64(0.14171262993477285) - actor/pg_loss:np.float64(-1.521850936114788e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005228894005995244) - actor/ppo_kl:np.float64(0.0006848103632677294) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3859841376543045) - perf/mfu/actor:np.float64(0.06075702681002515) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.21238327026367) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.3046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010412558913230896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0010412558913230896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:79.25 - response_length/max:350.0 - response_length/min:33.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:79.25 - response_length_non_aborted/max:350.0 - response_length_non_aborted/min:33.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:571.34375 - prompt_length/max:1118.0 - prompt_length/min:313.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.652040898799896e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.37218301370739937) - timing_s/agent_loop/generate_sequences/max:np.float64(2.4049112163484097) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8362914688404999) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.4049112163484097) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:657 - timing_s/agent_loop/slowest/response_length:350 - timing_s/gen:8.21405184455216 - timing_s/reward:0.06988294422626495 - timing_s/old_log_prob:2.433989616110921 - timing_s/adv:0.12871501594781876 - timing_s/update_actor:9.956711055710912 - timing_s/step:20.888712368905544 - timing_s/stop_profile:0.00011636689305305481 - timing_per_token_ms/update_actor:0.059781395934668524 - timing_per_token_ms/adv:0.0007728217970833059 - timing_per_token_ms/gen:0.40487242924645894 - perf/total_num_tokens:166552 - perf/time_per_step:20.888712368905544 - perf/throughput:996.6626775420913 (TaskRunner pid=2186821) Training Progress: 18%|█▊ | 18/100 [08:19<31:48, 23.28s/it] (TaskRunner pid=2186821) step:19 - global_seqlen/min:15760 - global_seqlen/max:31275 - global_seqlen/minmax_diff:15515 - global_seqlen/balanced_min:22648 - global_seqlen/balanced_max:22654 - global_seqlen/mean:22651.125 - actor/entropy:0.1481165736913681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31749820709228516 - training/rollout_probs_diff_mean:0.0037795137614011765 - training/rollout_probs_diff_std:0.012887347489595413 - training/rollout_actor_probs_pearson_corr:0.9977067112922668 - rollout_corr/rollout_is_mean:0.999906063079834 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.808632264845073e-05 - rollout_corr/rollout_is_max:1.5790883302688599 - rollout_corr/rollout_is_min:0.3648768961429596 - rollout_corr/rollout_is_std:0.03610475733876228 - rollout_corr/rollout_is_eff_sample_size:0.9986977867613321 - rollout_corr/rollout_is_seq_mean:0.9999219179153442 - rollout_corr/rollout_is_seq_std:0.004382098559290171 - rollout_corr/rollout_is_seq_max:1.0151665210723877 - rollout_corr/rollout_is_seq_min:0.9830559492111206 - rollout_corr/rollout_is_seq_max_deviation:0.016944050788879395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1473363060504198) - rollout_corr/training_log_ppl:np.float64(0.13077730423901812) - rollout_corr/kl:np.float64(0.000855688248144304) - rollout_corr/k3_kl:np.float64(0.0008919248291938686) - rollout_corr/rollout_ppl:np.float64(1.1463262890465558) - rollout_corr/rollout_log_ppl:np.float64(0.12992161509464495) - rollout_corr/log_ppl_diff:np.float64(0.0008556891443731729) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037165909125178587) - rollout_corr/log_ppl_diff_max:np.float64(0.025697849690914154) - rollout_corr/log_ppl_diff_min:np.float64(-0.02685999870300293) - rollout_corr/ppl_ratio:np.float64(1.000870683696121) - rollout_corr/chi2_token:np.float64(0.001970159588381648) - rollout_corr/chi2_seq:np.float64(0.1874959417618811) - actor/pg_loss:np.float64(0.00031265802681446075) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010279605339746922) - actor/ppo_kl:np.float64(8.951358755382444e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3371078446507454) - perf/mfu/actor:np.float64(0.065075271064578) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.37019348144531) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.23828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.23828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002059017773717642 - critic/advantages/max:0.375 - critic/advantages/min:-0.625 - critic/returns/mean:-0.002059017773717642 - critic/returns/max:0.375 - critic/returns/min:-0.625 - response_length/mean:88.69140625 - response_length/max:346.0 - response_length/min:34.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:88.69140625 - response_length_non_aborted/max:346.0 - response_length_non_aborted/min:34.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:619.15625 - prompt_length/max:1276.0 - prompt_length/min:361.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001458730548620224 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5067798048257828) - timing_s/agent_loop/generate_sequences/max:np.float64(2.389638025313616) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9768104215036146) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.389638025313616) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:551 - timing_s/agent_loop/slowest/response_length:346 - timing_s/gen:8.230718372389674 - timing_s/reward:0.07090778462588787 - timing_s/old_log_prob:2.5025619976222515 - timing_s/adv:0.14755584672093391 - timing_s/update_actor:10.074906524270773 - timing_s/step:21.11203101463616 - timing_s/stop_profile:4.0903687477111816e-05 - timing_per_token_ms/update_actor:0.055598267880021264 - timing_per_token_ms/adv:0.0008142854202657369 - timing_per_token_ms/gen:0.3625068651129564 - perf/total_num_tokens:181209 - perf/time_per_step:21.11203101463616 - perf/throughput:1072.9012753105965 (TaskRunner pid=2186821) Training Progress: 19%|█▉ | 19/100 [08:40<30:33, 22.63s/it] (TaskRunner pid=2186821) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2186821) validation generation end (TaskRunner pid=2186821) [prompt] user (TaskRunner pid=2186821) Your task is to answer the user's question using available tools. (TaskRunner pid=2186821) You have access to the following tools: (TaskRunner pid=2186821) Name: Axolotl (TaskRunner pid=2186821) Description: Collection of axolotl pictures and facts (TaskRunner pid=2186821) Documentation: (TaskRunner pid=2186821) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2186821) Parameters: {} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{url, source, description} (TaskRunner pid=2186821) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2186821) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2186821) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2186821) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Array[Object{fact, source}] (TaskRunner pid=2186821) (TaskRunner pid=2186821) Use the following format: (TaskRunner pid=2186821) Thought: you should always think about what to do (TaskRunner pid=2186821) Action: the action to take, should be one of the tool names. (TaskRunner pid=2186821) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2186821) (TaskRunner pid=2186821) Begin! (TaskRunner pid=2186821) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2186821) assistant (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: The user is asking for a random picture of an axolotl. The appropriate tool for this is getRandomAxolotlImage. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2187230) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2186821) step:20 - global_seqlen/min:16614 - global_seqlen/max:23306 - global_seqlen/minmax_diff:6692 - global_seqlen/balanced_min:19492 - global_seqlen/balanced_max:19494 - global_seqlen/mean:19493.5 - actor/entropy:0.15356434881687164 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25122255086898804 - training/rollout_probs_diff_mean:0.004298811312764883 - training/rollout_probs_diff_std:0.013858449645340443 - training/rollout_actor_probs_pearson_corr:0.9974499344825745 - rollout_corr/rollout_is_mean:0.9998043179512024 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.60412481718231e-05 - rollout_corr/rollout_is_max:1.7535219192504883 - rollout_corr/rollout_is_min:0.37117302417755127 - rollout_corr/rollout_is_std:0.039754461497068405 - rollout_corr/rollout_is_eff_sample_size:0.9984215418710358 - rollout_corr/rollout_is_seq_mean:0.9996814727783203 - rollout_corr/rollout_is_seq_std:0.005543193779885769 - rollout_corr/rollout_is_seq_max:1.0196911096572876 - rollout_corr/rollout_is_seq_min:0.9830915927886963 - rollout_corr/rollout_is_seq_max_deviation:0.019691109657287598 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.146904826629907) - rollout_corr/training_log_ppl:np.float64(0.13216020141771878) - rollout_corr/kl:np.float64(0.0012992605487696096) - rollout_corr/k3_kl:np.float64(0.0020795683532885967) - rollout_corr/rollout_ppl:np.float64(1.145381584763527) - rollout_corr/rollout_log_ppl:np.float64(0.13086094010941451) - rollout_corr/log_ppl_diff:np.float64(0.001299261308304267) - rollout_corr/log_ppl_abs_diff:np.float64(0.006794426411943277) - rollout_corr/log_ppl_diff_max:np.float64(0.02778749167919159) - rollout_corr/log_ppl_diff_min:np.float64(-0.08173477649688721) - rollout_corr/ppl_ratio:np.float64(1.0013526247348636) - rollout_corr/chi2_token:np.float64(0.006720415782183409) - rollout_corr/chi2_seq:np.float64(0.5300568870734423) - actor/pg_loss:np.float64(-5.703954957425594e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041540651000104845) - actor/ppo_kl:np.float64(0.0001717223559012382) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3030691556632519) - perf/mfu/actor:np.float64(0.056968639490365125) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.21083450317383) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5790441176470589) - val-aux/tooluse/reward/std@16:np.float64(0.032695917530049745) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5935735294117647) - val-aux/tooluse/reward/best@2/std:np.float64(0.0218901824701889) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5641176470588235) - val-aux/tooluse/reward/worst@2/std:np.float64(0.03385631277959569) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5788529411764705) - val-aux/tooluse/reward/maj@2/std:np.float64(0.03250575651427305) - val-aux/tooluse/reward/best@4/mean:np.float64(0.601264705882353) - val-aux/tooluse/reward/best@4/std:np.float64(0.00825903027552835) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5483970588235294) - val-aux/tooluse/reward/worst@4/std:np.float64(0.028060359020437742) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5848088235294117) - val-aux/tooluse/reward/maj@4/std:np.float64(0.02818717417262091) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6028676470588236) - val-aux/tooluse/reward/best@8/std:np.float64(0.0014612742862188033) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5370147058823529) - val-aux/tooluse/reward/worst@8/std:np.float64(0.017292925507350433) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5900882352941177) - val-aux/tooluse/reward/maj@8/std:np.float64(0.02319868605977267) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6029411764705882) - val-aux/tooluse/reward/best@16/std:np.float64(0.0) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.53125) - val-aux/tooluse/reward/worst@16/std:np.float64(0.007052989032520017) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.59425) - val-aux/tooluse/reward/maj@16/std:np.float64(0.017269632917924157) - val-aux/tooluse/score/mean@16:np.float64(0.5790441176470589) - val-aux/tooluse/score/std@16:np.float64(0.032695917530049745) - val-aux/tooluse/score/best@2/mean:np.float64(0.5935735294117647) - val-aux/tooluse/score/best@2/std:np.float64(0.0218901824701889) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5641176470588235) - val-aux/tooluse/score/worst@2/std:np.float64(0.03385631277959569) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5788529411764705) - val-aux/tooluse/score/maj@2/std:np.float64(0.03250575651427305) - val-aux/tooluse/score/best@4/mean:np.float64(0.601264705882353) - val-aux/tooluse/score/best@4/std:np.float64(0.00825903027552835) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5483970588235294) - val-aux/tooluse/score/worst@4/std:np.float64(0.028060359020437742) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5848088235294117) - val-aux/tooluse/score/maj@4/std:np.float64(0.02818717417262091) - val-aux/tooluse/score/best@8/mean:np.float64(0.6028676470588236) - val-aux/tooluse/score/best@8/std:np.float64(0.0014612742862188033) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5370147058823529) - val-aux/tooluse/score/worst@8/std:np.float64(0.017292925507350433) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5900882352941177) - val-aux/tooluse/score/maj@8/std:np.float64(0.02319868605977267) - val-aux/tooluse/score/best@16/mean:np.float64(0.6029411764705882) - val-aux/tooluse/score/best@16/std:np.float64(0.0) - val-aux/tooluse/score/worst@16/mean:np.float64(0.53125) - val-aux/tooluse/score/worst@16/std:np.float64(0.007052989032520017) - val-aux/tooluse/score/maj@16/mean:np.float64(0.59425) - val-aux/tooluse/score/maj@16/std:np.float64(0.017269632917924157) - val-core/tooluse/acc/mean@16:np.float64(0.5790441176470589) - val-aux/tooluse/acc/std@16:np.float64(0.032695917530049745) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5935735294117647) - val-aux/tooluse/acc/best@2/std:np.float64(0.0218901824701889) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5641176470588235) - val-aux/tooluse/acc/worst@2/std:np.float64(0.03385631277959569) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5788529411764705) - val-aux/tooluse/acc/maj@2/std:np.float64(0.03250575651427305) - val-aux/tooluse/acc/best@4/mean:np.float64(0.601264705882353) - val-aux/tooluse/acc/best@4/std:np.float64(0.00825903027552835) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5483970588235294) - val-aux/tooluse/acc/worst@4/std:np.float64(0.028060359020437742) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5848088235294117) - val-aux/tooluse/acc/maj@4/std:np.float64(0.02818717417262091) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6028676470588236) - val-aux/tooluse/acc/best@8/std:np.float64(0.0014612742862188033) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5370147058823529) - val-aux/tooluse/acc/worst@8/std:np.float64(0.017292925507350433) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5900882352941177) - val-aux/tooluse/acc/maj@8/std:np.float64(0.02319868605977267) - val-core/tooluse/acc/best@16/mean:np.float64(0.6029411764705882) - val-core/tooluse/acc/best@16/std:np.float64(0.0) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.53125) - val-aux/tooluse/acc/worst@16/std:np.float64(0.007052989032520017) - val-core/tooluse/acc/maj@16/mean:np.float64(0.59425) - val-core/tooluse/acc/maj@16/std:np.float64(0.017269632917924157) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.29296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.29296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0035095829516649246 - critic/advantages/max:0.875 - critic/advantages/min:-0.25 - critic/returns/mean:0.0035095829516649246 - critic/returns/max:0.875 - critic/returns/min:-0.25 - response_length/mean:69.703125 - response_length/max:276.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:69.703125 - response_length_non_aborted/max:276.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:539.46875 - prompt_length/max:857.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.998516619205475e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3804078008979559) - timing_s/agent_loop/generate_sequences/max:np.float64(1.9228412508964539) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8290234782471089) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.9228412508964539) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:664 - timing_s/agent_loop/slowest/response_length:270 - timing_s/gen:7.944174448028207 - timing_s/reward:0.06810303404927254 - timing_s/old_log_prob:2.4054074492305517 - timing_s/adv:0.15821969136595726 - timing_s/update_actor:9.64206663519144 - timing_s/step:20.303631827235222 - timing_s/testing:20.219341134652495 - timing_s/save_checkpoint:6.629967804998159 - timing_s/stop_profile:0.0001206323504447937 - timing_per_token_ms/update_actor:0.06182872903269962 - timing_per_token_ms/adv:0.0010145669798006852 - timing_per_token_ms/gen:0.4452014373474673 - perf/total_num_tokens:155948 - perf/time_per_step:20.303631827235222 - perf/throughput:960.0991667831312 (TaskRunner pid=2186821) Training Progress: 20%|██ | 20/100 [09:27<40:01, 30.02s/it] (TaskRunner pid=2186821) step:21 - global_seqlen/min:16441 - global_seqlen/max:27633 - global_seqlen/minmax_diff:11192 - global_seqlen/balanced_min:22054 - global_seqlen/balanced_max:22743 - global_seqlen/mean:22143.0 - actor/entropy:0.11762800067663193 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19509410858154297 - training/rollout_probs_diff_mean:0.0033869799226522446 - training/rollout_probs_diff_std:0.011857884004712105 - training/rollout_actor_probs_pearson_corr:0.9976400136947632 - rollout_corr/rollout_is_mean:1.0001535415649414 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.599507212638855 - rollout_corr/rollout_is_min:0.6298704147338867 - rollout_corr/rollout_is_std:0.03298158198595047 - rollout_corr/rollout_is_eff_sample_size:0.9989137540740176 - rollout_corr/rollout_is_seq_mean:1.000128149986267 - rollout_corr/rollout_is_seq_std:0.004816056229174137 - rollout_corr/rollout_is_seq_max:1.0216249227523804 - rollout_corr/rollout_is_seq_min:0.9850412011146545 - rollout_corr/rollout_is_seq_max_deviation:0.02162492275238037 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1228348519653082) - rollout_corr/training_log_ppl:np.float64(0.113011989480583) - rollout_corr/kl:np.float64(0.0010437153751752248) - rollout_corr/k3_kl:np.float64(0.001223540148629354) - rollout_corr/rollout_ppl:np.float64(1.121664835140109) - rollout_corr/rollout_log_ppl:np.float64(0.11196827405365184) - rollout_corr/log_ppl_diff:np.float64(0.0010437154269311577) - rollout_corr/log_ppl_abs_diff:np.float64(0.0046924148191465065) - rollout_corr/log_ppl_diff_max:np.float64(0.03626396507024765) - rollout_corr/log_ppl_diff_min:np.float64(-0.02010389044880867) - rollout_corr/ppl_ratio:np.float64(1.001067015575245) - rollout_corr/chi2_token:np.float64(0.002913662465289235) - rollout_corr/chi2_seq:np.float64(0.1944984500296414) - actor/pg_loss:np.float64(0.00016850605607032776) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(4.200268813292496e-05) - actor/ppo_kl:np.float64(0.0005368560964313929) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2613734155893326) - perf/mfu/actor:np.float64(0.06385031523538999) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(108.18473434448242) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.20703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.20703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009825623128563166 - critic/advantages/max:0.375 - critic/advantages/min:-0.625 - critic/returns/mean:0.0009825623128563166 - critic/returns/max:0.375 - critic/returns/min:-0.625 - response_length/mean:86.46875 - response_length/max:1508.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.46875 - response_length_non_aborted/max:1508.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:605.5 - prompt_length/max:1013.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.872415542602539e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.43551396764814854) - timing_s/agent_loop/generate_sequences/max:np.float64(7.799870667979121) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9425139114391641) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.799870667979121) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:717 - timing_s/agent_loop/slowest/response_length:1508 - timing_s/gen:13.690325535833836 - timing_s/reward:0.0666985884308815 - timing_s/old_log_prob:2.547838367521763 - timing_s/adv:0.14900783263146877 - timing_s/update_actor:10.043792566284537 - timing_s/step:26.608636500313878 - timing_s/stop_profile:0.00011618062853813171 - timing_per_token_ms/update_actor:0.05669846320668234 - timing_per_token_ms/adv:0.0008411678218368603 - timing_per_token_ms/gen:0.618464290559895 - perf/total_num_tokens:177144 - perf/time_per_step:26.608636500313878 - perf/throughput:832.1734185717784 (TaskRunner pid=2186821) Training Progress: 21%|██ | 21/100 [09:54<38:12, 29.01s/it] (TaskRunner pid=2186821) step:22 - global_seqlen/min:19020 - global_seqlen/max:31102 - global_seqlen/minmax_diff:12082 - global_seqlen/balanced_min:23093 - global_seqlen/balanced_max:23096 - global_seqlen/mean:23094.125 - actor/entropy:0.08905012160539627 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25533169507980347 - training/rollout_probs_diff_mean:0.0030403200071305037 - training/rollout_probs_diff_std:0.011853907257318497 - training/rollout_actor_probs_pearson_corr:0.9970695376396179 - rollout_corr/rollout_is_mean:1.0003975629806519 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.637653112411499 - rollout_corr/rollout_is_min:0.6212183833122253 - rollout_corr/rollout_is_std:0.03293094038963318 - rollout_corr/rollout_is_eff_sample_size:0.998917798405658 - rollout_corr/rollout_is_seq_mean:1.0005921125411987 - rollout_corr/rollout_is_seq_std:0.004469444043934345 - rollout_corr/rollout_is_seq_max:1.0216679573059082 - rollout_corr/rollout_is_seq_min:0.9860487580299377 - rollout_corr/rollout_is_seq_max_deviation:0.021667957305908203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0908911083824933) - rollout_corr/training_log_ppl:np.float64(0.08513677297378308) - rollout_corr/kl:np.float64(0.00010070971794018391) - rollout_corr/k3_kl:np.float64(0.0006697209643808932) - rollout_corr/rollout_ppl:np.float64(1.0907264747656882) - rollout_corr/rollout_log_ppl:np.float64(0.08503606256999774) - rollout_corr/log_ppl_diff:np.float64(0.00010071040378534235) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035854729321727064) - rollout_corr/log_ppl_diff_max:np.float64(0.020956501364707947) - rollout_corr/log_ppl_diff_min:np.float64(-0.01799200475215912) - rollout_corr/ppl_ratio:np.float64(1.0001139934174716) - rollout_corr/chi2_token:np.float64(0.0024869635235518217) - rollout_corr/chi2_seq:np.float64(0.1470104360487312) - actor/pg_loss:np.float64(0.0003595448797568679) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038905805922695436) - actor/ppo_kl:np.float64(0.0001602610907127655) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.760275274515152) - perf/mfu/actor:np.float64(0.06626374661068346) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(107.8788948059082) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002694834256544709 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002694834256544709 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:65.41015625 - response_length/max:272.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:65.41015625 - response_length_non_aborted/max:272.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:656.28125 - prompt_length/max:1224.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000116715207695961 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3845140617340803) - timing_s/agent_loop/generate_sequences/max:np.float64(2.0091099869459867) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7573715589533094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.0091099869459867) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:816 - timing_s/agent_loop/slowest/response_length:271 - timing_s/gen:7.685897460207343 - timing_s/reward:0.07205311581492424 - timing_s/old_log_prob:2.3913693632930517 - timing_s/adv:0.1531701236963272 - timing_s/update_actor:9.966865433380008 - timing_s/step:20.356785288080573 - timing_s/stop_profile:0.00012865103781223297 - timing_per_token_ms/update_actor:0.053946974790016984 - timing_per_token_ms/adv:0.0008290535130489205 - timing_per_token_ms/gen:0.45899656376275566 - perf/total_num_tokens:184753 - perf/time_per_step:20.356785288080573 - perf/throughput:1134.468172316098 (TaskRunner pid=2186821) Training Progress: 22%|██▏ | 22/100 [10:15<34:21, 26.43s/it] (TaskRunner pid=2186821) step:23 - global_seqlen/min:17417 - global_seqlen/max:24698 - global_seqlen/minmax_diff:7281 - global_seqlen/balanced_min:21093 - global_seqlen/balanced_max:21095 - global_seqlen/mean:21093.375 - actor/entropy:0.13183005154132843 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21512293815612793 - training/rollout_probs_diff_mean:0.003858409821987152 - training/rollout_probs_diff_std:0.013058317825198174 - training/rollout_actor_probs_pearson_corr:0.9974924325942993 - rollout_corr/rollout_is_mean:1.0002200603485107 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.972753882408142 - rollout_corr/rollout_is_min:0.5040687322616577 - rollout_corr/rollout_is_std:0.03722163662314415 - rollout_corr/rollout_is_eff_sample_size:0.99861717973235 - rollout_corr/rollout_is_seq_mean:1.0001062154769897 - rollout_corr/rollout_is_seq_std:0.004868939518928528 - rollout_corr/rollout_is_seq_max:1.0317291021347046 - rollout_corr/rollout_is_seq_min:0.9864935278892517 - rollout_corr/rollout_is_seq_max_deviation:0.03172910213470459 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1299134357832372) - rollout_corr/training_log_ppl:np.float64(0.11896105579216965) - rollout_corr/kl:np.float64(0.0009152648806676211) - rollout_corr/k3_kl:np.float64(0.0011235473831839454) - rollout_corr/rollout_ppl:np.float64(1.128787834662944) - rollout_corr/rollout_log_ppl:np.float64(0.1180457895679865) - rollout_corr/log_ppl_diff:np.float64(0.0009152662241831422) - rollout_corr/log_ppl_abs_diff:np.float64(0.0043638092174660414) - rollout_corr/log_ppl_diff_max:np.float64(0.026485398411750793) - rollout_corr/log_ppl_diff_min:np.float64(-0.024338722229003906) - rollout_corr/ppl_ratio:np.float64(1.0009346010629088) - rollout_corr/chi2_token:np.float64(0.002651476999744773) - rollout_corr/chi2_seq:np.float64(0.18567267898470163) - actor/pg_loss:np.float64(-0.00023290805984288454) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002592730015749112) - actor/ppo_kl:np.float64(0.00037620513790592724) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.36563779413700104) - perf/mfu/actor:np.float64(0.060515049754530925) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(107.87517166137695) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.20703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.20703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020310801919549704 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0020310801919549704 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:82.69921875 - response_length/max:325.0 - response_length/min:28.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:82.69921875 - response_length_non_aborted/max:325.0 - response_length_non_aborted/min:28.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:576.46875 - prompt_length/max:869.0 - prompt_length/min:249.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013703852891921997 - timing_s/agent_loop/generate_sequences/min:np.float64(0.39109655283391476) - timing_s/agent_loop/generate_sequences/max:np.float64(2.276912236586213) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8998566372392816) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.276912236586213) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:563 - timing_s/agent_loop/slowest/response_length:310 - timing_s/gen:8.317079961299896 - timing_s/reward:0.06873997114598751 - timing_s/old_log_prob:2.5027565844357014 - timing_s/adv:0.1404077261686325 - timing_s/update_actor:10.141143508255482 - timing_s/step:21.2581734880805 - timing_s/stop_profile:0.00011378340423107147 - timing_per_token_ms/update_actor:0.06009673362048203 - timing_per_token_ms/adv:0.000832060576891041 - timing_per_token_ms/gen:0.39285248506447007 - perf/total_num_tokens:168747 - perf/time_per_step:21.2581734880805 - perf/throughput:992.2477588127266 (TaskRunner pid=2186821) Training Progress: 23%|██▎ | 23/100 [10:36<31:56, 24.89s/it] (TaskRunner pid=2186821) step:24 - global_seqlen/min:13636 - global_seqlen/max:26715 - global_seqlen/minmax_diff:13079 - global_seqlen/balanced_min:21238 - global_seqlen/balanced_max:21243 - global_seqlen/mean:21240.875 - actor/entropy:0.11103028059005737 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22837159037590027 - training/rollout_probs_diff_mean:0.0033503517042845488 - training/rollout_probs_diff_std:0.012416457757353783 - training/rollout_actor_probs_pearson_corr:0.997409462928772 - rollout_corr/rollout_is_mean:0.9996703863143921 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00011018676741514355 - rollout_corr/rollout_is_max:1.648439884185791 - rollout_corr/rollout_is_min:0.39285746216773987 - rollout_corr/rollout_is_std:0.035718031227588654 - rollout_corr/rollout_is_eff_sample_size:0.9987248964737375 - rollout_corr/rollout_is_seq_mean:0.999692440032959 - rollout_corr/rollout_is_seq_std:0.005039681680500507 - rollout_corr/rollout_is_seq_max:1.017751932144165 - rollout_corr/rollout_is_seq_min:0.9750996828079224 - rollout_corr/rollout_is_seq_max_deviation:0.024900317192077637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1200664010830224) - rollout_corr/training_log_ppl:np.float64(0.10942026597331278) - rollout_corr/kl:np.float64(0.0012794950706833674) - rollout_corr/k3_kl:np.float64(0.001360495389545191) - rollout_corr/rollout_ppl:np.float64(1.11852665245533) - rollout_corr/rollout_log_ppl:np.float64(0.10814077021768753) - rollout_corr/log_ppl_diff:np.float64(0.0012794957556252484) - rollout_corr/log_ppl_abs_diff:np.float64(0.004733572176519374) - rollout_corr/log_ppl_diff_max:np.float64(0.04210108518600464) - rollout_corr/log_ppl_diff_min:np.float64(-0.019741535186767578) - rollout_corr/ppl_ratio:np.float64(1.0013062241487205) - rollout_corr/chi2_token:np.float64(0.0027310161385685205) - rollout_corr/chi2_seq:np.float64(0.12737260502763093) - actor/pg_loss:np.float64(0.0005497323581948876) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005835022438986925) - actor/ppo_kl:np.float64(0.0003079909663483704) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.519828625023365) - perf/mfu/actor:np.float64(0.061719308922082806) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(107.9288592338562) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.2890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013139772228896618 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013139772228896618 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:70.90234375 - response_length/max:253.0 - response_length/min:30.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:70.90234375 - response_length_non_aborted/max:253.0 - response_length_non_aborted/min:30.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:592.875 - prompt_length/max:1108.0 - prompt_length/min:229.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010242685675621033 - timing_s/agent_loop/generate_sequences/min:np.float64(0.43444598093628883) - timing_s/agent_loop/generate_sequences/max:np.float64(1.9199607912451029) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8339922519953689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.9199607912451029) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:585 - timing_s/agent_loop/slowest/response_length:253 - timing_s/gen:7.791998490691185 - timing_s/reward:0.0679673720151186 - timing_s/old_log_prob:2.4947714898735285 - timing_s/adv:0.14812434650957584 - timing_s/update_actor:9.833304403349757 - timing_s/step:20.42645957879722 - timing_s/stop_profile:0.00011198967695236206 - timing_per_token_ms/update_actor:0.05786781619960193 - timing_per_token_ms/adv:0.0008716940010097033 - timing_per_token_ms/gen:0.4292875594012002 - perf/total_num_tokens:169927 - perf/time_per_step:20.42645957879722 - perf/throughput:1039.870610864359 (TaskRunner pid=2186821) Training Progress: 24%|██▍ | 24/100 [10:56<29:51, 23.57s/it] (TaskRunner pid=2186821) step:25 - global_seqlen/min:18821 - global_seqlen/max:27946 - global_seqlen/minmax_diff:9125 - global_seqlen/balanced_min:21456 - global_seqlen/balanced_max:25618 - global_seqlen/mean:22195.25 - actor/entropy:0.08765517175197601 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6064720749855042 - training/rollout_probs_diff_mean:0.0026657134294509888 - training/rollout_probs_diff_std:0.012547273188829422 - training/rollout_actor_probs_pearson_corr:0.9967637062072754 - rollout_corr/rollout_is_mean:0.9996585845947266 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013911108544562012 - rollout_corr/rollout_is_max:1.9980249404907227 - rollout_corr/rollout_is_min:0.0571809858083725 - rollout_corr/rollout_is_std:0.030806252732872963 - rollout_corr/rollout_is_eff_sample_size:0.9990511607411355 - rollout_corr/rollout_is_seq_mean:0.9994648694992065 - rollout_corr/rollout_is_seq_std:0.004381819628179073 - rollout_corr/rollout_is_seq_max:1.016879916191101 - rollout_corr/rollout_is_seq_min:0.9840549230575562 - rollout_corr/rollout_is_seq_max_deviation:0.016879916191101074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1075696521438658) - rollout_corr/training_log_ppl:np.float64(0.09969949652077048) - rollout_corr/kl:np.float64(0.001138967890781828) - rollout_corr/k3_kl:np.float64(0.0011121713446122783) - rollout_corr/rollout_ppl:np.float64(1.1061658468097448) - rollout_corr/rollout_log_ppl:np.float64(0.0985605280729942) - rollout_corr/log_ppl_diff:np.float64(0.0011389684477762785) - rollout_corr/log_ppl_abs_diff:np.float64(0.0045304325576580595) - rollout_corr/log_ppl_diff_max:np.float64(0.025265678763389587) - rollout_corr/log_ppl_diff_min:np.float64(-0.018175512552261353) - rollout_corr/ppl_ratio:np.float64(1.001158632338047) - rollout_corr/chi2_token:np.float64(0.0019411584362387657) - rollout_corr/chi2_seq:np.float64(0.12271243007853627) - actor/pg_loss:np.float64(-0.0003345268778502941) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004053400334669277) - actor/ppo_kl:np.float64(-2.8612570993402642e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3388681709766388) - perf/mfu/actor:np.float64(0.06226702638192828) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(107.82539987564087) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.2109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00012606941163539886 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.00012606941163539886 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:112.3203125 - response_length/max:5331.0 - response_length/min:32.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:112.3203125 - response_length_non_aborted/max:5331.0 - response_length_non_aborted/min:32.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:581.28125 - prompt_length/max:976.0 - prompt_length/min:312.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010652653872966766 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4800858963280916) - timing_s/agent_loop/generate_sequences/max:np.float64(26.05691055394709) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.047626505183871) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.05691055394709) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:538 - timing_s/agent_loop/slowest/response_length:5331 - timing_s/gen:32.44191267900169 - timing_s/reward:0.07280621491372585 - timing_s/old_log_prob:2.595277577638626 - timing_s/adv:0.12046796642243862 - timing_s/update_actor:10.293081505224109 - timing_s/step:45.611834364011884 - timing_s/stop_profile:0.0001846402883529663 - timing_per_token_ms/update_actor:0.05796894327178174 - timing_per_token_ms/adv:0.0006784557868374912 - timing_per_token_ms/gen:1.12825737911253 - perf/total_num_tokens:177562 - perf/time_per_step:45.611834364011884 - perf/throughput:486.61165045166956 (TaskRunner pid=2186821) Training Progress: 25%|██▌ | 25/100 [11:42<37:44, 30.19s/it] (TaskRunner pid=2186821) step:26 - global_seqlen/min:14046 - global_seqlen/max:23609 - global_seqlen/minmax_diff:9563 - global_seqlen/balanced_min:20171 - global_seqlen/balanced_max:20173 - global_seqlen/mean:20172.0 - actor/entropy:0.0700807198882103 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26595649123191833 - training/rollout_probs_diff_mean:0.0029006022959947586 - training/rollout_probs_diff_std:0.01220697071403265 - training/rollout_actor_probs_pearson_corr:0.9960450530052185 - rollout_corr/rollout_is_mean:0.9999938011169434 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00017257248691748828 - rollout_corr/rollout_is_max:1.7586426734924316 - rollout_corr/rollout_is_min:0.4281517565250397 - rollout_corr/rollout_is_std:0.032941799610853195 - rollout_corr/rollout_is_eff_sample_size:0.9989161330886634 - rollout_corr/rollout_is_seq_mean:0.9998683333396912 - rollout_corr/rollout_is_seq_std:0.004356196615844965 - rollout_corr/rollout_is_seq_max:1.019521951675415 - rollout_corr/rollout_is_seq_min:0.9868797659873962 - rollout_corr/rollout_is_seq_max_deviation:0.01952195167541504 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0663551758043468) - rollout_corr/training_log_ppl:np.float64(0.06292674879159676) - rollout_corr/kl:np.float64(0.0008353343489628795) - rollout_corr/k3_kl:np.float64(0.0011243757822083297) - rollout_corr/rollout_ppl:np.float64(1.065414234995842) - rollout_corr/rollout_log_ppl:np.float64(0.06209141379486027) - rollout_corr/log_ppl_diff:np.float64(0.000835334996736492) - rollout_corr/log_ppl_abs_diff:np.float64(0.004272670279533486) - rollout_corr/log_ppl_diff_max:np.float64(0.03366580605506897) - rollout_corr/log_ppl_diff_min:np.float64(-0.027683019638061523) - rollout_corr/ppl_ratio:np.float64(1.0008572218939662) - rollout_corr/chi2_token:np.float64(0.002693302696570754) - rollout_corr/chi2_seq:np.float64(0.3027184521779418) - actor/pg_loss:np.float64(-9.508256334811449e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.491847984259948e-05) - actor/ppo_kl:np.float64(0.0001946200343354576) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3588371900841594) - perf/mfu/actor:np.float64(0.05812026834110374) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(107.7638053894043) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.5234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0037103083450347185 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0037103083450347185 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:67.90625 - response_length/max:266.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:67.90625 - response_length_non_aborted/max:266.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:562.46875 - prompt_length/max:919.0 - prompt_length/min:263.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00019651837646961212 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3680893238633871) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7371354475617409) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7864918088016566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7371354475617409) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:358 - timing_s/agent_loop/slowest/response_length:266 - timing_s/gen:7.719304949045181 - timing_s/reward:0.06777948699891567 - timing_s/old_log_prob:2.484109837561846 - timing_s/adv:0.11778771877288818 - timing_s/update_actor:9.966723799705505 - timing_s/step:20.44154209829867 - timing_s/stop_profile:0.00010528787970542908 - timing_per_token_ms/update_actor:0.061760880178623245 - timing_per_token_ms/adv:0.0007298961355646947 - timing_per_token_ms/gen:0.44404653411442596 - perf/total_num_tokens:161376 - perf/time_per_step:20.44154209829867 - perf/throughput:986.8140037085997 (TaskRunner pid=2186821) Training Progress: 26%|██▌ | 26/100 [12:02<33:39, 27.29s/it] (TaskRunner pid=2186821) step:27 - global_seqlen/min:17466 - global_seqlen/max:29164 - global_seqlen/minmax_diff:11698 - global_seqlen/balanced_min:20570 - global_seqlen/balanced_max:20936 - global_seqlen/mean:20675.5 - actor/entropy:0.11803295463323593 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5083484649658203 - training/rollout_probs_diff_mean:0.0025574055034667253 - training/rollout_probs_diff_std:0.01195543073117733 - training/rollout_actor_probs_pearson_corr:0.9974974393844604 - rollout_corr/rollout_is_mean:1.0000547170639038 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0002340385690331459 - rollout_corr/rollout_is_max:1.6482614278793335 - rollout_corr/rollout_is_min:0.2417639046907425 - rollout_corr/rollout_is_std:0.0319945327937603 - rollout_corr/rollout_is_eff_sample_size:0.9989776345500216 - rollout_corr/rollout_is_seq_mean:0.9999533295631409 - rollout_corr/rollout_is_seq_std:0.004586139693856239 - rollout_corr/rollout_is_seq_max:1.0172306299209595 - rollout_corr/rollout_is_seq_min:0.9795685410499573 - rollout_corr/rollout_is_seq_max_deviation:0.020431458950042725 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1372413127683103) - rollout_corr/training_log_ppl:np.float64(0.07928600198010827) - rollout_corr/kl:np.float64(0.0015031445066711058) - rollout_corr/k3_kl:np.float64(0.0008379250390412873) - rollout_corr/rollout_ppl:np.float64(1.1352315209805965) - rollout_corr/rollout_log_ppl:np.float64(0.07778285632412008) - rollout_corr/log_ppl_diff:np.float64(0.0015031456559881917) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037975421919327346) - rollout_corr/log_ppl_diff_max:np.float64(0.029949679970741272) - rollout_corr/log_ppl_diff_min:np.float64(-0.013691864907741547) - rollout_corr/ppl_ratio:np.float64(1.001522270962596) - rollout_corr/chi2_token:np.float64(0.00010768184438347816) - rollout_corr/chi2_seq:np.float64(0.07508241990581155) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0008647177151033247) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.06010155784178026) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.37154006958008) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:83.453125 - response_length/max:1211.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:83.453125 - response_length_non_aborted/max:1211.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:562.65625 - prompt_length/max:938.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.183771908283234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.38778611458837986) - timing_s/agent_loop/generate_sequences/max:np.float64(6.374784026294947) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8545178418789874) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.374784026294947) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:594 - timing_s/agent_loop/slowest/response_length:1211 - timing_s/gen:12.180952291935682 - timing_s/reward:0.06439157575368881 - timing_s/old_log_prob:2.445002494379878 - timing_s/adv:0.15308405458927155 - timing_s/update_actor:9.922992508858442 - timing_s/step:24.854339936748147 - timing_s/stop_profile:0.00012011826038360596 - timing_per_token_ms/update_actor:0.05999245791430946 - timing_per_token_ms/adv:0.0009255160370321851 - timing_per_token_ms/gen:0.5701625300475418 - perf/total_num_tokens:165404 - perf/time_per_step:24.854339936748147 - perf/throughput:831.866790774453 (TaskRunner pid=2186821) Training Progress: 27%|██▋ | 27/100 [12:27<32:19, 26.57s/it] (TaskRunner pid=2186821) step:28 - global_seqlen/min:19642 - global_seqlen/max:26155 - global_seqlen/minmax_diff:6513 - global_seqlen/balanced_min:22360 - global_seqlen/balanced_max:22987 - global_seqlen/mean:22440.5 - actor/entropy:0.07071752846240997 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27728819847106934 - training/rollout_probs_diff_mean:0.0028314990922808647 - training/rollout_probs_diff_std:0.012502463534474373 - training/rollout_actor_probs_pearson_corr:0.996059238910675 - rollout_corr/rollout_is_mean:1.000273585319519 - rollout_corr/rollout_is_ratio_fraction_high:5.3225463489070535e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.511430263519287 - rollout_corr/rollout_is_min:0.5043056607246399 - rollout_corr/rollout_is_std:0.038419388234615326 - rollout_corr/rollout_is_eff_sample_size:0.9985268392369275 - rollout_corr/rollout_is_seq_mean:1.0004533529281616 - rollout_corr/rollout_is_seq_std:0.004857113119214773 - rollout_corr/rollout_is_seq_max:1.0251727104187012 - rollout_corr/rollout_is_seq_min:0.9895175099372864 - rollout_corr/rollout_is_seq_max_deviation:0.025172710418701172 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.067921708803624) - rollout_corr/training_log_ppl:np.float64(0.0640178765725068) - rollout_corr/kl:np.float64(0.0006492141450493705) - rollout_corr/k3_kl:np.float64(0.0009374625031384589) - rollout_corr/rollout_ppl:np.float64(1.0671715633943677) - rollout_corr/rollout_log_ppl:np.float64(0.06336866177298361) - rollout_corr/log_ppl_diff:np.float64(0.000649214799523179) - rollout_corr/log_ppl_abs_diff:np.float64(0.0038353906229531276) - rollout_corr/log_ppl_diff_max:np.float64(0.07548682391643524) - rollout_corr/log_ppl_diff_min:np.float64(-0.019205041229724884) - rollout_corr/ppl_ratio:np.float64(1.000673201866448) - rollout_corr/chi2_token:np.float64(0.0022449721582233906) - rollout_corr/chi2_seq:np.float64(0.21228693006560206) - actor/pg_loss:np.float64(4.5200344175100327e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016121282078529475) - actor/ppo_kl:np.float64(0.00046171173494613527) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5175500065088272) - perf/mfu/actor:np.float64(0.06538513659624791) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(106.19146728515625) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.30078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.047636788338422775 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.047636788338422775 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:73.390625 - response_length/max:1460.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:73.390625 - response_length_non_aborted/max:1460.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:627.875 - prompt_length/max:1106.0 - prompt_length/min:380.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015819258987903595 - timing_s/agent_loop/generate_sequences/min:np.float64(0.40291469171643257) - timing_s/agent_loop/generate_sequences/max:np.float64(7.467439124360681) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8463720413274132) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.467439124360681) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:669 - timing_s/agent_loop/slowest/response_length:1460 - timing_s/gen:13.573095802217722 - timing_s/reward:0.0977156050503254 - timing_s/old_log_prob:2.419363608583808 - timing_s/adv:0.13476705364882946 - timing_s/update_actor:9.975308978930116 - timing_s/step:26.28450295329094 - timing_s/stop_profile:0.00012133829295635223 - timing_per_token_ms/update_actor:0.05556532262499786 - timing_per_token_ms/adv:0.0007506910142868333 - timing_per_token_ms/gen:0.7224343092515287 - perf/total_num_tokens:179524 - perf/time_per_step:26.28450295329094 - perf/throughput:853.7540177144704 (TaskRunner pid=2186821) (TaskRunner pid=2186821) Training Progress: 28%|██▊ | 28/100 [12:54<31:49, 26.52s/it] (TaskRunner pid=2186821) step:29 - global_seqlen/min:15036 - global_seqlen/max:27275 - global_seqlen/minmax_diff:12239 - global_seqlen/balanced_min:19923 - global_seqlen/balanced_max:20114 - global_seqlen/mean:19971.0 - actor/entropy:0.07582876086235046 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4399607479572296 - training/rollout_probs_diff_mean:0.003591712797060609 - training/rollout_probs_diff_std:0.016582729294896126 - training/rollout_actor_probs_pearson_corr:0.9936688542366028 - rollout_corr/rollout_is_mean:0.9996539354324341 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00030108389910310507 - rollout_corr/rollout_is_max:1.7586421966552734 - rollout_corr/rollout_is_min:0.41507747769355774 - rollout_corr/rollout_is_std:0.040559686720371246 - rollout_corr/rollout_is_eff_sample_size:0.9983566631311757 - rollout_corr/rollout_is_seq_mean:0.9998385310173035 - rollout_corr/rollout_is_seq_std:0.005180018953979015 - rollout_corr/rollout_is_seq_max:1.0198853015899658 - rollout_corr/rollout_is_seq_min:0.9842075109481812 - rollout_corr/rollout_is_seq_max_deviation:0.01988530158996582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0759340664371848) - rollout_corr/training_log_ppl:np.float64(0.07084084819325653) - rollout_corr/kl:np.float64(0.0010461350252413126) - rollout_corr/k3_kl:np.float64(0.0009072819976125596) - rollout_corr/rollout_ppl:np.float64(1.0747522730380297) - rollout_corr/rollout_log_ppl:np.float64(0.06979471295926487) - rollout_corr/log_ppl_diff:np.float64(0.001046135233991663) - rollout_corr/log_ppl_abs_diff:np.float64(0.003849974991680938) - rollout_corr/log_ppl_diff_max:np.float64(0.023498207330703735) - rollout_corr/log_ppl_diff_min:np.float64(-0.017289936542510986) - rollout_corr/ppl_ratio:np.float64(1.0010618069209158) - rollout_corr/chi2_token:np.float64(0.0015023290179669857) - rollout_corr/chi2_seq:np.float64(0.05637039919383824) - actor/pg_loss:np.float64(-0.00046433694660663605) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015797335072420537) - actor/ppo_kl:np.float64(3.598459767317763e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3323812186717987) - perf/mfu/actor:np.float64(0.05735046944052229) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(104.62966918945312) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005519871483556926 - critic/advantages/max:0.5 - critic/advantages/min:-0.5 - critic/returns/mean:-0.0005519871483556926 - critic/returns/max:0.5 - critic/returns/min:-0.5 - response_length/mean:77.84375 - response_length/max:771.0 - response_length/min:34.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:77.84375 - response_length_non_aborted/max:771.0 - response_length_non_aborted/min:34.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:546.25 - prompt_length/max:973.0 - prompt_length/min:248.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010416656732559204 - timing_s/agent_loop/generate_sequences/min:np.float64(0.48116141743957996) - timing_s/agent_loop/generate_sequences/max:np.float64(4.179505605250597) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8550040024201735) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.179505605250597) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:869 - timing_s/agent_loop/slowest/response_length:771 - timing_s/gen:10.254074046388268 - timing_s/reward:0.06841097585856915 - timing_s/old_log_prob:2.5635030772536993 - timing_s/adv:0.12210474163293839 - timing_s/update_actor:10.09960120730102 - timing_s/step:23.193453731015325 - timing_s/stop_profile:0.00011895038187503815 - timing_per_token_ms/update_actor:0.06321416808936095 - timing_per_token_ms/adv:0.0007642628162894846 - timing_per_token_ms/gen:0.514556104294875 - perf/total_num_tokens:159768 - perf/time_per_step:23.193453731015325 - perf/throughput:861.061928577454 (TaskRunner pid=2186821) Training Progress: 29%|██▉ | 29/100 [13:17<30:13, 25.54s/it] (TaskRunner pid=2186821) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2186821) validation generation end (TaskRunner pid=2186821) [prompt] user (TaskRunner pid=2186821) Your task is to answer the user's question using available tools. (TaskRunner pid=2186821) You have access to the following tools: (TaskRunner pid=2186821) Name: Axolotl (TaskRunner pid=2186821) Description: Collection of axolotl pictures and facts (TaskRunner pid=2186821) Documentation: (TaskRunner pid=2186821) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2186821) Parameters: {} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{url, source, description} (TaskRunner pid=2186821) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2186821) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2186821) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2186821) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Array[Object{fact, source}] (TaskRunner pid=2186821) (TaskRunner pid=2186821) Use the following format: (TaskRunner pid=2186821) Thought: you should always think about what to do (TaskRunner pid=2186821) Action: the action to take, should be one of the tool names. (TaskRunner pid=2186821) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2186821) (TaskRunner pid=2186821) Begin! (TaskRunner pid=2186821) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2186821) assistant (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: I should retrieve a random axolotl image. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2187230) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2186821) step:30 - global_seqlen/min:19226 - global_seqlen/max:26898 - global_seqlen/minmax_diff:7672 - global_seqlen/balanced_min:21566 - global_seqlen/balanced_max:22171 - global_seqlen/mean:21817.625 - actor/entropy:0.06238436698913574 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7039055824279785 - training/rollout_probs_diff_mean:0.0026540097314864397 - training/rollout_probs_diff_std:0.01356723066419363 - training/rollout_actor_probs_pearson_corr:0.9944779872894287 - rollout_corr/rollout_is_mean:1.0004538297653198 - rollout_corr/rollout_is_ratio_fraction_high:0.0001689974160399288 - rollout_corr/rollout_is_ratio_fraction_low:0.0002534961386118084 - rollout_corr/rollout_is_max:3.4121248722076416 - rollout_corr/rollout_is_min:0.1737738400697708 - rollout_corr/rollout_is_std:0.03465764969587326 - rollout_corr/rollout_is_eff_sample_size:0.9988012396239597 - rollout_corr/rollout_is_seq_mean:1.000083565711975 - rollout_corr/rollout_is_seq_std:0.005635960027575493 - rollout_corr/rollout_is_seq_max:1.043095588684082 - rollout_corr/rollout_is_seq_min:0.9757270812988281 - rollout_corr/rollout_is_seq_max_deviation:0.04309558868408203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0580853130668402) - rollout_corr/training_log_ppl:np.float64(0.05492819545452221) - rollout_corr/kl:np.float64(0.0017055816426392312) - rollout_corr/k3_kl:np.float64(0.0016535084109373654) - rollout_corr/rollout_ppl:np.float64(1.056259830482304) - rollout_corr/rollout_log_ppl:np.float64(0.05322261318906385) - rollout_corr/log_ppl_diff:np.float64(0.0017055822654583608) - rollout_corr/log_ppl_abs_diff:np.float64(0.0049114449311673525) - rollout_corr/log_ppl_diff_max:np.float64(0.04566342383623123) - rollout_corr/log_ppl_diff_min:np.float64(-0.026509910821914673) - rollout_corr/ppl_ratio:np.float64(1.0017440812662244) - rollout_corr/chi2_token:np.float64(0.003007725579664111) - rollout_corr/chi2_seq:np.float64(0.3540997370146215) - actor/pg_loss:np.float64(0.00015358044765889645) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022342565353028476) - actor/ppo_kl:np.float64(0.0011215957858858872) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.46293783374130726) - perf/mfu/actor:np.float64(0.06281907614267182) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(104.64212036132812) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5909926470588235) - val-aux/tooluse/reward/std@16:np.float64(0.012859342546703338) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5954705882352942) - val-aux/tooluse/reward/best@2/std:np.float64(0.013220998226621249) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5876029411764706) - val-aux/tooluse/reward/worst@2/std:np.float64(0.008763174760187467) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5916323529411764) - val-aux/tooluse/reward/maj@2/std:np.float64(0.012651906992079987) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6000588235294118) - val-aux/tooluse/reward/best@4/std:np.float64(0.013506587183457316) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5853529411764706) - val-aux/tooluse/reward/worst@4/std:np.float64(0.005837772244194399) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5908235294117647) - val-aux/tooluse/reward/maj@4/std:np.float64(0.0082866439864225) - val-aux/tooluse/reward/best@8/mean:np.float64(0.606014705882353) - val-aux/tooluse/reward/best@8/std:np.float64(0.013056772908291563) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.582529411764706) - val-aux/tooluse/reward/worst@8/std:np.float64(0.0071660966485577494) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.59025) - val-aux/tooluse/reward/maj@8/std:np.float64(0.00598619518189356) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6118823529411765) - val-aux/tooluse/reward/best@16/std:np.float64(0.009539283558630249) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.579) - val-aux/tooluse/reward/worst@16/std:np.float64(0.0071079175114432055) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5894558823529411) - val-aux/tooluse/reward/maj@16/std:np.float64(0.004057091488921709) - val-aux/tooluse/score/mean@16:np.float64(0.5909926470588235) - val-aux/tooluse/score/std@16:np.float64(0.012859342546703338) - val-aux/tooluse/score/best@2/mean:np.float64(0.5954705882352942) - val-aux/tooluse/score/best@2/std:np.float64(0.013220998226621249) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5876029411764706) - val-aux/tooluse/score/worst@2/std:np.float64(0.008763174760187467) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5916323529411764) - val-aux/tooluse/score/maj@2/std:np.float64(0.012651906992079987) - val-aux/tooluse/score/best@4/mean:np.float64(0.6000588235294118) - val-aux/tooluse/score/best@4/std:np.float64(0.013506587183457316) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5853529411764706) - val-aux/tooluse/score/worst@4/std:np.float64(0.005837772244194399) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5908235294117647) - val-aux/tooluse/score/maj@4/std:np.float64(0.0082866439864225) - val-aux/tooluse/score/best@8/mean:np.float64(0.606014705882353) - val-aux/tooluse/score/best@8/std:np.float64(0.013056772908291563) - val-aux/tooluse/score/worst@8/mean:np.float64(0.582529411764706) - val-aux/tooluse/score/worst@8/std:np.float64(0.0071660966485577494) - val-aux/tooluse/score/maj@8/mean:np.float64(0.59025) - val-aux/tooluse/score/maj@8/std:np.float64(0.00598619518189356) - val-aux/tooluse/score/best@16/mean:np.float64(0.6118823529411765) - val-aux/tooluse/score/best@16/std:np.float64(0.009539283558630249) - val-aux/tooluse/score/worst@16/mean:np.float64(0.579) - val-aux/tooluse/score/worst@16/std:np.float64(0.0071079175114432055) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5894558823529411) - val-aux/tooluse/score/maj@16/std:np.float64(0.004057091488921709) - val-core/tooluse/acc/mean@16:np.float64(0.5909926470588235) - val-aux/tooluse/acc/std@16:np.float64(0.012859342546703338) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5954705882352942) - val-aux/tooluse/acc/best@2/std:np.float64(0.013220998226621249) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5876029411764706) - val-aux/tooluse/acc/worst@2/std:np.float64(0.008763174760187467) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5916323529411764) - val-aux/tooluse/acc/maj@2/std:np.float64(0.012651906992079987) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6000588235294118) - val-aux/tooluse/acc/best@4/std:np.float64(0.013506587183457316) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5853529411764706) - val-aux/tooluse/acc/worst@4/std:np.float64(0.005837772244194399) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5908235294117647) - val-aux/tooluse/acc/maj@4/std:np.float64(0.0082866439864225) - val-aux/tooluse/acc/best@8/mean:np.float64(0.606014705882353) - val-aux/tooluse/acc/best@8/std:np.float64(0.013056772908291563) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.582529411764706) - val-aux/tooluse/acc/worst@8/std:np.float64(0.0071660966485577494) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.59025) - val-aux/tooluse/acc/maj@8/std:np.float64(0.00598619518189356) - val-core/tooluse/acc/best@16/mean:np.float64(0.6118823529411765) - val-core/tooluse/acc/best@16/std:np.float64(0.009539283558630249) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.579) - val-aux/tooluse/acc/worst@16/std:np.float64(0.0071079175114432055) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5894558823529411) - val-core/tooluse/acc/maj@16/std:np.float64(0.004057091488921709) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005973002873361111 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005973002873361111 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:92.45703125 - response_length/max:1304.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:92.45703125 - response_length_non_aborted/max:1304.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:589.34375 - prompt_length/max:920.0 - prompt_length/min:237.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010444968938827515 - timing_s/agent_loop/generate_sequences/min:np.float64(0.40952254831790924) - timing_s/agent_loop/generate_sequences/max:np.float64(6.932171165943146) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9622031442122534) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.932171165943146) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:878 - timing_s/agent_loop/slowest/response_length:1304 - timing_s/gen:12.870617235079408 - timing_s/reward:0.07138467766344547 - timing_s/old_log_prob:2.45608664304018 - timing_s/adv:0.12734755128622055 - timing_s/update_actor:10.13449834100902 - timing_s/step:25.741351233795285 - timing_s/testing:19.185494609177113 - timing_s/save_checkpoint:6.638724008575082 - timing_s/stop_profile:0.00011019594967365265 - timing_per_token_ms/update_actor:0.05806371191301196 - timing_per_token_ms/adv:0.0007296139662670693 - timing_per_token_ms/gen:0.5437752856089995 - perf/total_num_tokens:174541 - perf/time_per_step:25.741351233795285 - perf/throughput:847.5710852100138 (TaskRunner pid=2186821) Training Progress: 30%|███ | 30/100 [14:09<38:55, 33.36s/it] (TaskRunner pid=2186821) step:31 - global_seqlen/min:19774 - global_seqlen/max:29175 - global_seqlen/minmax_diff:9401 - global_seqlen/balanced_min:23005 - global_seqlen/balanced_max:23006 - global_seqlen/mean:23005.375 - actor/entropy:0.03996901959180832 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17960777878761292 - training/rollout_probs_diff_mean:0.0019618344958871603 - training/rollout_probs_diff_std:0.01042963471263647 - training/rollout_actor_probs_pearson_corr:0.9953662753105164 - rollout_corr/rollout_is_mean:0.9997086524963379 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.267627759370953e-05 - rollout_corr/rollout_is_max:1.9287195205688477 - rollout_corr/rollout_is_min:0.4071235954761505 - rollout_corr/rollout_is_std:0.02685657888650894 - rollout_corr/rollout_is_eff_sample_size:0.9992787083888894 - rollout_corr/rollout_is_seq_mean:0.9997310638427734 - rollout_corr/rollout_is_seq_std:0.0035869060084223747 - rollout_corr/rollout_is_seq_max:1.0151323080062866 - rollout_corr/rollout_is_seq_min:0.986702561378479 - rollout_corr/rollout_is_seq_max_deviation:0.015132308006286621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.041689793113619) - rollout_corr/training_log_ppl:np.float64(0.040028332825556845) - rollout_corr/kl:np.float64(0.0014436434921676344) - rollout_corr/k3_kl:np.float64(0.0017771313949621753) - rollout_corr/rollout_ppl:np.float64(1.0401167469099164) - rollout_corr/rollout_log_ppl:np.float64(0.03858468905241352) - rollout_corr/log_ppl_diff:np.float64(0.0014436437640483746) - rollout_corr/log_ppl_abs_diff:np.float64(0.004751489484931426) - rollout_corr/log_ppl_diff_max:np.float64(0.0360741950571537) - rollout_corr/log_ppl_diff_min:np.float64(-0.036473751068115234) - rollout_corr/ppl_ratio:np.float64(1.0014785744715482) - rollout_corr/chi2_token:np.float64(0.004648031201213598) - rollout_corr/chi2_seq:np.float64(0.19924632366746664) - actor/pg_loss:np.float64(-0.0004911237629130483) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008345951882802183) - actor/ppo_kl:np.float64(0.0008265877906108443) - actor/pg_clipfrac_lower:np.float64(8.311169949593022e-05) - actor/grad_norm:np.float64(0.34211142361164093) - perf/mfu/actor:np.float64(0.0673868177896328) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(105.53921890258789) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005139454733580351 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005139454733580351 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:62.32421875 - response_length/max:210.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:62.32421875 - response_length_non_aborted/max:210.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:656.59375 - prompt_length/max:1263.0 - prompt_length/min:387.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.692243576049805e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.33299739845097065) - timing_s/agent_loop/generate_sequences/max:np.float64(1.530181709676981) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7185311768771498) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.530181709676981) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:650 - timing_s/agent_loop/slowest/response_length:210 - timing_s/gen:7.8236676305532455 - timing_s/reward:0.06585505791008472 - timing_s/old_log_prob:2.5302583891898394 - timing_s/adv:0.17716680094599724 - timing_s/update_actor:9.842310443520546 - timing_s/step:20.54083206690848 - timing_s/stop_profile:0.00011724978685379028 - timing_per_token_ms/update_actor:0.05347831997696487 - timing_per_token_ms/adv:0.0009626380842846358 - timing_per_token_ms/gen:0.4903583597965055 - perf/total_num_tokens:184043 - perf/time_per_step:20.54083206690848 - perf/throughput:1119.982624124654 (TaskRunner pid=2186821) Training Progress: 31%|███ | 31/100 [14:29<33:57, 29.53s/it] (TaskRunner pid=2186821) step:32 - global_seqlen/min:20077 - global_seqlen/max:30353 - global_seqlen/minmax_diff:10276 - global_seqlen/balanced_min:22764 - global_seqlen/balanced_max:22766 - global_seqlen/mean:22764.75 - actor/entropy:0.058379802852869034 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.215124249458313 - training/rollout_probs_diff_mean:0.0027994606643915176 - training/rollout_probs_diff_std:0.012955953367054462 - training/rollout_actor_probs_pearson_corr:0.9946268200874329 - rollout_corr/rollout_is_mean:1.0002855062484741 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00026770177646540105 - rollout_corr/rollout_is_max:1.9838507175445557 - rollout_corr/rollout_is_min:0.47813889384269714 - rollout_corr/rollout_is_std:0.03440564125776291 - rollout_corr/rollout_is_eff_sample_size:0.9988184838515212 - rollout_corr/rollout_is_seq_mean:1.00044584274292 - rollout_corr/rollout_is_seq_std:0.005109013989567757 - rollout_corr/rollout_is_seq_max:1.0229918956756592 - rollout_corr/rollout_is_seq_min:0.977187991142273 - rollout_corr/rollout_is_seq_max_deviation:0.02299189567565918 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0563389654271305) - rollout_corr/training_log_ppl:np.float64(0.05326080689445689) - rollout_corr/kl:np.float64(-0.0001750273929412849) - rollout_corr/k3_kl:np.float64(0.0007903854122223528) - rollout_corr/rollout_ppl:np.float64(1.0564603647217155) - rollout_corr/rollout_log_ppl:np.float64(0.053435834034416985) - rollout_corr/log_ppl_diff:np.float64(-0.00017502713996009334) - rollout_corr/log_ppl_abs_diff:np.float64(0.004308496162195752) - rollout_corr/log_ppl_diff_max:np.float64(0.03538075089454651) - rollout_corr/log_ppl_diff_min:np.float64(-0.024942420423030853) - rollout_corr/ppl_ratio:np.float64(0.9998457913752645) - rollout_corr/chi2_token:np.float64(0.00362777104601264) - rollout_corr/chi2_seq:np.float64(0.14483983302488923) - actor/pg_loss:np.float64(-6.408430635929108e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.547965469711926e-05) - actor/ppo_kl:np.float64(-0.00034023252456172326) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.28764557279646397) - perf/mfu/actor:np.float64(0.06682677227071634) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(105.11217880249023) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.28125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003597242757678032 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0003597242757678032 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:58.3671875 - response_length/max:233.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:58.3671875 - response_length_non_aborted/max:233.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:653.03125 - prompt_length/max:1127.0 - prompt_length/min:356.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001139231026172638 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3185564037412405) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7221029493957758) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7222602762558381) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7221029493957758) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:655 - timing_s/agent_loop/slowest/response_length:233 - timing_s/gen:7.733005931600928 - timing_s/reward:0.0733408834785223 - timing_s/old_log_prob:2.400623183697462 - timing_s/adv:0.1432609986513853 - timing_s/update_actor:9.934849480167031 - timing_s/step:20.369870480149984 - timing_s/stop_profile:0.00011805444955825806 - timing_per_token_ms/update_actor:0.054551716360639976 - timing_per_token_ms/adv:0.0007866383259830731 - timing_per_token_ms/gen:0.517534863579235 - perf/total_num_tokens:182118 - perf/time_per_step:20.369870480149984 - perf/throughput:1117.56969796071 (TaskRunner pid=2186821) Training Progress: 32%|███▏ | 32/100 [14:50<30:22, 26.80s/it] (TaskRunner pid=2186821) step:33 - global_seqlen/min:16178 - global_seqlen/max:29326 - global_seqlen/minmax_diff:13148 - global_seqlen/balanced_min:21676 - global_seqlen/balanced_max:21678 - global_seqlen/mean:21676.75 - actor/entropy:0.0693797692656517 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20558980107307434 - training/rollout_probs_diff_mean:0.002638759557157755 - training/rollout_probs_diff_std:0.01155263464897871 - training/rollout_actor_probs_pearson_corr:0.9964811205863953 - rollout_corr/rollout_is_mean:0.9993718266487122 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7535778284072876 - rollout_corr/rollout_is_min:0.550489604473114 - rollout_corr/rollout_is_std:0.030473601073026657 - rollout_corr/rollout_is_eff_sample_size:0.9990710313267293 - rollout_corr/rollout_is_seq_mean:0.9992624521255493 - rollout_corr/rollout_is_seq_std:0.004016957711428404 - rollout_corr/rollout_is_seq_max:1.0129367113113403 - rollout_corr/rollout_is_seq_min:0.9828990697860718 - rollout_corr/rollout_is_seq_max_deviation:0.017100930213928223 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0694485981948674) - rollout_corr/training_log_ppl:np.float64(0.06431107393132152) - rollout_corr/kl:np.float64(0.0013496077967758424) - rollout_corr/k3_kl:np.float64(0.0008158942239892553) - rollout_corr/rollout_ppl:np.float64(1.0679213302209973) - rollout_corr/rollout_log_ppl:np.float64(0.06296146580757522) - rollout_corr/log_ppl_diff:np.float64(0.0013496081237462931) - rollout_corr/log_ppl_abs_diff:np.float64(0.0033077265543397516) - rollout_corr/log_ppl_diff_max:np.float64(0.030497655272483826) - rollout_corr/log_ppl_diff_min:np.float64(-0.02234750986099243) - rollout_corr/ppl_ratio:np.float64(1.0013646930456161) - rollout_corr/chi2_token:np.float64(0.0004525245167315006) - rollout_corr/chi2_seq:np.float64(0.02955265948548913) - actor/pg_loss:np.float64(0.00014979462139308453) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005366694895201363) - actor/ppo_kl:np.float64(0.00013419862975272423) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22239085286855698) - perf/mfu/actor:np.float64(0.06334595781035764) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(105.15528106689453) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.2265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002715145703405142 - critic/advantages/max:0.75 - critic/advantages/min:-0.25 - critic/returns/mean:0.002715145703405142 - critic/returns/max:0.75 - critic/returns/min:-0.25 - response_length/mean:66.1796875 - response_length/max:336.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:66.1796875 - response_length_non_aborted/max:336.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:611.21875 - prompt_length/max:1162.0 - prompt_length/min:306.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.579584002494812e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3692999556660652) - timing_s/agent_loop/generate_sequences/max:np.float64(2.191156793385744) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.794952805248613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.191156793385744) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:725 - timing_s/agent_loop/slowest/response_length:336 - timing_s/gen:7.94648259319365 - timing_s/reward:0.07058548182249069 - timing_s/old_log_prob:2.3656924217939377 - timing_s/adv:0.12340663187205791 - timing_s/update_actor:9.866736637428403 - timing_s/step:20.458989161998034 - timing_s/stop_profile:0.000116715207695961 - timing_per_token_ms/update_actor:0.056897001611337046 - timing_per_token_ms/adv:0.0007116301559969663 - timing_per_token_ms/gen:0.46904040805062275 - perf/total_num_tokens:173414 - perf/time_per_step:20.458989161998034 - perf/throughput:1059.5220432622311 (TaskRunner pid=2186821) Training Progress: 33%|███▎ | 33/100 [15:10<27:49, 24.91s/it] (TaskRunner pid=2186821) step:34 - global_seqlen/min:14444 - global_seqlen/max:27751 - global_seqlen/minmax_diff:13307 - global_seqlen/balanced_min:20350 - global_seqlen/balanced_max:20354 - global_seqlen/mean:20352.5 - actor/entropy:0.059980034828186035 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30164065957069397 - training/rollout_probs_diff_mean:0.0026671173982322216 - training/rollout_probs_diff_std:0.013040225952863693 - training/rollout_actor_probs_pearson_corr:0.9948471784591675 - rollout_corr/rollout_is_mean:0.9997946619987488 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.862475856905803e-05 - rollout_corr/rollout_is_max:1.7989736795425415 - rollout_corr/rollout_is_min:0.39213547110557556 - rollout_corr/rollout_is_std:0.03193019703030586 - rollout_corr/rollout_is_eff_sample_size:0.9989809655996994 - rollout_corr/rollout_is_seq_mean:0.9996563196182251 - rollout_corr/rollout_is_seq_std:0.004905271343886852 - rollout_corr/rollout_is_seq_max:1.0206897258758545 - rollout_corr/rollout_is_seq_min:0.9807711243629456 - rollout_corr/rollout_is_seq_max_deviation:0.020689725875854492 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.063439808320254) - rollout_corr/training_log_ppl:np.float64(0.0590252774886153) - rollout_corr/kl:np.float64(0.004505336012158523) - rollout_corr/k3_kl:np.float64(0.0028327311513623547) - rollout_corr/rollout_ppl:np.float64(1.0584965557791293) - rollout_corr/rollout_log_ppl:np.float64(0.0545199412074453) - rollout_corr/log_ppl_diff:np.float64(0.0045053363029978755) - rollout_corr/log_ppl_abs_diff:np.float64(0.006607099527116134) - rollout_corr/log_ppl_diff_max:np.float64(0.08666488528251648) - rollout_corr/log_ppl_diff_min:np.float64(-0.022808261215686798) - rollout_corr/ppl_ratio:np.float64(1.0046292732004076) - rollout_corr/chi2_token:np.float64(-0.0004916850011795759) - rollout_corr/chi2_seq:np.float64(-0.04734163451939821) - actor/pg_loss:np.float64(0.00031800323631614447) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012070769662386738) - actor/ppo_kl:np.float64(0.0036442531011715573) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.42901158053427935) - perf/mfu/actor:np.float64(0.0595971830165199) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(105.2801284790039) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016727285692468286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016727285692468286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:56.921875 - response_length/max:396.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:56.921875 - response_length_non_aborted/max:396.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:579.09375 - prompt_length/max:1092.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011215731501579285 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3856560830026865) - timing_s/agent_loop/generate_sequences/max:np.float64(2.39207573980093) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7164672844373854) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.39207573980093) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:544 - timing_s/agent_loop/slowest/response_length:396 - timing_s/gen:8.13699022680521 - timing_s/reward:0.0686158835887909 - timing_s/old_log_prob:2.3138375636190176 - timing_s/adv:0.126399464905262 - timing_s/update_actor:9.841181263327599 - timing_s/step:20.574989868327975 - timing_s/stop_profile:0.000116029754281044 - timing_per_token_ms/update_actor:0.060442091041196404 - timing_per_token_ms/adv:0.0007763141193051344 - timing_per_token_ms/gen:0.5583989999180078 - perf/total_num_tokens:162820 - perf/time_per_step:20.574989868327975 - perf/throughput:989.1863923262259 (TaskRunner pid=2186821) Training Progress: 34%|███▍ | 34/100 [15:31<25:59, 23.62s/it] (TaskRunner pid=2186821) step:35 - global_seqlen/min:15227 - global_seqlen/max:24733 - global_seqlen/minmax_diff:9506 - global_seqlen/balanced_min:19951 - global_seqlen/balanced_max:21079 - global_seqlen/mean:20144.0 - actor/entropy:0.08968596160411835 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31812596321105957 - training/rollout_probs_diff_mean:0.002727388869971037 - training/rollout_probs_diff_std:0.01201782375574112 - training/rollout_actor_probs_pearson_corr:0.9970560073852539 - rollout_corr/rollout_is_mean:0.9999868273735046 - rollout_corr/rollout_is_ratio_fraction_high:8.750437700655311e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.750437700655311e-05 - rollout_corr/rollout_is_max:2.5565640926361084 - rollout_corr/rollout_is_min:0.2724709212779999 - rollout_corr/rollout_is_std:0.03504587337374687 - rollout_corr/rollout_is_eff_sample_size:0.998773174430212 - rollout_corr/rollout_is_seq_mean:1.000199317932129 - rollout_corr/rollout_is_seq_std:0.005238836631178856 - rollout_corr/rollout_is_seq_max:1.0265891551971436 - rollout_corr/rollout_is_seq_min:0.9850262999534607 - rollout_corr/rollout_is_seq_max_deviation:0.026589155197143555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0985043994151056) - rollout_corr/training_log_ppl:np.float64(0.08988781498010212) - rollout_corr/kl:np.float64(0.0004833310352572173) - rollout_corr/k3_kl:np.float64(0.000792355003492248) - rollout_corr/rollout_ppl:np.float64(1.0979190492071211) - rollout_corr/rollout_log_ppl:np.float64(0.08940448312796434) - rollout_corr/log_ppl_diff:np.float64(0.0004833318521377805) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035221213611293933) - rollout_corr/log_ppl_diff_max:np.float64(0.01619189977645874) - rollout_corr/log_ppl_diff_min:np.float64(-0.013677679002285004) - rollout_corr/ppl_ratio:np.float64(1.0004962927196175) - rollout_corr/chi2_token:np.float64(0.002293122000992298) - rollout_corr/chi2_seq:np.float64(0.13364829821512103) - actor/pg_loss:np.float64(0.0008271452970802784) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003447730305197183) - actor/ppo_kl:np.float64(-2.4284116122430532e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.9031488941982388) - perf/mfu/actor:np.float64(0.05909075740494717) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(105.10081100463867) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.30078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003040777053683996 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003040777053683996 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:89.28125 - response_length/max:2139.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:89.28125 - response_length_non_aborted/max:2139.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:540.21875 - prompt_length/max:885.0 - prompt_length/min:308.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012391433119773865 - timing_s/agent_loop/generate_sequences/min:np.float64(0.30213432759046555) - timing_s/agent_loop/generate_sequences/max:np.float64(10.661705248057842) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8839926462824224) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.661705248057842) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:409 - timing_s/agent_loop/slowest/response_length:2139 - timing_s/gen:16.348600704222918 - timing_s/reward:0.06963882967829704 - timing_s/old_log_prob:2.36224221624434 - timing_s/adv:0.12294385582208633 - timing_s/update_actor:9.799568887799978 - timing_s/step:28.78938370384276 - timing_s/stop_profile:5.4035335779190063e-05 - timing_per_token_ms/update_actor:0.060809477312102726 - timing_per_token_ms/adv:0.0007629061744321283 - timing_per_token_ms/gen:0.7152870451620107 - perf/total_num_tokens:161152 - perf/time_per_step:28.78938370384276 - perf/throughput:699.7023697075952 (TaskRunner pid=2186821) Training Progress: 35%|███▌ | 35/100 [16:00<27:16, 25.18s/it] (TaskRunner pid=2186821) step:36 - global_seqlen/min:16676 - global_seqlen/max:27932 - global_seqlen/minmax_diff:11256 - global_seqlen/balanced_min:21549 - global_seqlen/balanced_max:21553 - global_seqlen/mean:21551.375 - actor/entropy:0.07761423289775848 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41011911630630493 - training/rollout_probs_diff_mean:0.003021328477188945 - training/rollout_probs_diff_std:0.013311835005879402 - training/rollout_actor_probs_pearson_corr:0.9957634210586548 - rollout_corr/rollout_is_mean:1.000321865081787 - rollout_corr/rollout_is_ratio_fraction_high:5.8048412029165775e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0002321936481166631 - rollout_corr/rollout_is_max:2.495490789413452 - rollout_corr/rollout_is_min:0.3960106372833252 - rollout_corr/rollout_is_std:0.035987354815006256 - rollout_corr/rollout_is_eff_sample_size:0.9987074176748162 - rollout_corr/rollout_is_seq_mean:1.0003538131713867 - rollout_corr/rollout_is_seq_std:0.004218936897814274 - rollout_corr/rollout_is_seq_max:1.024636149406433 - rollout_corr/rollout_is_seq_min:0.9873470664024353 - rollout_corr/rollout_is_seq_max_deviation:0.024636149406433105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.070111496374011) - rollout_corr/training_log_ppl:np.float64(0.06521142009137293) - rollout_corr/kl:np.float64(0.0006779064406963187) - rollout_corr/k3_kl:np.float64(0.0012826667600925054) - rollout_corr/rollout_ppl:np.float64(1.0693590212613344) - rollout_corr/rollout_log_ppl:np.float64(0.0645335132899163) - rollout_corr/log_ppl_diff:np.float64(0.0006779068087325868) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036133930468622566) - rollout_corr/log_ppl_diff_max:np.float64(0.05608616769313812) - rollout_corr/log_ppl_diff_min:np.float64(-0.02145320177078247) - rollout_corr/ppl_ratio:np.float64(1.000697374343872) - rollout_corr/chi2_token:np.float64(0.00475959712639451) - rollout_corr/chi2_seq:np.float64(0.12739462754689157) - actor/pg_loss:np.float64(0.00013416889123618603) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003608425431593787) - actor/ppo_kl:np.float64(0.00046725354701693256) - actor/pg_clipfrac_lower:np.float64(0.00010813219614647096) - actor/grad_norm:np.float64(0.6092813964933157) - perf/mfu/actor:np.float64(0.06314133162780663) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(105.17453002929688) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007546293782070279 - critic/advantages/max:0.5 - critic/advantages/min:-0.75 - critic/returns/mean:0.0007546293782070279 - critic/returns/max:0.5 - critic/returns/min:-0.75 - response_length/mean:67.29296875 - response_length/max:490.0 - response_length/min:28.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:67.29296875 - response_length_non_aborted/max:490.0 - response_length_non_aborted/min:28.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:606.1875 - prompt_length/max:1262.0 - prompt_length/min:275.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010015442967414856 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4382358267903328) - timing_s/agent_loop/generate_sequences/max:np.float64(2.827908920124173) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7667157535033766) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.827908920124173) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:538 - timing_s/agent_loop/slowest/response_length:490 - timing_s/gen:8.673977378755808 - timing_s/reward:0.07145562395453453 - timing_s/old_log_prob:2.3388969115912914 - timing_s/adv:0.12017137371003628 - timing_s/update_actor:9.798165818676353 - timing_s/step:21.088564475998282 - timing_s/stop_profile:0.00011042505502700806 - timing_per_token_ms/update_actor:0.05683028239889771 - timing_per_token_ms/adv:0.0006970052590034063 - timing_per_token_ms/gen:0.503510615821432 - perf/total_num_tokens:172411 - perf/time_per_step:21.088564475998282 - perf/throughput:1021.9460421086727 (TaskRunner pid=2186821) Training Progress: 36%|███▌ | 36/100 [16:21<25:34, 23.97s/it] (TaskRunner pid=2186821) step:37 - global_seqlen/min:17678 - global_seqlen/max:25837 - global_seqlen/minmax_diff:8159 - global_seqlen/balanced_min:22043 - global_seqlen/balanced_max:22044 - global_seqlen/mean:22043.875 - actor/entropy:0.053846292197704315 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102310240268707 - training/rollout_probs_diff_mean:0.0019501528004184365 - training/rollout_probs_diff_std:0.010592268779873848 - training/rollout_actor_probs_pearson_corr:0.9961838722229004 - rollout_corr/rollout_is_mean:1.0001059770584106 - rollout_corr/rollout_is_ratio_fraction_high:6.089021553634666e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.089021553634666e-05 - rollout_corr/rollout_is_max:2.2787020206451416 - rollout_corr/rollout_is_min:0.48018404841423035 - rollout_corr/rollout_is_std:0.027188541367650032 - rollout_corr/rollout_is_eff_sample_size:0.9992616863331922 - rollout_corr/rollout_is_seq_mean:1.0002007484436035 - rollout_corr/rollout_is_seq_std:0.0036662432830780745 - rollout_corr/rollout_is_seq_max:1.0102393627166748 - rollout_corr/rollout_is_seq_min:0.9874526858329773 - rollout_corr/rollout_is_seq_max_deviation:0.012547314167022705 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0563712483271956) - rollout_corr/training_log_ppl:np.float64(0.05318279404536952) - rollout_corr/kl:np.float64(0.001255028403726044) - rollout_corr/k3_kl:np.float64(0.0015611843593314055) - rollout_corr/rollout_ppl:np.float64(1.0549211804755032) - rollout_corr/rollout_log_ppl:np.float64(0.051927765378877666) - rollout_corr/log_ppl_diff:np.float64(0.001255028666491853) - rollout_corr/log_ppl_abs_diff:np.float64(0.004249296282068826) - rollout_corr/log_ppl_diff_max:np.float64(0.1324537694454193) - rollout_corr/log_ppl_diff_min:np.float64(-0.03140898421406746) - rollout_corr/ppl_ratio:np.float64(1.0013139895163476) - rollout_corr/chi2_token:np.float64(0.0030906859319657087) - rollout_corr/chi2_seq:np.float64(0.12431538687087595) - actor/pg_loss:np.float64(-0.00024408008903265) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011908543528988957) - actor/ppo_kl:np.float64(0.001073729523811906) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4571026675403118) - perf/mfu/actor:np.float64(0.06499115126884214) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(105.23357439041138) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:3.805638334597461e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.375 - critic/returns/mean:3.805638334597461e-05 - critic/returns/max:0.75 - critic/returns/min:-0.375 - response_length/mean:64.15234375 - response_length/max:271.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:64.15234375 - response_length_non_aborted/max:271.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:624.71875 - prompt_length/max:1226.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.803473949432373e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3923158682882786) - timing_s/agent_loop/generate_sequences/max:np.float64(1.9276735410094261) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7800679888678133) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.9276735410094261) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:602 - timing_s/agent_loop/slowest/response_length:271 - timing_s/gen:7.8111619502305984 - timing_s/reward:0.07087624818086624 - timing_s/old_log_prob:2.438014440238476 - timing_s/adv:0.18665758706629276 - timing_s/update_actor:9.909947611391544 - timing_s/step:20.5035115275532 - timing_s/stop_profile:0.00012186728417873383 - timing_per_token_ms/update_actor:0.056194450904114776 - timing_per_token_ms/adv:0.001058443598654347 - timing_per_token_ms/gen:0.4756233300998964 - perf/total_num_tokens:176351 - perf/time_per_step:20.5035115275532 - perf/throughput:1075.12681280847 (TaskRunner pid=2186821) Training Progress: 37%|███▋ | 37/100 [16:41<24:05, 22.95s/it] (TaskRunner pid=2186821) step:38 - global_seqlen/min:13258 - global_seqlen/max:26121 - global_seqlen/minmax_diff:12863 - global_seqlen/balanced_min:20379 - global_seqlen/balanced_max:20382 - global_seqlen/mean:20381.25 - actor/entropy:0.06500189751386642 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491897225379944 - training/rollout_probs_diff_mean:0.0023390362039208412 - training/rollout_probs_diff_std:0.010639370419085026 - training/rollout_actor_probs_pearson_corr:0.9967024326324463 - rollout_corr/rollout_is_mean:1.0000048875808716 - rollout_corr/rollout_is_ratio_fraction_high:5.3989850130164996e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.3989850130164996e-05 - rollout_corr/rollout_is_max:2.091334819793701 - rollout_corr/rollout_is_min:0.47679027915000916 - rollout_corr/rollout_is_std:0.02989501692354679 - rollout_corr/rollout_is_eff_sample_size:0.9991070859642656 - rollout_corr/rollout_is_seq_mean:0.9997793436050415 - rollout_corr/rollout_is_seq_std:0.0031527727842330933 - rollout_corr/rollout_is_seq_max:1.0111106634140015 - rollout_corr/rollout_is_seq_min:0.9876687526702881 - rollout_corr/rollout_is_seq_max_deviation:0.012331247329711914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.054818483069539) - rollout_corr/training_log_ppl:np.float64(0.05188429238990011) - rollout_corr/kl:np.float64(0.0015946714813068752) - rollout_corr/k3_kl:np.float64(0.0008704724540683495) - rollout_corr/rollout_ppl:np.float64(1.0530467741191387) - rollout_corr/rollout_log_ppl:np.float64(0.05028962024681505) - rollout_corr/log_ppl_diff:np.float64(0.0015946721449040524) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036676473699230883) - rollout_corr/log_ppl_diff_max:np.float64(0.07446032762527466) - rollout_corr/log_ppl_diff_min:np.float64(-0.015028625726699829) - rollout_corr/ppl_ratio:np.float64(1.0016227085143328) - rollout_corr/chi2_token:np.float64(-0.00014289445243775845) - rollout_corr/chi2_seq:np.float64(0.04593630717135966) - actor/pg_loss:np.float64(8.015218190848827e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017665459745330736) - actor/ppo_kl:np.float64(0.001015524675167967) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3129507377743721) - perf/mfu/actor:np.float64(0.059943721685694284) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(105.16881561279297) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00839542131870985 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:0.00839542131870985 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:72.3515625 - response_length/max:491.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:72.3515625 - response_length_non_aborted/max:491.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:564.5625 - prompt_length/max:1048.0 - prompt_length/min:236.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010813586413860321 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3235168755054474) - timing_s/agent_loop/generate_sequences/max:np.float64(3.0202493146061897) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7767814341204939) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.0202493146061897) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:603 - timing_s/agent_loop/slowest/response_length:491 - timing_s/gen:8.734145360067487 - timing_s/reward:0.06875830888748169 - timing_s/old_log_prob:2.37712980248034 - timing_s/adv:0.15083018876612186 - timing_s/update_actor:9.828686144202948 - timing_s/step:21.244715809822083 - timing_s/stop_profile:0.00011254660785198212 - timing_per_token_ms/update_actor:0.06028019714322568 - timing_per_token_ms/adv:0.0009250548222393245 - timing_per_token_ms/gen:0.47155519706659577 - perf/total_num_tokens:163050 - perf/time_per_step:21.244715809822083 - perf/throughput:959.3562080306635 (TaskRunner pid=2186821) Training Progress: 38%|███▊ | 38/100 [17:03<23:11, 22.45s/it] (TaskRunner pid=2186821) step:39 - global_seqlen/min:16428 - global_seqlen/max:23702 - global_seqlen/minmax_diff:7274 - global_seqlen/balanced_min:20682 - global_seqlen/balanced_max:20694 - global_seqlen/mean:20689.0 - actor/entropy:0.055495698004961014 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6983649730682373 - training/rollout_probs_diff_mean:0.002238169312477112 - training/rollout_probs_diff_std:0.011980739422142506 - training/rollout_actor_probs_pearson_corr:0.9951125979423523 - rollout_corr/rollout_is_mean:1.0000755786895752 - rollout_corr/rollout_is_ratio_fraction_high:9.626492101233453e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.626492101233453e-05 - rollout_corr/rollout_is_max:2.7193048000335693 - rollout_corr/rollout_is_min:0.20711462199687958 - rollout_corr/rollout_is_std:0.02946525439620018 - rollout_corr/rollout_is_eff_sample_size:0.9991326708663405 - rollout_corr/rollout_is_seq_mean:1.0002851486206055 - rollout_corr/rollout_is_seq_std:0.0035014813765883446 - rollout_corr/rollout_is_seq_max:1.0238215923309326 - rollout_corr/rollout_is_seq_min:0.9858936667442322 - rollout_corr/rollout_is_seq_max_deviation:0.023821592330932617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0499316733330488) - rollout_corr/training_log_ppl:np.float64(0.04757236606246806) - rollout_corr/kl:np.float64(0.0004891578880901193) - rollout_corr/k3_kl:np.float64(0.0009574129157077088) - rollout_corr/rollout_ppl:np.float64(1.049403002485633) - rollout_corr/rollout_log_ppl:np.float64(0.04708320748159167) - rollout_corr/log_ppl_diff:np.float64(0.0004891585804216447) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032806963981784065) - rollout_corr/log_ppl_diff_max:np.float64(0.04780048131942749) - rollout_corr/log_ppl_diff_min:np.float64(-0.01982712373137474) - rollout_corr/ppl_ratio:np.float64(1.00051209772937) - rollout_corr/chi2_token:np.float64(0.002779747359454632) - rollout_corr/chi2_seq:np.float64(0.13739958638325334) - actor/pg_loss:np.float64(-0.00018740887753665447) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002879022322304081) - actor/ppo_kl:np.float64(0.00038772055777691605) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3251549620181322) - perf/mfu/actor:np.float64(0.0605771512458298) - perf/max_memory_allocated_gb:np.float64(123.83254766464233) - perf/max_memory_reserved_gb:np.float64(129.6171875) - perf/cpu_memory_used_gb:np.float64(105.30107116699219) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012706969864666462 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012706969864666462 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:81.15625 - response_length/max:527.0 - response_length/min:28.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:81.15625 - response_length_non_aborted/max:527.0 - response_length_non_aborted/min:28.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:565.375 - prompt_length/max:954.0 - prompt_length/min:249.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016955658793449402 - timing_s/agent_loop/generate_sequences/min:np.float64(0.36986831575632095) - timing_s/agent_loop/generate_sequences/max:np.float64(2.838096482679248) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8149757949649938) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.838096482679248) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:824 - timing_s/agent_loop/slowest/response_length:481 - timing_s/gen:8.887258915230632 - timing_s/reward:0.0693649910390377 - timing_s/old_log_prob:2.369052767753601 - timing_s/adv:0.12555080838501453 - timing_s/update_actor:9.831901574507356 - timing_s/step:21.371477210894227 - timing_s/stop_profile:0.00011979974806308746 - timing_per_token_ms/update_actor:0.059402953106163635 - timing_per_token_ms/adv:0.0007585601550643732 - timing_per_token_ms/gen:0.4277656389695144 - perf/total_num_tokens:165512 - perf/time_per_step:21.371477210894227 - perf/throughput:968.0659785863407 (TaskRunner pid=2186821) Training Progress: 39%|███▉ | 39/100 [17:24<22:30, 22.14s/it] (TaskRunner pid=2186821) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2186821) validation generation end (TaskRunner pid=2186821) [prompt] user (TaskRunner pid=2186821) Your task is to answer the user's question using available tools. (TaskRunner pid=2186821) You have access to the following tools: (TaskRunner pid=2186821) Name: Axolotl (TaskRunner pid=2186821) Description: Collection of axolotl pictures and facts (TaskRunner pid=2186821) Documentation: (TaskRunner pid=2186821) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2186821) Parameters: {} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{url, source, description} (TaskRunner pid=2186821) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2186821) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2186821) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2186821) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Array[Object{fact, source}] (TaskRunner pid=2186821) (TaskRunner pid=2186821) Use the following format: (TaskRunner pid=2186821) Thought: you should always think about what to do (TaskRunner pid=2186821) Action: the action to take, should be one of the tool names. (TaskRunner pid=2186821) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2186821) (TaskRunner pid=2186821) Begin! (TaskRunner pid=2186821) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2186821) assistant (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: I need to retrieve a random axolotl image. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) Output: {"url": "https://example.com/axolotl.jpg", "source": "Example Source", "description": "A random axolotl image."} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (TaskRunner pid=2186821) step:40 - global_seqlen/min:17572 - global_seqlen/max:38084 - global_seqlen/minmax_diff:20512 - global_seqlen/balanced_min:22935 - global_seqlen/balanced_max:29987 - global_seqlen/mean:23848.25 - actor/entropy:0.07335927337408066 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30168360471725464 - training/rollout_probs_diff_mean:0.0021310201846063137 - training/rollout_probs_diff_std:0.010791613720357418 - training/rollout_actor_probs_pearson_corr:0.9971112608909607 - rollout_corr/rollout_is_mean:1.0000592470169067 - rollout_corr/rollout_is_ratio_fraction_high:2.9710618036915548e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.9421236073831096e-05 - rollout_corr/rollout_is_max:3.9915175437927246 - rollout_corr/rollout_is_min:0.39147502183914185 - rollout_corr/rollout_is_std:0.028655080124735832 - rollout_corr/rollout_is_eff_sample_size:0.9991793220604662 - rollout_corr/rollout_is_seq_mean:1.000290036201477 - rollout_corr/rollout_is_seq_std:0.004574858583509922 - rollout_corr/rollout_is_seq_max:1.027777075767517 - rollout_corr/rollout_is_seq_min:0.9840593934059143 - rollout_corr/rollout_is_seq_max_deviation:0.02777707576751709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0781929157674313) - rollout_corr/training_log_ppl:np.float64(0.07268419407239435) - rollout_corr/kl:np.float64(0.0005998070718362669) - rollout_corr/k3_kl:np.float64(0.0007844794648895759) - rollout_corr/rollout_ppl:np.float64(1.0774605688638985) - rollout_corr/rollout_log_ppl:np.float64(0.07208438607057133) - rollout_corr/log_ppl_diff:np.float64(0.0005998080018230212) - rollout_corr/log_ppl_abs_diff:np.float64(0.003549798686833583) - rollout_corr/log_ppl_diff_max:np.float64(0.06122833490371704) - rollout_corr/log_ppl_diff_min:np.float64(-0.014639295637607574) - rollout_corr/ppl_ratio:np.float64(1.0006191153079271) - rollout_corr/chi2_token:np.float64(0.002139686606824398) - rollout_corr/chi2_seq:np.float64(0.17847824771888554) - actor/pg_loss:np.float64(7.238984107971191e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006121165451986599) - actor/ppo_kl:np.float64(0.0003513268969848049) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4484660439193249) - perf/mfu/actor:np.float64(0.06574558948179025) - perf/max_memory_allocated_gb:np.float64(123.90003252029419) - perf/max_memory_reserved_gb:np.float64(129.65625) - perf/cpu_memory_used_gb:np.float64(105.1033821105957) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.6029411764705882) - val-aux/tooluse/reward/std@16:np.float64(0.02940272832919785) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6147499999999999) - val-aux/tooluse/reward/best@2/std:np.float64(0.022248753794127413) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5904999999999999) - val-aux/tooluse/reward/worst@2/std:np.float64(0.027611707925390205) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.6022500000000001) - val-aux/tooluse/reward/maj@2/std:np.float64(0.02944584121410833) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6234558823529412) - val-aux/tooluse/reward/best@4/std:np.float64(0.014052815089364564) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5796617647058824) - val-aux/tooluse/reward/worst@4/std:np.float64(0.02336001991482749) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.6049411764705882) - val-aux/tooluse/reward/maj@4/std:np.float64(0.02291053389830618) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6292941176470589) - val-aux/tooluse/reward/best@8/std:np.float64(0.007879264728912006) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5696323529411764) - val-aux/tooluse/reward/worst@8/std:np.float64(0.017057065204151636) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.6052205882352941) - val-aux/tooluse/reward/maj@8/std:np.float64(0.014745684027387577) - val-aux/tooluse/reward/best@16/mean:np.float64(0.631985294117647) - val-aux/tooluse/reward/best@16/std:np.float64(0.0028614693819481343) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5623382352941177) - val-aux/tooluse/reward/worst@16/std:np.float64(0.010161534800673043) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.6046617647058824) - val-aux/tooluse/reward/maj@16/std:np.float64(0.008845598082368821) - val-aux/tooluse/score/mean@16:np.float64(0.6029411764705882) - val-aux/tooluse/score/std@16:np.float64(0.02940272832919785) - val-aux/tooluse/score/best@2/mean:np.float64(0.6147499999999999) - val-aux/tooluse/score/best@2/std:np.float64(0.022248753794127413) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5904999999999999) - val-aux/tooluse/score/worst@2/std:np.float64(0.027611707925390205) - val-aux/tooluse/score/maj@2/mean:np.float64(0.6022500000000001) - val-aux/tooluse/score/maj@2/std:np.float64(0.02944584121410833) - val-aux/tooluse/score/best@4/mean:np.float64(0.6234558823529412) - val-aux/tooluse/score/best@4/std:np.float64(0.014052815089364564) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5796617647058824) - val-aux/tooluse/score/worst@4/std:np.float64(0.02336001991482749) - val-aux/tooluse/score/maj@4/mean:np.float64(0.6049411764705882) - val-aux/tooluse/score/maj@4/std:np.float64(0.02291053389830618) - val-aux/tooluse/score/best@8/mean:np.float64(0.6292941176470589) - val-aux/tooluse/score/best@8/std:np.float64(0.007879264728912006) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5696323529411764) - val-aux/tooluse/score/worst@8/std:np.float64(0.017057065204151636) - val-aux/tooluse/score/maj@8/mean:np.float64(0.6052205882352941) - val-aux/tooluse/score/maj@8/std:np.float64(0.014745684027387577) - val-aux/tooluse/score/best@16/mean:np.float64(0.631985294117647) - val-aux/tooluse/score/best@16/std:np.float64(0.0028614693819481343) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5623382352941177) - val-aux/tooluse/score/worst@16/std:np.float64(0.010161534800673043) - val-aux/tooluse/score/maj@16/mean:np.float64(0.6046617647058824) - val-aux/tooluse/score/maj@16/std:np.float64(0.008845598082368821) - val-core/tooluse/acc/mean@16:np.float64(0.6029411764705882) - val-aux/tooluse/acc/std@16:np.float64(0.02940272832919785) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6147499999999999) - val-aux/tooluse/acc/best@2/std:np.float64(0.022248753794127413) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5904999999999999) - val-aux/tooluse/acc/worst@2/std:np.float64(0.027611707925390205) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.6022500000000001) - val-aux/tooluse/acc/maj@2/std:np.float64(0.02944584121410833) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6234558823529412) - val-aux/tooluse/acc/best@4/std:np.float64(0.014052815089364564) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5796617647058824) - val-aux/tooluse/acc/worst@4/std:np.float64(0.02336001991482749) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.6049411764705882) - val-aux/tooluse/acc/maj@4/std:np.float64(0.02291053389830618) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6292941176470589) - val-aux/tooluse/acc/best@8/std:np.float64(0.007879264728912006) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5696323529411764) - val-aux/tooluse/acc/worst@8/std:np.float64(0.017057065204151636) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.6052205882352941) - val-aux/tooluse/acc/maj@8/std:np.float64(0.014745684027387577) - val-core/tooluse/acc/best@16/mean:np.float64(0.631985294117647) - val-core/tooluse/acc/best@16/std:np.float64(0.0028614693819481343) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5623382352941177) - val-aux/tooluse/acc/worst@16/std:np.float64(0.010161534800673043) - val-core/tooluse/acc/maj@16/mean:np.float64(0.6046617647058824) - val-core/tooluse/acc/maj@16/std:np.float64(0.008845598082368821) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.2890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016006595687940717 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016006595687940717 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:131.4765625 - response_length/max:8192.0 - response_length/min:24.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:131.4765625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:613.78125 - prompt_length/max:912.0 - prompt_length/min:307.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010418333113193512 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3271222449839115) - timing_s/agent_loop/generate_sequences/max:np.float64(46.09953301586211) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1392208699180628) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(46.09953301586211) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:672 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.11029217392206 - timing_s/reward:0.07384588196873665 - timing_s/old_log_prob:2.571855530142784 - timing_s/adv:0.12050843611359596 - timing_s/update_actor:10.665874579921365 - timing_s/step:65.6263302564621 - timing_s/testing:68.75736511871219 - timing_s/save_checkpoint:6.596431376412511 - timing_s/stop_profile:3.9482489228248596e-05 - timing_per_token_ms/update_actor:0.055904912204885915 - timing_per_token_ms/adv:0.0006316419240069814 - timing_per_token_ms/gen:1.5482290146153088 - perf/total_num_tokens:190786 - perf/time_per_step:65.6263302564621 - perf/throughput:363.39453854577994 (TaskRunner pid=2186821) Training Progress: 40%|████ | 40/100 [19:45<57:47, 57.80s/it] (WorkerDict pid=2187230) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2186821) step:41 - global_seqlen/min:15674 - global_seqlen/max:24578 - global_seqlen/minmax_diff:8904 - global_seqlen/balanced_min:20535 - global_seqlen/balanced_max:20541 - global_seqlen/mean:20539.375 - actor/entropy:0.07643657177686691 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917799949645996 - training/rollout_probs_diff_mean:0.0027147741056978703 - training/rollout_probs_diff_std:0.011642711237072945 - training/rollout_actor_probs_pearson_corr:0.9968329071998596 - rollout_corr/rollout_is_mean:0.9998001456260681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.1945353334303945e-05 - rollout_corr/rollout_is_max:1.729753017425537 - rollout_corr/rollout_is_min:0.4949631989002228 - rollout_corr/rollout_is_std:0.03315731883049011 - rollout_corr/rollout_is_eff_sample_size:0.9989016212755863 - rollout_corr/rollout_is_seq_mean:0.9995066523551941 - rollout_corr/rollout_is_seq_std:0.0038974119815975428 - rollout_corr/rollout_is_seq_max:1.0098634958267212 - rollout_corr/rollout_is_seq_min:0.98482745885849 - rollout_corr/rollout_is_seq_max_deviation:0.01517254114151001 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0758632854558527) - rollout_corr/training_log_ppl:np.float64(0.07077660156205923) - rollout_corr/kl:np.float64(0.0014577797956701888) - rollout_corr/k3_kl:np.float64(0.0007550243655709138) - rollout_corr/rollout_ppl:np.float64(1.0742474226281047) - rollout_corr/rollout_log_ppl:np.float64(0.06931882170171555) - rollout_corr/log_ppl_diff:np.float64(0.0014577798603436776) - rollout_corr/log_ppl_abs_diff:np.float64(0.003779194882822523) - rollout_corr/log_ppl_diff_max:np.float64(0.03274904936552048) - rollout_corr/log_ppl_diff_min:np.float64(-0.010052099823951721) - rollout_corr/ppl_ratio:np.float64(1.001476010074839) - rollout_corr/chi2_token:np.float64(-0.00011365464888513088) - rollout_corr/chi2_seq:np.float64(0.02725961757823825) - actor/pg_loss:np.float64(-0.00021655205637216568) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004216127999825403) - actor/ppo_kl:np.float64(0.0004809263298355404) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5103011401370168) - perf/mfu/actor:np.float64(0.06065802129387392) - perf/max_memory_allocated_gb:np.float64(123.90003252029419) - perf/max_memory_reserved_gb:np.float64(129.65625) - perf/cpu_memory_used_gb:np.float64(105.3956527709961) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.30859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008687860332429409 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008687860332429409 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:75.19921875 - response_length/max:766.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:75.19921875 - response_length_non_aborted/max:766.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:566.65625 - prompt_length/max:869.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.878531813621521e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.41349999606609344) - timing_s/agent_loop/generate_sequences/max:np.float64(4.30983848311007) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8175561821408337) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.30983848311007) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:390 - timing_s/agent_loop/slowest/response_length:766 - timing_s/gen:9.863125897943974 - timing_s/reward:0.06934075988829136 - timing_s/old_log_prob:2.39577972702682 - timing_s/adv:0.11989111825823784 - timing_s/update_actor:9.806306451559067 - timing_s/step:22.343010434880853 - timing_s/stop_profile:0.00011316686868667603 - timing_per_token_ms/update_actor:0.059679922414624755 - timing_per_token_ms/adv:0.0007296419575707503 - timing_per_token_ms/gen:0.5123435612666342 - perf/total_num_tokens:164315 - perf/time_per_step:22.343010434880853 - perf/throughput:919.2751827182115 (TaskRunner pid=2186821) Training Progress: 41%|████ | 41/100 [20:07<46:23, 47.18s/it] (TaskRunner pid=2186821) step:42 - global_seqlen/min:21165 - global_seqlen/max:34194 - global_seqlen/minmax_diff:13029 - global_seqlen/balanced_min:24539 - global_seqlen/balanced_max:31659 - global_seqlen/mean:25811.25 - actor/entropy:0.07392468303442001 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27729910612106323 - training/rollout_probs_diff_mean:0.0021082742605358362 - training/rollout_probs_diff_std:0.01012810692191124 - training/rollout_actor_probs_pearson_corr:0.9975377917289734 - rollout_corr/rollout_is_mean:0.9997605085372925 - rollout_corr/rollout_is_ratio_fraction_high:2.5400051526958123e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.620015094289556e-05 - rollout_corr/rollout_is_max:2.553802013397217 - rollout_corr/rollout_is_min:0.3235243856906891 - rollout_corr/rollout_is_std:0.02876407653093338 - rollout_corr/rollout_is_eff_sample_size:0.9991728953617344 - rollout_corr/rollout_is_seq_mean:0.9996140599250793 - rollout_corr/rollout_is_seq_std:0.004266318399459124 - rollout_corr/rollout_is_seq_max:1.0344802141189575 - rollout_corr/rollout_is_seq_min:0.9836998581886292 - rollout_corr/rollout_is_seq_max_deviation:0.03448021411895752 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0920946435071528) - rollout_corr/training_log_ppl:np.float64(0.08408006005447533) - rollout_corr/kl:np.float64(0.0005000570283248962) - rollout_corr/k3_kl:np.float64(0.0010528380998082731) - rollout_corr/rollout_ppl:np.float64(1.091538175009191) - rollout_corr/rollout_log_ppl:np.float64(0.08358000234875362) - rollout_corr/log_ppl_diff:np.float64(0.0005000577057217015) - rollout_corr/log_ppl_abs_diff:np.float64(0.0038723172820027685) - rollout_corr/log_ppl_diff_max:np.float64(0.023278705775737762) - rollout_corr/log_ppl_diff_min:np.float64(-0.03352736681699753) - rollout_corr/ppl_ratio:np.float64(1.000517011852935) - rollout_corr/chi2_token:np.float64(0.004048591945320368) - rollout_corr/chi2_seq:np.float64(0.18387177004478872) - actor/pg_loss:np.float64(-5.3570838645100594e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002444236888550222) - actor/ppo_kl:np.float64(-0.0004227498058924084) - actor/pg_clipfrac_lower:np.float64(9.300595411332324e-05) - actor/grad_norm:np.float64(0.4181612895336002) - perf/mfu/actor:np.float64(0.07086692709215626) - perf/max_memory_allocated_gb:np.float64(123.93576097488403) - perf/max_memory_reserved_gb:np.float64(129.6953125) - perf/cpu_memory_used_gb:np.float64(105.23678207397461) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.22265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.22265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003387731732800603 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:0.003387731732800603 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:153.7890625 - response_length/max:8192.0 - response_length/min:26.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:153.7890625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:652.8125 - prompt_length/max:977.0 - prompt_length/min:378.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010457634925842285 - timing_s/agent_loop/generate_sequences/min:np.float64(0.31972256302833557) - timing_s/agent_loop/generate_sequences/max:np.float64(46.44789435528219) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2525429144661757) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(46.44789435528219) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:699 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.59366038069129 - timing_s/reward:0.078092772513628 - timing_s/old_log_prob:2.7927729152143 - timing_s/adv:0.1697284672409296 - timing_s/update_actor:10.735485468059778 - timing_s/step:66.45596579648554 - timing_s/stop_profile:0.00010811164975166321 - timing_per_token_ms/update_actor:0.05199034078192541 - timing_per_token_ms/adv:0.0008219694282576861 - timing_per_token_ms/gen:1.3358816454328497 - perf/total_num_tokens:206490 - perf/time_per_step:66.45596579648554 - perf/throughput:388.3962815173623 (TaskRunner pid=2186821) Training Progress: 42%|████▏ | 42/100 [21:14<51:12, 52.98s/it] (TaskRunner pid=2186821) step:43 - global_seqlen/min:18354 - global_seqlen/max:25664 - global_seqlen/minmax_diff:7310 - global_seqlen/balanced_min:21209 - global_seqlen/balanced_max:21624 - global_seqlen/mean:21321.25 - actor/entropy:0.07183060795068741 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28977781534194946 - training/rollout_probs_diff_mean:0.0022768594790250063 - training/rollout_probs_diff_std:0.010436582379043102 - training/rollout_actor_probs_pearson_corr:0.9971587657928467 - rollout_corr/rollout_is_mean:1.000083327293396 - rollout_corr/rollout_is_ratio_fraction_high:4.535147309070453e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.535147309070453e-05 - rollout_corr/rollout_is_max:2.7140064239501953 - rollout_corr/rollout_is_min:0.45750218629837036 - rollout_corr/rollout_is_std:0.027441667392849922 - rollout_corr/rollout_is_eff_sample_size:0.9992476405980566 - rollout_corr/rollout_is_seq_mean:0.9999966621398926 - rollout_corr/rollout_is_seq_std:0.003775334218516946 - rollout_corr/rollout_is_seq_max:1.0180143117904663 - rollout_corr/rollout_is_seq_min:0.9826856255531311 - rollout_corr/rollout_is_seq_max_deviation:0.01801431179046631 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0670840749517083) - rollout_corr/training_log_ppl:np.float64(0.06295874141892455) - rollout_corr/kl:np.float64(0.0009728101756785179) - rollout_corr/k3_kl:np.float64(0.0007313828259920968) - rollout_corr/rollout_ppl:np.float64(1.0660361503250897) - rollout_corr/rollout_log_ppl:np.float64(0.06198593066574176) - rollout_corr/log_ppl_diff:np.float64(0.0009728107531827845) - rollout_corr/log_ppl_abs_diff:np.float64(0.003364504732644491) - rollout_corr/log_ppl_diff_max:np.float64(0.03172561526298523) - rollout_corr/log_ppl_diff_min:np.float64(-0.018520094454288483) - rollout_corr/ppl_ratio:np.float64(1.0009897574782372) - rollout_corr/chi2_token:np.float64(0.0010270671918988228) - rollout_corr/chi2_seq:np.float64(0.08471314096823335) - actor/pg_loss:np.float64(-0.0001882786164060235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.512019510613754e-05) - actor/ppo_kl:np.float64(0.0005932934972046056) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.282594732940197) - perf/mfu/actor:np.float64(0.062468442465150584) - perf/max_memory_allocated_gb:np.float64(123.93576097488403) - perf/max_memory_reserved_gb:np.float64(129.6953125) - perf/cpu_memory_used_gb:np.float64(105.21401596069336) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003684807161334902 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0003684807161334902 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:86.1328125 - response_length/max:1415.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.1328125 - response_length_non_aborted/max:1415.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:580.15625 - prompt_length/max:1266.0 - prompt_length/min:348.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016595236957073212 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4643787033855915) - timing_s/agent_loop/generate_sequences/max:np.float64(7.269553942605853) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9456485975315445) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.269553942605853) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:734 - timing_s/agent_loop/slowest/response_length:1415 - timing_s/gen:13.545456225052476 - timing_s/reward:0.05989343300461769 - timing_s/old_log_prob:2.328991327434778 - timing_s/adv:0.118406618013978 - timing_s/update_actor:9.972880678251386 - timing_s/step:26.058882731944323 - timing_s/stop_profile:5.457177758216858e-05 - timing_per_token_ms/update_actor:0.05846796434455875 - timing_per_token_ms/adv:0.0006941819664300756 - timing_per_token_ms/gen:0.6143064047642847 - perf/total_num_tokens:170570 - perf/time_per_step:26.058882731944323 - perf/throughput:818.1950937544729 (TaskRunner pid=2186821) Training Progress: 43%|████▎ | 43/100 [21:40<42:39, 44.91s/it] (TaskRunner pid=2186821) step:44 - global_seqlen/min:16718 - global_seqlen/max:24596 - global_seqlen/minmax_diff:7878 - global_seqlen/balanced_min:19782 - global_seqlen/balanced_max:19875 - global_seqlen/mean:19794.0 - actor/entropy:0.11136162281036377 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3994426727294922 - training/rollout_probs_diff_mean:0.003316521877422929 - training/rollout_probs_diff_std:0.0127424830570817 - training/rollout_actor_probs_pearson_corr:0.9972448945045471 - rollout_corr/rollout_is_mean:0.9998719096183777 - rollout_corr/rollout_is_ratio_fraction_high:4.708097912953235e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.41619582590647e-05 - rollout_corr/rollout_is_max:2.335594892501831 - rollout_corr/rollout_is_min:0.35784852504730225 - rollout_corr/rollout_is_std:0.034832630306482315 - rollout_corr/rollout_is_eff_sample_size:0.9987878014318493 - rollout_corr/rollout_is_seq_mean:0.9999310970306396 - rollout_corr/rollout_is_seq_std:0.004329073242843151 - rollout_corr/rollout_is_seq_max:1.0172615051269531 - rollout_corr/rollout_is_seq_min:0.9809362888336182 - rollout_corr/rollout_is_seq_max_deviation:0.019063711166381836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.099434219300747) - rollout_corr/training_log_ppl:np.float64(0.09056166440495872) - rollout_corr/kl:np.float64(0.0008932641900831761) - rollout_corr/k3_kl:np.float64(0.0006877755507560743) - rollout_corr/rollout_ppl:np.float64(1.0984748476184905) - rollout_corr/rollout_log_ppl:np.float64(0.0896683992468752) - rollout_corr/log_ppl_diff:np.float64(0.0008932651580835227) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032141390693141147) - rollout_corr/log_ppl_diff_max:np.float64(0.02444542944431305) - rollout_corr/log_ppl_diff_min:np.float64(-0.021616309881210327) - rollout_corr/ppl_ratio:np.float64(1.0009050138760358) - rollout_corr/chi2_token:np.float64(0.0010405783541500568) - rollout_corr/chi2_seq:np.float64(0.11621747259050608) - actor/pg_loss:np.float64(0.00010626553557813168) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00026321199129597517) - actor/ppo_kl:np.float64(0.0003092601962748631) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15711453557014465) - perf/mfu/actor:np.float64(0.05722604473924678) - perf/max_memory_allocated_gb:np.float64(123.93576097488403) - perf/max_memory_reserved_gb:np.float64(129.6953125) - perf/cpu_memory_used_gb:np.float64(105.15819549560547) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.020021187141537666 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.020021187141537666 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:82.96875 - response_length/max:875.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:82.96875 - response_length_non_aborted/max:875.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:535.59375 - prompt_length/max:966.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015789084136486053 - timing_s/agent_loop/generate_sequences/min:np.float64(0.372029734775424) - timing_s/agent_loop/generate_sequences/max:np.float64(4.6919932551681995) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8595984542262158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.6919932551681995) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:631 - timing_s/agent_loop/slowest/response_length:875 - timing_s/gen:10.455417986959219 - timing_s/reward:0.06863421201705933 - timing_s/old_log_prob:2.3291213549673557 - timing_s/adv:0.11922159977257252 - timing_s/update_actor:10.073063461109996 - timing_s/step:23.13018073886633 - timing_s/stop_profile:2.7492642402648926e-05 - timing_per_token_ms/update_actor:0.06361184867327217 - timing_per_token_ms/adv:0.000752889763138909 - timing_per_token_ms/gen:0.4922513176534472 - perf/total_num_tokens:158352 - perf/time_per_step:23.13018073886633 - perf/throughput:855.765038045706 (TaskRunner pid=2186821) Training Progress: 44%|████▍ | 44/100 [22:03<35:49, 38.39s/it] (TaskRunner pid=2186821) step:45 - global_seqlen/min:14085 - global_seqlen/max:26674 - global_seqlen/minmax_diff:12589 - global_seqlen/balanced_min:20169 - global_seqlen/balanced_max:20492 - global_seqlen/mean:20257.125 - actor/entropy:0.082231804728508 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28978031873703003 - training/rollout_probs_diff_mean:0.0025141704827547073 - training/rollout_probs_diff_std:0.011035851202905178 - training/rollout_actor_probs_pearson_corr:0.9973099827766418 - rollout_corr/rollout_is_mean:1.0000840425491333 - rollout_corr/rollout_is_ratio_fraction_high:3.800402919296175e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.60080583859235e-05 - rollout_corr/rollout_is_max:2.712303876876831 - rollout_corr/rollout_is_min:0.4166933298110962 - rollout_corr/rollout_is_std:0.03049706295132637 - rollout_corr/rollout_is_eff_sample_size:0.9990709123388344 - rollout_corr/rollout_is_seq_mean:0.9997838139533997 - rollout_corr/rollout_is_seq_std:0.0036828855518251657 - rollout_corr/rollout_is_seq_max:1.0171589851379395 - rollout_corr/rollout_is_seq_min:0.9791997075080872 - rollout_corr/rollout_is_seq_max_deviation:0.020800292491912842 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0860979747958481) - rollout_corr/training_log_ppl:np.float64(0.08043255114489511) - rollout_corr/kl:np.float64(0.00037733940664175236) - rollout_corr/k3_kl:np.float64(0.0005529123273552497) - rollout_corr/rollout_ppl:np.float64(1.0856703212484717) - rollout_corr/rollout_log_ppl:np.float64(0.08005521112045244) - rollout_corr/log_ppl_diff:np.float64(0.00037734002455636073) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029143495780772355) - rollout_corr/log_ppl_diff_max:np.float64(0.018863260746002197) - rollout_corr/log_ppl_diff_min:np.float64(-0.01456015557050705) - rollout_corr/ppl_ratio:np.float64(1.0003871629014611) - rollout_corr/chi2_token:np.float64(0.0014226087369024754) - rollout_corr/chi2_seq:np.float64(0.1279268313664943) - actor/pg_loss:np.float64(-8.747633546590805e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.040654236334376e-05) - actor/ppo_kl:np.float64(-0.0002579366225310409) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8475264199078083) - perf/mfu/actor:np.float64(0.05950522032589149) - perf/max_memory_allocated_gb:np.float64(123.93576097488403) - perf/max_memory_reserved_gb:np.float64(129.6953125) - perf/cpu_memory_used_gb:np.float64(105.37935066223145) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.25390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001752935815602541 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001752935815602541 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:102.78515625 - response_length/max:1256.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:102.78515625 - response_length_non_aborted/max:1256.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:530.25 - prompt_length/max:908.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8245151042938232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.27106592431664467) - timing_s/agent_loop/generate_sequences/max:np.float64(6.654289215803146) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9271790457423776) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.654289215803146) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:537 - timing_s/agent_loop/slowest/response_length:1256 - timing_s/gen:12.38987491466105 - timing_s/reward:0.06999921426177025 - timing_s/old_log_prob:2.330798080191016 - timing_s/adv:0.15700971148908138 - timing_s/update_actor:9.93787350319326 - timing_s/step:24.969913018867373 - timing_s/stop_profile:0.0001339409500360489 - timing_per_token_ms/update_actor:0.06132332144364797 - timing_per_token_ms/adv:0.0009688548565571459 - timing_per_token_ms/gen:0.4708651584639171 - perf/total_num_tokens:162057 - perf/time_per_step:24.969913018867373 - perf/throughput:811.2613361806118 (TaskRunner pid=2186821) Training Progress: 45%|████▌ | 45/100 [22:28<31:30, 34.38s/it] (TaskRunner pid=2186821) step:46 - global_seqlen/min:17428 - global_seqlen/max:23917 - global_seqlen/minmax_diff:6489 - global_seqlen/balanced_min:20060 - global_seqlen/balanced_max:20848 - global_seqlen/mean:20174.625 - actor/entropy:0.09392610937356949 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29012516140937805 - training/rollout_probs_diff_mean:0.0027429680339992046 - training/rollout_probs_diff_std:0.01123241800814867 - training/rollout_actor_probs_pearson_corr:0.9974663257598877 - rollout_corr/rollout_is_mean:1.0000561475753784 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6417487940052524e-05 - rollout_corr/rollout_is_max:1.643405556678772 - rollout_corr/rollout_is_min:0.48185718059539795 - rollout_corr/rollout_is_std:0.03119845874607563 - rollout_corr/rollout_is_eff_sample_size:0.9990277216053979 - rollout_corr/rollout_is_seq_mean:1.000095009803772 - rollout_corr/rollout_is_seq_std:0.004581367131322622 - rollout_corr/rollout_is_seq_max:1.0168646574020386 - rollout_corr/rollout_is_seq_min:0.9809992909431458 - rollout_corr/rollout_is_seq_max_deviation:0.019000709056854248 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0998238995671272) - rollout_corr/training_log_ppl:np.float64(0.09232246626197593) - rollout_corr/kl:np.float64(0.0015090222330158554) - rollout_corr/k3_kl:np.float64(0.001248485781694314) - rollout_corr/rollout_ppl:np.float64(1.0981056988239288) - rollout_corr/rollout_log_ppl:np.float64(0.09081344355763576) - rollout_corr/log_ppl_diff:np.float64(0.0015090227043401683) - rollout_corr/log_ppl_abs_diff:np.float64(0.004330582409238559) - rollout_corr/log_ppl_diff_max:np.float64(0.07530085742473602) - rollout_corr/log_ppl_diff_min:np.float64(-0.018227368593215942) - rollout_corr/ppl_ratio:np.float64(1.001543278573081) - rollout_corr/chi2_token:np.float64(0.002071536611765623) - rollout_corr/chi2_seq:np.float64(0.11723963636904955) - actor/pg_loss:np.float64(0.0004588030278682709) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007307079285965301) - actor/ppo_kl:np.float64(0.001100135578603556) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6038151159882545) - perf/mfu/actor:np.float64(0.05801001940211663) - perf/max_memory_allocated_gb:np.float64(123.93576097488403) - perf/max_memory_reserved_gb:np.float64(129.6953125) - perf/cpu_memory_used_gb:np.float64(105.2485237121582) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.30859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06194640323519707 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06194640323519707 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:69.23828125 - response_length/max:1476.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:69.23828125 - response_length_non_aborted/max:1476.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:561.21875 - prompt_length/max:874.0 - prompt_length/min:334.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011284090578556061 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3448892552405596) - timing_s/agent_loop/generate_sequences/max:np.float64(7.281411862000823) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7418267740067677) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.281411862000823) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:560 - timing_s/agent_loop/slowest/response_length:1476 - timing_s/gen:13.29588232561946 - timing_s/reward:0.06900179758667946 - timing_s/old_log_prob:2.376630149781704 - timing_s/adv:0.11069468967616558 - timing_s/update_actor:10.065325751900673 - timing_s/step:26.003062879666686 - timing_s/stop_profile:0.00010774284601211548 - timing_per_token_ms/update_actor:0.06236377226280955 - timing_per_token_ms/adv:0.0006858534525187308 - timing_per_token_ms/gen:0.7501203004580795 - perf/total_num_tokens:161397 - perf/time_per_step:26.003062879666686 - perf/throughput:775.8557172038267 (TaskRunner pid=2186821) Training Progress: 46%|████▌ | 46/100 [22:54<28:41, 31.88s/it] (TaskRunner pid=2186821) step:47 - global_seqlen/min:18152 - global_seqlen/max:28031 - global_seqlen/minmax_diff:9879 - global_seqlen/balanced_min:22762 - global_seqlen/balanced_max:23887 - global_seqlen/mean:22905.875 - actor/entropy:0.09966740757226944 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2412756383419037 - training/rollout_probs_diff_mean:0.0026348319370299578 - training/rollout_probs_diff_std:0.010780632495880127 - training/rollout_actor_probs_pearson_corr:0.9977898597717285 - rollout_corr/rollout_is_mean:1.0003139972686768 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.5962166293757036e-05 - rollout_corr/rollout_is_max:1.9975781440734863 - rollout_corr/rollout_is_min:0.46159979701042175 - rollout_corr/rollout_is_std:0.031940460205078125 - rollout_corr/rollout_is_eff_sample_size:0.9989814414657525 - rollout_corr/rollout_is_seq_mean:1.0001976490020752 - rollout_corr/rollout_is_seq_std:0.003636913141235709 - rollout_corr/rollout_is_seq_max:1.0171098709106445 - rollout_corr/rollout_is_seq_min:0.9841878414154053 - rollout_corr/rollout_is_seq_max_deviation:0.01710987091064453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1117577347904444) - rollout_corr/training_log_ppl:np.float64(0.10209458148892736) - rollout_corr/kl:np.float64(0.0008142971971238921) - rollout_corr/k3_kl:np.float64(0.0006282242253579373) - rollout_corr/rollout_ppl:np.float64(1.1108278771862388) - rollout_corr/rollout_log_ppl:np.float64(0.10128028324106708) - rollout_corr/log_ppl_diff:np.float64(0.0008142982478602789) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029353400677791797) - rollout_corr/log_ppl_diff_max:np.float64(0.018541395664215088) - rollout_corr/log_ppl_diff_min:np.float64(-0.010267950594425201) - rollout_corr/ppl_ratio:np.float64(1.0008228835649788) - rollout_corr/chi2_token:np.float64(0.000930295791476965) - rollout_corr/chi2_seq:np.float64(0.22962282574735582) - actor/pg_loss:np.float64(-0.0001242222497239709) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020498751109698787) - actor/ppo_kl:np.float64(0.000591708142607672) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3714657872915268) - perf/mfu/actor:np.float64(0.06615055278229345) - perf/max_memory_allocated_gb:np.float64(123.93576097488403) - perf/max_memory_reserved_gb:np.float64(129.6953125) - perf/cpu_memory_used_gb:np.float64(105.32518005371094) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.30078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001897004316560924 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001897004316560924 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:108.62109375 - response_length/max:2422.0 - response_length/min:31.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:108.62109375 - response_length_non_aborted/max:2422.0 - response_length_non_aborted/min:31.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:607.1875 - prompt_length/max:1010.0 - prompt_length/min:245.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010773725807666779 - timing_s/agent_loop/generate_sequences/min:np.float64(0.426454596221447) - timing_s/agent_loop/generate_sequences/max:np.float64(12.002666305750608) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.004392302937049) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.002666305750608) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:597 - timing_s/agent_loop/slowest/response_length:2422 - timing_s/gen:17.8655637614429 - timing_s/reward:0.063464829698205 - timing_s/old_log_prob:2.4131723940372467 - timing_s/adv:0.1255542878061533 - timing_s/update_actor:10.06117982417345 - timing_s/step:30.562387846410275 - timing_s/stop_profile:0.00011102668941020966 - timing_per_token_ms/update_actor:0.05490501794939863 - timing_per_token_ms/adv:0.0006851642199116673 - timing_per_token_ms/gen:0.6424844018212285 - perf/total_num_tokens:183247 - perf/time_per_step:30.562387846410275 - perf/throughput:749.4792329418863 (TaskRunner pid=2186821) Training Progress: 47%|████▋ | 47/100 [23:25<27:49, 31.50s/it] (TaskRunner pid=2186821) step:48 - global_seqlen/min:20343 - global_seqlen/max:27737 - global_seqlen/minmax_diff:7394 - global_seqlen/balanced_min:22200 - global_seqlen/balanced_max:22251 - global_seqlen/mean:22210.625 - actor/entropy:0.09961522370576859 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2254718542098999 - training/rollout_probs_diff_mean:0.0025464282371103764 - training/rollout_probs_diff_std:0.01014900766313076 - training/rollout_actor_probs_pearson_corr:0.9980698227882385 - rollout_corr/rollout_is_mean:1.0002561807632446 - rollout_corr/rollout_is_ratio_fraction_high:3.7184396205702797e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.7184396205702797e-05 - rollout_corr/rollout_is_max:3.3722012042999268 - rollout_corr/rollout_is_min:0.4945202171802521 - rollout_corr/rollout_is_std:0.02979516051709652 - rollout_corr/rollout_is_eff_sample_size:0.9991133928184341 - rollout_corr/rollout_is_seq_mean:1.0001027584075928 - rollout_corr/rollout_is_seq_std:0.00305532431229949 - rollout_corr/rollout_is_seq_max:1.012468934059143 - rollout_corr/rollout_is_seq_min:0.9907573461532593 - rollout_corr/rollout_is_seq_max_deviation:0.012468934059143066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0933619011193514) - rollout_corr/training_log_ppl:np.float64(0.08660966086517874) - rollout_corr/kl:np.float64(0.0004609096540573593) - rollout_corr/k3_kl:np.float64(0.0005896797931505635) - rollout_corr/rollout_ppl:np.float64(1.092862972524017) - rollout_corr/rollout_log_ppl:np.float64(0.08614875143848622) - rollout_corr/log_ppl_diff:np.float64(0.000460909426692524) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027968133090325864) - rollout_corr/log_ppl_diff_max:np.float64(0.016001291573047638) - rollout_corr/log_ppl_diff_min:np.float64(-0.0244421549141407) - rollout_corr/ppl_ratio:np.float64(1.000469961669296) - rollout_corr/chi2_token:np.float64(0.0016092471778392792) - rollout_corr/chi2_seq:np.float64(0.4790192150976509) - actor/pg_loss:np.float64(7.210543844848871e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004697930889960844) - actor/ppo_kl:np.float64(0.00019561957630287452) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.34813886508345604) - perf/mfu/actor:np.float64(0.06468839128324108) - perf/max_memory_allocated_gb:np.float64(123.93576097488403) - perf/max_memory_reserved_gb:np.float64(129.6953125) - perf/cpu_memory_used_gb:np.float64(105.24616241455078) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00257966760545969 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.00257966760545969 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:105.05078125 - response_length/max:691.0 - response_length/min:30.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:105.05078125 - response_length_non_aborted/max:691.0 - response_length_non_aborted/min:30.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:589.03125 - prompt_length/max:829.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014214962720870972 - timing_s/agent_loop/generate_sequences/min:np.float64(0.45205202884972095) - timing_s/agent_loop/generate_sequences/max:np.float64(4.285150667652488) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.102171099431871) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.285150667652488) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:498 - timing_s/agent_loop/slowest/response_length:691 - timing_s/gen:10.130033632740378 - timing_s/reward:0.07094166986644268 - timing_s/old_log_prob:2.514764489606023 - timing_s/adv:0.16063092462718487 - timing_s/update_actor:9.641014646738768 - timing_s/step:22.60114766471088 - timing_s/stop_profile:2.9416754841804504e-05 - timing_per_token_ms/update_actor:0.05425902381595952 - timing_per_token_ms/adv:0.0009040207368499585 - timing_per_token_ms/gen:0.37667919654707094 - perf/total_num_tokens:177685 - perf/time_per_step:22.60114766471088 - perf/throughput:982.7211135246624 (TaskRunner pid=2186821) Training Progress: 48%|████▊ | 48/100 [23:47<24:59, 28.83s/it] (TaskRunner pid=2186821) step:49 - global_seqlen/min:15269 - global_seqlen/max:39751 - global_seqlen/minmax_diff:24482 - global_seqlen/balanced_min:21436 - global_seqlen/balanced_max:28565 - global_seqlen/mean:23346.0 - actor/entropy:0.07806675136089325 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4250344932079315 - training/rollout_probs_diff_mean:0.0019978315103799105 - training/rollout_probs_diff_std:0.009737016633152962 - training/rollout_actor_probs_pearson_corr:0.9977913498878479 - rollout_corr/rollout_is_mean:1.000077247619629 - rollout_corr/rollout_is_ratio_fraction_high:8.841732778819278e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.631299766013399e-05 - rollout_corr/rollout_is_max:2.5243096351623535 - rollout_corr/rollout_is_min:0.29284602403640747 - rollout_corr/rollout_is_std:0.027524080127477646 - rollout_corr/rollout_is_eff_sample_size:0.9992432365081862 - rollout_corr/rollout_is_seq_mean:0.9997789263725281 - rollout_corr/rollout_is_seq_std:0.0035121606197208166 - rollout_corr/rollout_is_seq_max:1.014818787574768 - rollout_corr/rollout_is_seq_min:0.9882442951202393 - rollout_corr/rollout_is_seq_max_deviation:0.014818787574768066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.122090880293399) - rollout_corr/training_log_ppl:np.float64(0.11161522206384689) - rollout_corr/kl:np.float64(0.0010494113613734868) - rollout_corr/k3_kl:np.float64(0.0006251590251382666) - rollout_corr/rollout_ppl:np.float64(1.1208723136223853) - rollout_corr/rollout_log_ppl:np.float64(0.11056580980766739) - rollout_corr/log_ppl_diff:np.float64(0.0010494122561794939) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029947067396278726) - rollout_corr/log_ppl_diff_max:np.float64(0.020535331219434738) - rollout_corr/log_ppl_diff_min:np.float64(-0.0099649578332901) - rollout_corr/ppl_ratio:np.float64(1.0010588942095637) - rollout_corr/chi2_token:np.float64(0.000384156359359622) - rollout_corr/chi2_seq:np.float64(0.1662737203296274) - actor/pg_loss:np.float64(5.563255399465561e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.3674568840069696e-05) - actor/ppo_kl:np.float64(0.0002904789593323187) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25468049570918083) - perf/mfu/actor:np.float64(0.06431388737840894) - perf/max_memory_allocated_gb:np.float64(123.93576097488403) - perf/max_memory_reserved_gb:np.float64(129.6953125) - perf/cpu_memory_used_gb:np.float64(105.16648864746094) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.2109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001210212241858244 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:-0.001210212241858244 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:176.71875 - response_length/max:8192.0 - response_length/min:28.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:176.71875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:28.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:552.84375 - prompt_length/max:911.0 - prompt_length/min:301.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8314068913459778e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3820417635142803) - timing_s/agent_loop/generate_sequences/max:np.float64(46.87782824598253) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.4162356741871918) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(46.87782824598253) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:551 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.775812443345785 - timing_s/reward:0.07412644848227501 - timing_s/old_log_prob:2.590009979903698 - timing_s/adv:0.14392185769975185 - timing_s/update_actor:10.792937148362398 - timing_s/step:66.46433951705694 - timing_s/stop_profile:5.177408456802368e-05 - timing_per_token_ms/update_actor:0.0577879355583526 - timing_per_token_ms/adv:0.000770591630791955 - timing_per_token_ms/gen:1.1665741035222321 - perf/total_num_tokens:186768 - perf/time_per_step:66.46433951705694 - perf/throughput:351.25602946838353 (TaskRunner pid=2186821) Training Progress: 49%|████▉ | 49/100 [24:54<34:06, 40.13s/it] (TaskRunner pid=2186821) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2186821) validation generation end (TaskRunner pid=2186821) [prompt] user (TaskRunner pid=2186821) Your task is to answer the user's question using available tools. (TaskRunner pid=2186821) You have access to the following tools: (TaskRunner pid=2186821) Name: Axolotl (TaskRunner pid=2186821) Description: Collection of axolotl pictures and facts (TaskRunner pid=2186821) Documentation: (TaskRunner pid=2186821) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2186821) Parameters: {} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{url, source, description} (TaskRunner pid=2186821) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2186821) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2186821) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2186821) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Array[Object{fact, source}] (TaskRunner pid=2186821) (TaskRunner pid=2186821) Use the following format: (TaskRunner pid=2186821) Thought: you should always think about what to do (TaskRunner pid=2186821) Action: the action to take, should be one of the tool names. (TaskRunner pid=2186821) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2186821) (TaskRunner pid=2186821) Begin! (TaskRunner pid=2186821) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2186821) assistant (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: The user is asking for a random picture of an axolotl. I should use the getRandomAxolotlImage tool. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) Output: {"url": "https://example.com/axolotl.jpg", "source": "Example Source", "description": "A random axolotl image"} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2187230) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2186821) step:50 - global_seqlen/min:16756 - global_seqlen/max:36411 - global_seqlen/minmax_diff:19655 - global_seqlen/balanced_min:22543 - global_seqlen/balanced_max:29635 - global_seqlen/mean:24549.625 - actor/entropy:0.08976611495018005 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3012014329433441 - training/rollout_probs_diff_mean:0.0018682791851460934 - training/rollout_probs_diff_std:0.008796855807304382 - training/rollout_actor_probs_pearson_corr:0.9983659386634827 - rollout_corr/rollout_is_mean:0.9999872446060181 - rollout_corr/rollout_is_ratio_fraction_high:3.908311191480607e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.3027758598327637 - rollout_corr/rollout_is_min:0.5101031064987183 - rollout_corr/rollout_is_std:0.02582818642258644 - rollout_corr/rollout_is_eff_sample_size:0.9993332304846022 - rollout_corr/rollout_is_seq_mean:0.9999366402626038 - rollout_corr/rollout_is_seq_std:0.00335451471619308 - rollout_corr/rollout_is_seq_max:1.0112617015838623 - rollout_corr/rollout_is_seq_min:0.9882235527038574 - rollout_corr/rollout_is_seq_max_deviation:0.011776447296142578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.124918662942946) - rollout_corr/training_log_ppl:np.float64(0.11467544128390728) - rollout_corr/kl:np.float64(0.0017736391622875658) - rollout_corr/k3_kl:np.float64(0.0015390691617088725) - rollout_corr/rollout_ppl:np.float64(1.1229061437770724) - rollout_corr/rollout_log_ppl:np.float64(0.11290180086689361) - rollout_corr/log_ppl_diff:np.float64(0.0017736404170136666) - rollout_corr/log_ppl_abs_diff:np.float64(0.0039835598199715605) - rollout_corr/log_ppl_diff_max:np.float64(0.1006503701210022) - rollout_corr/log_ppl_diff_min:np.float64(-0.015296325087547302) - rollout_corr/ppl_ratio:np.float64(1.0018262404482812) - rollout_corr/chi2_token:np.float64(0.00164889101870358) - rollout_corr/chi2_seq:np.float64(0.5349687321577221) - actor/pg_loss:np.float64(0.0006220416398718953) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005291439956636168) - actor/ppo_kl:np.float64(0.0012731698314567552) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4314403794705868) - perf/mfu/actor:np.float64(0.06670971231331849) - perf/max_memory_allocated_gb:np.float64(124.21099805831909) - perf/max_memory_reserved_gb:np.float64(129.96875) - perf/cpu_memory_used_gb:np.float64(105.13151931762695) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5946691176470589) - val-aux/tooluse/reward/std@16:np.float64(0.05462138878725001) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6183676470588236) - val-aux/tooluse/reward/best@2/std:np.float64(0.03430129848288785) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5704411764705882) - val-aux/tooluse/reward/worst@2/std:np.float64(0.058941969646180176) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5942794117647059) - val-aux/tooluse/reward/maj@2/std:np.float64(0.054794894189696144) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6291911764705882) - val-aux/tooluse/reward/best@4/std:np.float64(0.012049278888024803) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5448088235294117) - val-aux/tooluse/reward/worst@4/std:np.float64(0.05302535093522716) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.6025882352941176) - val-aux/tooluse/reward/maj@4/std:np.float64(0.04548161354817276) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6319558823529412) - val-aux/tooluse/reward/best@8/std:np.float64(0.002952964202460843) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5223235294117647) - val-aux/tooluse/reward/worst@8/std:np.float64(0.037123127795031824) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.6091617647058823) - val-aux/tooluse/reward/maj@8/std:np.float64(0.0330116181608613) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6323529411764706) - val-aux/tooluse/reward/best@16/std:np.float64(0.0) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5086911764705881) - val-aux/tooluse/reward/worst@16/std:np.float64(0.020941080549456682) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.6127794117647059) - val-aux/tooluse/reward/maj@16/std:np.float64(0.022916265813138516) - val-aux/tooluse/score/mean@16:np.float64(0.5946691176470589) - val-aux/tooluse/score/std@16:np.float64(0.05462138878725001) - val-aux/tooluse/score/best@2/mean:np.float64(0.6183676470588236) - val-aux/tooluse/score/best@2/std:np.float64(0.03430129848288785) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5704411764705882) - val-aux/tooluse/score/worst@2/std:np.float64(0.058941969646180176) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5942794117647059) - val-aux/tooluse/score/maj@2/std:np.float64(0.054794894189696144) - val-aux/tooluse/score/best@4/mean:np.float64(0.6291911764705882) - val-aux/tooluse/score/best@4/std:np.float64(0.012049278888024803) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5448088235294117) - val-aux/tooluse/score/worst@4/std:np.float64(0.05302535093522716) - val-aux/tooluse/score/maj@4/mean:np.float64(0.6025882352941176) - val-aux/tooluse/score/maj@4/std:np.float64(0.04548161354817276) - val-aux/tooluse/score/best@8/mean:np.float64(0.6319558823529412) - val-aux/tooluse/score/best@8/std:np.float64(0.002952964202460843) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5223235294117647) - val-aux/tooluse/score/worst@8/std:np.float64(0.037123127795031824) - val-aux/tooluse/score/maj@8/mean:np.float64(0.6091617647058823) - val-aux/tooluse/score/maj@8/std:np.float64(0.0330116181608613) - val-aux/tooluse/score/best@16/mean:np.float64(0.6323529411764706) - val-aux/tooluse/score/best@16/std:np.float64(0.0) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5086911764705881) - val-aux/tooluse/score/worst@16/std:np.float64(0.020941080549456682) - val-aux/tooluse/score/maj@16/mean:np.float64(0.6127794117647059) - val-aux/tooluse/score/maj@16/std:np.float64(0.022916265813138516) - val-core/tooluse/acc/mean@16:np.float64(0.5946691176470589) - val-aux/tooluse/acc/std@16:np.float64(0.05462138878725001) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6183676470588236) - val-aux/tooluse/acc/best@2/std:np.float64(0.03430129848288785) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5704411764705882) - val-aux/tooluse/acc/worst@2/std:np.float64(0.058941969646180176) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5942794117647059) - val-aux/tooluse/acc/maj@2/std:np.float64(0.054794894189696144) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6291911764705882) - val-aux/tooluse/acc/best@4/std:np.float64(0.012049278888024803) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5448088235294117) - val-aux/tooluse/acc/worst@4/std:np.float64(0.05302535093522716) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.6025882352941176) - val-aux/tooluse/acc/maj@4/std:np.float64(0.04548161354817276) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6319558823529412) - val-aux/tooluse/acc/best@8/std:np.float64(0.002952964202460843) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5223235294117647) - val-aux/tooluse/acc/worst@8/std:np.float64(0.037123127795031824) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.6091617647058823) - val-aux/tooluse/acc/maj@8/std:np.float64(0.0330116181608613) - val-core/tooluse/acc/best@16/mean:np.float64(0.6323529411764706) - val-core/tooluse/acc/best@16/std:np.float64(0.0) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5086911764705881) - val-aux/tooluse/acc/worst@16/std:np.float64(0.020941080549456682) - val-core/tooluse/acc/maj@16/mean:np.float64(0.6127794117647059) - val-core/tooluse/acc/maj@16/std:np.float64(0.022916265813138516) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009196744300425053 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009196744300425053 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:199.89453125 - response_length/max:8192.0 - response_length/min:31.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:199.89453125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:31.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:567.28125 - prompt_length/max:979.0 - prompt_length/min:337.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.8900023102760315e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.44583102129399776) - timing_s/agent_loop/generate_sequences/max:np.float64(47.51443203166127) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.536721398064401) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(47.51443203166127) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:451 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:53.57204416953027 - timing_s/reward:0.07473987527191639 - timing_s/old_log_prob:2.581733364611864 - timing_s/adv:0.13276498205959797 - timing_s/update_actor:10.821437932550907 - timing_s/step:67.26940510421991 - timing_s/testing:65.86754743941128 - timing_s/save_checkpoint:6.677435263991356 - timing_s/stop_profile:0.00011324696242809296 - timing_per_token_ms/update_actor:0.055099812790169435 - timing_per_token_ms/adv:0.0006760031062572135 - timing_per_token_ms/gen:1.0468810538668882 - perf/total_num_tokens:196397 - perf/time_per_step:67.26940510421991 - perf/throughput:364.94488039496525 (TaskRunner pid=2186821) Training Progress: 50%|█████ | 50/100 [27:14<58:22, 70.06s/it] (TaskRunner pid=2186821) step:51 - global_seqlen/min:14767 - global_seqlen/max:29124 - global_seqlen/minmax_diff:14357 - global_seqlen/balanced_min:21630 - global_seqlen/balanced_max:22074 - global_seqlen/mean:21693.875 - actor/entropy:0.12435629218816757 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3175737261772156 - training/rollout_probs_diff_mean:0.003009780077263713 - training/rollout_probs_diff_std:0.011481983587145805 - training/rollout_actor_probs_pearson_corr:0.997955858707428 - rollout_corr/rollout_is_mean:1.000101089477539 - rollout_corr/rollout_is_ratio_fraction_high:3.387648757779971e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.775297515559942e-05 - rollout_corr/rollout_is_max:2.740844488143921 - rollout_corr/rollout_is_min:0.373458594083786 - rollout_corr/rollout_is_std:0.031623516231775284 - rollout_corr/rollout_is_eff_sample_size:0.999001071335511 - rollout_corr/rollout_is_seq_mean:1.00031316280365 - rollout_corr/rollout_is_seq_std:0.004349833354353905 - rollout_corr/rollout_is_seq_max:1.0305124521255493 - rollout_corr/rollout_is_seq_min:0.983889102935791 - rollout_corr/rollout_is_seq_max_deviation:0.030512452125549316 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.131070591043681) - rollout_corr/training_log_ppl:np.float64(0.11959081599707133) - rollout_corr/kl:np.float64(0.001032684409398854) - rollout_corr/k3_kl:np.float64(0.0016965258523402582) - rollout_corr/rollout_ppl:np.float64(1.129901341162622) - rollout_corr/rollout_log_ppl:np.float64(0.11855813040529029) - rollout_corr/log_ppl_diff:np.float64(0.0010326855917810462) - rollout_corr/log_ppl_abs_diff:np.float64(0.004712027519417461) - rollout_corr/log_ppl_diff_max:np.float64(0.09949277341365814) - rollout_corr/log_ppl_diff_min:np.float64(-0.029285922646522522) - rollout_corr/ppl_ratio:np.float64(1.0010969194117934) - rollout_corr/chi2_token:np.float64(0.0035823392681777477) - rollout_corr/chi2_seq:np.float64(0.29991551698185503) - actor/pg_loss:np.float64(-0.0002541770227253437) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014090517897784594) - actor/ppo_kl:np.float64(0.0007771755741927677) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6457180678844452) - perf/mfu/actor:np.float64(0.06291742564140312) - perf/max_memory_allocated_gb:np.float64(124.21099805831909) - perf/max_memory_reserved_gb:np.float64(129.96875) - perf/cpu_memory_used_gb:np.float64(105.57783508300781) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018746400251984596 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018746400251984596 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:115.30859375 - response_length/max:1215.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:115.30859375 - response_length_non_aborted/max:1215.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:562.625 - prompt_length/max:897.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.553290247917175e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.39970578998327255) - timing_s/agent_loop/generate_sequences/max:np.float64(6.475353104993701) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1480888391961344) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.475353104993701) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:760 - timing_s/agent_loop/slowest/response_length:1215 - timing_s/gen:12.405727479606867 - timing_s/reward:0.06768927536904812 - timing_s/old_log_prob:2.509016489610076 - timing_s/adv:0.1504184529185295 - timing_s/update_actor:9.920974630862474 - timing_s/step:25.149370880797505 - timing_s/stop_profile:2.7706846594810486e-05 - timing_per_token_ms/update_actor:0.05716460654713874 - timing_per_token_ms/adv:0.0008667103786122207 - timing_per_token_ms/gen:0.42026245738700047 - perf/total_num_tokens:173551 - perf/time_per_step:25.149370880797505 - perf/throughput:862.601100553338 (TaskRunner pid=2186821) Training Progress: 51%|█████ | 51/100 [27:39<46:12, 56.59s/it] (TaskRunner pid=2186821) step:52 - global_seqlen/min:19290 - global_seqlen/max:27664 - global_seqlen/minmax_diff:8374 - global_seqlen/balanced_min:23346 - global_seqlen/balanced_max:23356 - global_seqlen/mean:23352.25 - actor/entropy:0.12085407227277756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25337326526641846 - training/rollout_probs_diff_mean:0.002615503966808319 - training/rollout_probs_diff_std:0.010014807805418968 - training/rollout_actor_probs_pearson_corr:0.9984092712402344 - rollout_corr/rollout_is_mean:0.9996906518936157 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8456315994262695 - rollout_corr/rollout_is_min:0.6130654215812683 - rollout_corr/rollout_is_std:0.02864883840084076 - rollout_corr/rollout_is_eff_sample_size:0.9991793220604662 - rollout_corr/rollout_is_seq_mean:0.9995668530464172 - rollout_corr/rollout_is_seq_std:0.003674387000501156 - rollout_corr/rollout_is_seq_max:1.0129340887069702 - rollout_corr/rollout_is_seq_min:0.9801408052444458 - rollout_corr/rollout_is_seq_max_deviation:0.0198591947555542 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1478937310166657) - rollout_corr/training_log_ppl:np.float64(0.12319744565320434) - rollout_corr/kl:np.float64(0.0015379079779478388) - rollout_corr/k3_kl:np.float64(0.001347341913481362) - rollout_corr/rollout_ppl:np.float64(1.1460376735776663) - rollout_corr/rollout_log_ppl:np.float64(0.12165953587100375) - rollout_corr/log_ppl_diff:np.float64(0.001537909782200586) - rollout_corr/log_ppl_abs_diff:np.float64(0.0040589987256680615) - rollout_corr/log_ppl_diff_max:np.float64(0.06000564992427826) - rollout_corr/log_ppl_diff_min:np.float64(-0.021004170179367065) - rollout_corr/ppl_ratio:np.float64(1.0015658575575799) - rollout_corr/chi2_token:np.float64(0.0014379981439560652) - rollout_corr/chi2_seq:np.float64(0.033965713577345014) - actor/pg_loss:np.float64(6.602378562092781e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001250935423286137) - actor/ppo_kl:np.float64(0.0007017719746542639) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4841025471687317) - perf/mfu/actor:np.float64(0.06692017148756052) - perf/max_memory_allocated_gb:np.float64(124.21099805831909) - perf/max_memory_reserved_gb:np.float64(129.96875) - perf/cpu_memory_used_gb:np.float64(105.40730667114258) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.07546635717153549 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.07546635717153549 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:103.4453125 - response_length/max:1022.0 - response_length/min:31.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:103.4453125 - response_length_non_aborted/max:1022.0 - response_length_non_aborted/min:31.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:626.3125 - prompt_length/max:1165.0 - prompt_length/min:218.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.55478036403656e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4993522707372904) - timing_s/agent_loop/generate_sequences/max:np.float64(5.592471895739436) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1045378572234767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.592471895739436) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:592 - timing_s/agent_loop/slowest/response_length:1019 - timing_s/gen:11.401132667437196 - timing_s/reward:0.07064570114016533 - timing_s/old_log_prob:2.4060371667146683 - timing_s/adv:0.10630786046385765 - timing_s/update_actor:10.102623455226421 - timing_s/step:24.1730930916965 - timing_s/stop_profile:0.00013321824371814728 - timing_per_token_ms/update_actor:0.05407735579669208 - timing_per_token_ms/adv:0.0005690450623808072 - timing_per_token_ms/gen:0.430523852708904 - perf/total_num_tokens:186818 - perf/time_per_step:24.1730930916965 - perf/throughput:966.0431088159561 (TaskRunner pid=2186821) Training Progress: 52%|█████▏ | 52/100 [28:03<37:30, 46.88s/it] (TaskRunner pid=2186821) step:53 - global_seqlen/min:19341 - global_seqlen/max:29758 - global_seqlen/minmax_diff:10417 - global_seqlen/balanced_min:23533 - global_seqlen/balanced_max:23537 - global_seqlen/mean:23536.0 - actor/entropy:0.1103062704205513 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1908014416694641 - training/rollout_probs_diff_mean:0.0027612014673650265 - training/rollout_probs_diff_std:0.010019850917160511 - training/rollout_actor_probs_pearson_corr:0.9982837438583374 - rollout_corr/rollout_is_mean:1.000266671180725 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6279518604278564 - rollout_corr/rollout_is_min:0.6303539872169495 - rollout_corr/rollout_is_std:0.02898392453789711 - rollout_corr/rollout_is_eff_sample_size:0.9991611132954431 - rollout_corr/rollout_is_seq_mean:1.0000847578048706 - rollout_corr/rollout_is_seq_std:0.004257576074451208 - rollout_corr/rollout_is_seq_max:1.0180537700653076 - rollout_corr/rollout_is_seq_min:0.9865372180938721 - rollout_corr/rollout_is_seq_max_deviation:0.018053770065307617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1220540376380086) - rollout_corr/training_log_ppl:np.float64(0.11223160912049934) - rollout_corr/kl:np.float64(0.0013152080609515338) - rollout_corr/k3_kl:np.float64(0.0017837993639648886) - rollout_corr/rollout_ppl:np.float64(1.1206734855659306) - rollout_corr/rollout_log_ppl:np.float64(0.11091639963706257) - rollout_corr/log_ppl_diff:np.float64(0.0013152094834367745) - rollout_corr/log_ppl_abs_diff:np.float64(0.00488566211424768) - rollout_corr/log_ppl_diff_max:np.float64(0.04811738058924675) - rollout_corr/log_ppl_diff_min:np.float64(-0.03103354573249817) - rollout_corr/ppl_ratio:np.float64(1.0013458174653351) - rollout_corr/chi2_token:np.float64(0.004454549867659807) - rollout_corr/chi2_seq:np.float64(0.12215380812995136) - actor/pg_loss:np.float64(5.1266979426145554e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008156286603480112) - actor/ppo_kl:np.float64(0.0009775941209895223) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.44834135100245476) - perf/mfu/actor:np.float64(0.06891288697657735) - perf/max_memory_allocated_gb:np.float64(124.21099805831909) - perf/max_memory_reserved_gb:np.float64(129.96875) - perf/cpu_memory_used_gb:np.float64(105.36750411987305) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.36328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.36328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009108836762607098 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.009108836762607098 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:70.8125 - response_length/max:416.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:70.8125 - response_length_non_aborted/max:416.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:664.6875 - prompt_length/max:1060.0 - prompt_length/min:344.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010358728468418121 - timing_s/agent_loop/generate_sequences/min:np.float64(0.393720630556345) - timing_s/agent_loop/generate_sequences/max:np.float64(2.6122940834611654) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8346773711455171) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.6122940834611654) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:757 - timing_s/agent_loop/slowest/response_length:416 - timing_s/gen:8.846651358529925 - timing_s/reward:0.0705890879034996 - timing_s/old_log_prob:2.3532503712922335 - timing_s/adv:0.1298077031970024 - timing_s/update_actor:10.047533832490444 - timing_s/step:21.53699954226613 - timing_s/stop_profile:0.0001185983419418335 - timing_per_token_ms/update_actor:0.05336258196215608 - timing_per_token_ms/adv:0.0006894103883253442 - timing_per_token_ms/gen:0.48801033531166843 - perf/total_num_tokens:188288 - perf/time_per_step:21.53699954226613 - perf/throughput:1092.8170358090435 (TaskRunner pid=2186821) Training Progress: 53%|█████▎ | 53/100 [28:25<30:46, 39.29s/it] (TaskRunner pid=2186821) step:54 - global_seqlen/min:17680 - global_seqlen/max:29563 - global_seqlen/minmax_diff:11883 - global_seqlen/balanced_min:22511 - global_seqlen/balanced_max:24781 - global_seqlen/mean:22892.5 - actor/entropy:0.1276119500398636 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16880491375923157 - training/rollout_probs_diff_mean:0.002570601413026452 - training/rollout_probs_diff_std:0.009399130940437317 - training/rollout_actor_probs_pearson_corr:0.9986492991447449 - rollout_corr/rollout_is_mean:0.9997713565826416 - rollout_corr/rollout_is_ratio_fraction_high:4.492362859309651e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0160369873046875 - rollout_corr/rollout_is_min:0.6142134070396423 - rollout_corr/rollout_is_std:0.027496999129652977 - rollout_corr/rollout_is_eff_sample_size:0.9992443077697168 - rollout_corr/rollout_is_seq_mean:0.9996440410614014 - rollout_corr/rollout_is_seq_std:0.004165541380643845 - rollout_corr/rollout_is_seq_max:1.0110089778900146 - rollout_corr/rollout_is_seq_min:0.9851983189582825 - rollout_corr/rollout_is_seq_max_deviation:0.01480168104171753 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1480430448427796) - rollout_corr/training_log_ppl:np.float64(0.13302937911794288) - rollout_corr/kl:np.float64(0.0006584005663512471) - rollout_corr/k3_kl:np.float64(0.000867622551947278) - rollout_corr/rollout_ppl:np.float64(1.1472381530329585) - rollout_corr/rollout_log_ppl:np.float64(0.1323709783682716) - rollout_corr/log_ppl_diff:np.float64(0.0006584007496712729) - rollout_corr/log_ppl_abs_diff:np.float64(0.004271910132956691) - rollout_corr/log_ppl_diff_max:np.float64(0.02569088339805603) - rollout_corr/log_ppl_diff_min:np.float64(-0.02233685553073883) - rollout_corr/ppl_ratio:np.float64(1.0006750079337507) - rollout_corr/chi2_token:np.float64(0.002260186243802309) - rollout_corr/chi2_seq:np.float64(0.07397437910549343) - actor/pg_loss:np.float64(0.00020802486687898636) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000179950991878286) - actor/ppo_kl:np.float64(-0.00013384538609706453) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.38605260476469994) - perf/mfu/actor:np.float64(0.06479785026833042) - perf/max_memory_allocated_gb:np.float64(124.21099805831909) - perf/max_memory_reserved_gb:np.float64(129.96875) - perf/cpu_memory_used_gb:np.float64(105.33172225952148) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.3359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005261680111289024 - critic/advantages/max:0.875 - critic/advantages/min:-0.25 - critic/returns/mean:-0.005261680111289024 - critic/returns/max:0.875 - critic/returns/min:-0.25 - response_length/mean:86.953125 - response_length/max:3212.0 - response_length/min:21.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.953125 - response_length_non_aborted/max:3212.0 - response_length_non_aborted/min:21.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:628.4375 - prompt_length/max:1114.0 - prompt_length/min:309.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010302290320396423 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3814200274646282) - timing_s/agent_loop/generate_sequences/max:np.float64(15.636041261255741) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8792525449098321) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.636041261255741) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:939 - timing_s/agent_loop/slowest/response_length:3212 - timing_s/gen:21.579095566645265 - timing_s/reward:0.07174609415233135 - timing_s/old_log_prob:2.4244725070893764 - timing_s/adv:0.1419228781014681 - timing_s/update_actor:10.297520756721497 - timing_s/step:34.60039337351918 - timing_s/stop_profile:0.00012378767132759094 - timing_per_token_ms/update_actor:0.05622758958568033 - timing_per_token_ms/adv:0.000774942001209283 - timing_per_token_ms/gen:0.9694113012868493 - perf/total_num_tokens:183140 - perf/time_per_step:34.60039337351918 - perf/throughput:661.6254258404005 (TaskRunner pid=2186821) Training Progress: 54%|█████▍ | 54/100 [28:59<29:03, 37.90s/it] (TaskRunner pid=2186821) step:55 - global_seqlen/min:16713 - global_seqlen/max:45215 - global_seqlen/minmax_diff:28502 - global_seqlen/balanced_min:21738 - global_seqlen/balanced_max:29108 - global_seqlen/mean:23703.875 - actor/entropy:0.07887188345193863 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4859731197357178 - training/rollout_probs_diff_mean:0.0015228097327053547 - training/rollout_probs_diff_std:0.008203672245144844 - training/rollout_actor_probs_pearson_corr:0.9984710216522217 - rollout_corr/rollout_is_mean:1.0000531673431396 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6295211834367365e-05 - rollout_corr/rollout_is_max:1.7102925777435303 - rollout_corr/rollout_is_min:0.483701229095459 - rollout_corr/rollout_is_std:0.021106518805027008 - rollout_corr/rollout_is_eff_sample_size:0.9995549514599201 - rollout_corr/rollout_is_seq_mean:1.0000070333480835 - rollout_corr/rollout_is_seq_std:0.0038948149885982275 - rollout_corr/rollout_is_seq_max:1.0148099660873413 - rollout_corr/rollout_is_seq_min:0.9825122952461243 - rollout_corr/rollout_is_seq_max_deviation:0.017487704753875732 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1692785769701004) - rollout_corr/training_log_ppl:np.float64(0.14855145355159038) - rollout_corr/kl:np.float64(0.0006678900791925457) - rollout_corr/k3_kl:np.float64(0.0006207468698828933) - rollout_corr/rollout_ppl:np.float64(1.1684450306929648) - rollout_corr/rollout_log_ppl:np.float64(0.14788356213284715) - rollout_corr/log_ppl_diff:np.float64(0.0006678914187432383) - rollout_corr/log_ppl_abs_diff:np.float64(0.0033845430089058937) - rollout_corr/log_ppl_diff_max:np.float64(0.03670760989189148) - rollout_corr/log_ppl_diff_min:np.float64(-0.017420127987861633) - rollout_corr/ppl_ratio:np.float64(1.0006813164800406) - rollout_corr/chi2_token:np.float64(0.001132474048063159) - rollout_corr/chi2_seq:np.float64(0.06912964303046465) - actor/pg_loss:np.float64(-7.906113751232624e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041015441820491105) - actor/ppo_kl:np.float64(0.00029518094004288287) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5896490663290024) - perf/mfu/actor:np.float64(0.06501912698074269) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.26312637329102) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005960255395621061 - critic/advantages/max:0.75 - critic/advantages/min:-0.75 - critic/returns/mean:-0.005960255395621061 - critic/returns/max:0.75 - critic/returns/min:-0.75 - response_length/mean:138.77734375 - response_length/max:8192.0 - response_length/min:27.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:138.77734375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:601.96875 - prompt_length/max:1168.0 - prompt_length/min:353.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010286830365657806 - timing_s/agent_loop/generate_sequences/min:np.float64(0.355673611164093) - timing_s/agent_loop/generate_sequences/max:np.float64(44.80524764209986) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1217403762275353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(44.80524764209986) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:943 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:50.57065521925688 - timing_s/reward:0.07363844849169254 - timing_s/old_log_prob:2.7997681498527527 - timing_s/adv:0.12123258598148823 - timing_s/update_actor:10.887110909447074 - timing_s/step:64.53863121382892 - timing_s/stop_profile:0.00011526048183441162 - timing_per_token_ms/update_actor:0.0574120840445237 - timing_per_token_ms/adv:0.0006393078451386547 - timing_per_token_ms/gen:1.4234428806050856 - perf/total_num_tokens:189631 - perf/time_per_step:64.53863121382892 - perf/throughput:367.2819604968766 (TaskRunner pid=2186821) Training Progress: 55%|█████▌ | 55/100 [30:04<34:25, 45.90s/it] (TaskRunner pid=2186821) step:56 - global_seqlen/min:17882 - global_seqlen/max:26146 - global_seqlen/minmax_diff:8264 - global_seqlen/balanced_min:20935 - global_seqlen/balanced_max:20940 - global_seqlen/mean:20938.75 - actor/entropy:0.15791335701942444 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15484029054641724 - training/rollout_probs_diff_mean:0.003054075175896287 - training/rollout_probs_diff_std:0.009589949622750282 - training/rollout_actor_probs_pearson_corr:0.9987651705741882 - rollout_corr/rollout_is_mean:0.9999533891677856 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.56943884678185e-05 - rollout_corr/rollout_is_max:1.6175631284713745 - rollout_corr/rollout_is_min:0.4758986532688141 - rollout_corr/rollout_is_std:0.02825288288295269 - rollout_corr/rollout_is_eff_sample_size:0.999202411252767 - rollout_corr/rollout_is_seq_mean:1.0000014305114746 - rollout_corr/rollout_is_seq_std:0.0038902456872165203 - rollout_corr/rollout_is_seq_max:1.0237252712249756 - rollout_corr/rollout_is_seq_min:0.9889193177223206 - rollout_corr/rollout_is_seq_max_deviation:0.023725271224975586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.159901893697679) - rollout_corr/training_log_ppl:np.float64(0.14259462099289522) - rollout_corr/kl:np.float64(0.0006148523082032398) - rollout_corr/k3_kl:np.float64(0.0006523710138228722) - rollout_corr/rollout_ppl:np.float64(1.1592776998877525) - rollout_corr/rollout_log_ppl:np.float64(0.14197976690047653) - rollout_corr/log_ppl_diff:np.float64(0.0006148540924186818) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037170934592722915) - rollout_corr/log_ppl_diff_max:np.float64(0.014851272106170654) - rollout_corr/log_ppl_diff_min:np.float64(-0.03255283832550049) - rollout_corr/ppl_ratio:np.float64(1.0006288131698966) - rollout_corr/chi2_token:np.float64(0.0015645867679268122) - rollout_corr/chi2_seq:np.float64(0.08710163505747914) - actor/pg_loss:np.float64(-0.00026795431040227413) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023003532442089636) - actor/ppo_kl:np.float64(0.0002243879264014481) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3356231153011322) - perf/mfu/actor:np.float64(0.06142335826158035) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.24317932128906) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.36328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.36328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004697148688137531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004697148688137531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:59.4609375 - response_length/max:204.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:59.4609375 - response_length_non_aborted/max:204.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:594.875 - prompt_length/max:1258.0 - prompt_length/min:345.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.982047438621521e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.35091947205364704) - timing_s/agent_loop/generate_sequences/max:np.float64(1.5059345662593842) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7302920202855603) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.5059345662593842) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:525 - timing_s/agent_loop/slowest/response_length:204 - timing_s/gen:7.449523292481899 - timing_s/reward:0.06859931536018848 - timing_s/old_log_prob:2.372446844354272 - timing_s/adv:0.13953724689781666 - timing_s/update_actor:9.892535397782922 - timing_s/step:20.00921219587326 - timing_s/stop_profile:6.455369293689728e-05 - timing_per_token_ms/update_actor:0.05905638706813278 - timing_per_token_ms/adv:0.0008330084585864525 - timing_per_token_ms/gen:0.4893918862489751 - perf/total_num_tokens:167510 - perf/time_per_step:20.00921219587326 - perf/throughput:1046.4554923515905 (TaskRunner pid=2186821) Training Progress: 56%|█████▌ | 56/100 [30:24<27:58, 38.15s/it] (TaskRunner pid=2186821) step:57 - global_seqlen/min:16082 - global_seqlen/max:24795 - global_seqlen/minmax_diff:8713 - global_seqlen/balanced_min:21223 - global_seqlen/balanced_max:21232 - global_seqlen/mean:21230.125 - actor/entropy:0.16003233194351196 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.20162266492843628 - training/rollout_probs_diff_mean:0.002576044760644436 - training/rollout_probs_diff_std:0.00805547833442688 - training/rollout_actor_probs_pearson_corr:0.9991845488548279 - rollout_corr/rollout_is_mean:1.0000295639038086 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.571394443511963 - rollout_corr/rollout_is_min:0.646244466304779 - rollout_corr/rollout_is_std:0.02526354230940342 - rollout_corr/rollout_is_eff_sample_size:0.9993622796198363 - rollout_corr/rollout_is_seq_mean:0.9999566674232483 - rollout_corr/rollout_is_seq_std:0.0036897705867886543 - rollout_corr/rollout_is_seq_max:1.013642430305481 - rollout_corr/rollout_is_seq_min:0.9889538884162903 - rollout_corr/rollout_is_seq_max_deviation:0.013642430305480957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.179719006177038) - rollout_corr/training_log_ppl:np.float64(0.15739093450974906) - rollout_corr/kl:np.float64(0.0006776170480460308) - rollout_corr/k3_kl:np.float64(0.00047190926551243795) - rollout_corr/rollout_ppl:np.float64(1.1788540529087186) - rollout_corr/rollout_log_ppl:np.float64(0.15671331644625752) - rollout_corr/log_ppl_diff:np.float64(0.0006776180634915363) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031513193862338085) - rollout_corr/log_ppl_diff_max:np.float64(0.019533269107341766) - rollout_corr/log_ppl_diff_min:np.float64(-0.01278078556060791) - rollout_corr/ppl_ratio:np.float64(1.0006878417916596) - rollout_corr/chi2_token:np.float64(0.000515652121976018) - rollout_corr/chi2_seq:np.float64(0.05156450951471925) - actor/pg_loss:np.float64(-1.4181132428348064e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014391447621164843) - actor/ppo_kl:np.float64(0.00033023956053312986) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26680830866098404) - perf/mfu/actor:np.float64(0.062066599319523005) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.42323303222656) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004888130817562342 - critic/advantages/max:0.875 - critic/advantages/min:-0.375 - critic/returns/mean:0.004888130817562342 - critic/returns/max:0.875 - critic/returns/min:-0.375 - response_length/mean:59.53515625 - response_length/max:270.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:59.53515625 - response_length_non_aborted/max:270.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:603.90625 - prompt_length/max:1026.0 - prompt_length/min:318.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016187690198421478 - timing_s/agent_loop/generate_sequences/min:np.float64(0.39738952182233334) - timing_s/agent_loop/generate_sequences/max:np.float64(1.81196073256433) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7398540380090708) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.81196073256433) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1026 - timing_s/agent_loop/slowest/response_length:270 - timing_s/gen:7.804680312052369 - timing_s/reward:0.06766689196228981 - timing_s/old_log_prob:2.3378173131495714 - timing_s/adv:0.13024831004440784 - timing_s/update_actor:9.890585040673614 - timing_s/step:20.31976050697267 - timing_s/stop_profile:2.7019530534744263e-05 - timing_per_token_ms/update_actor:0.058234378275408255 - timing_per_token_ms/adv:0.0007668837915721636 - timing_per_token_ms/gen:0.5120845293650265 - perf/total_num_tokens:169841 - perf/time_per_step:20.31976050697267 - perf/throughput:1044.8019302547852 (TaskRunner pid=2186821) Training Progress: 57%|█████▋ | 57/100 [30:44<23:30, 32.81s/it] (TaskRunner pid=2186821) step:58 - global_seqlen/min:20317 - global_seqlen/max:27720 - global_seqlen/minmax_diff:7403 - global_seqlen/balanced_min:23493 - global_seqlen/balanced_max:23496 - global_seqlen/mean:23495.0 - actor/entropy:0.14813433587551117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.16887801885604858 - training/rollout_probs_diff_mean:0.002329382346943021 - training/rollout_probs_diff_std:0.00764509616419673 - training/rollout_actor_probs_pearson_corr:0.9992225766181946 - rollout_corr/rollout_is_mean:1.000121831893921 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.402772068977356 - rollout_corr/rollout_is_min:0.5061036348342896 - rollout_corr/rollout_is_std:0.023870911449193954 - rollout_corr/rollout_is_eff_sample_size:0.9994306231798915 - rollout_corr/rollout_is_seq_mean:1.00006902217865 - rollout_corr/rollout_is_seq_std:0.0030043653678148985 - rollout_corr/rollout_is_seq_max:1.0119823217391968 - rollout_corr/rollout_is_seq_min:0.9921797513961792 - rollout_corr/rollout_is_seq_max_deviation:0.011982321739196777 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.166721629910171) - rollout_corr/training_log_ppl:np.float64(0.14553378148048068) - rollout_corr/kl:np.float64(0.0002019084464137677) - rollout_corr/k3_kl:np.float64(0.00036560735284885126) - rollout_corr/rollout_ppl:np.float64(1.1665141666308045) - rollout_corr/rollout_log_ppl:np.float64(0.1453318719068193) - rollout_corr/log_ppl_diff:np.float64(0.00020190957366139628) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026053522124129813) - rollout_corr/log_ppl_diff_max:np.float64(0.015294685959815979) - rollout_corr/log_ppl_diff_min:np.float64(-0.012961626052856445) - rollout_corr/ppl_ratio:np.float64(1.0002088455948979) - rollout_corr/chi2_token:np.float64(0.0010437711607664824) - rollout_corr/chi2_seq:np.float64(0.05736219207756221) - actor/pg_loss:np.float64(-5.3566182032227516e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(6.442556200525473e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14063120260834694) - perf/mfu/actor:np.float64(0.06897480514788557) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.48910903930664) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-8.184913895092905e-06 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-8.184913895092905e-06 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:59.65625 - response_length/max:523.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:59.65625 - response_length_non_aborted/max:523.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:674.5625 - prompt_length/max:1096.0 - prompt_length/min:394.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.8215348720550537e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.368517130613327) - timing_s/agent_loop/generate_sequences/max:np.float64(3.0505698453634977) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7685564753192011) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.0505698453634977) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:656 - timing_s/agent_loop/slowest/response_length:523 - timing_s/gen:8.94864528439939 - timing_s/reward:0.07396487332880497 - timing_s/old_log_prob:2.347243933007121 - timing_s/adv:0.11986367218196392 - timing_s/update_actor:9.917755153030157 - timing_s/step:21.496061181649566 - timing_s/stop_profile:0.00011764466762542725 - timing_per_token_ms/update_actor:0.05276524341897296 - timing_per_token_ms/adv:0.0006377084070119383 - timing_per_token_ms/gen:0.5859511055787971 - perf/total_num_tokens:187960 - perf/time_per_step:21.496061181649566 - perf/throughput:1092.9909345465046 (TaskRunner pid=2186821) Training Progress: 58%|█████▊ | 58/100 [31:06<20:35, 29.43s/it] (TaskRunner pid=2186821) step:59 - global_seqlen/min:15432 - global_seqlen/max:23039 - global_seqlen/minmax_diff:7607 - global_seqlen/balanced_min:20035 - global_seqlen/balanced_max:20036 - global_seqlen/mean:20035.5 - actor/entropy:0.11730516701936722 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24134418368339539 - training/rollout_probs_diff_mean:0.0019489236874505877 - training/rollout_probs_diff_std:0.008438796736299992 - training/rollout_actor_probs_pearson_corr:0.9988566040992737 - rollout_corr/rollout_is_mean:0.9997319579124451 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7523545026779175 - rollout_corr/rollout_is_min:0.5325787663459778 - rollout_corr/rollout_is_std:0.023595910519361496 - rollout_corr/rollout_is_eff_sample_size:0.99944312606201 - rollout_corr/rollout_is_seq_mean:0.9999149441719055 - rollout_corr/rollout_is_seq_std:0.0039452421478927135 - rollout_corr/rollout_is_seq_max:1.0276597738265991 - rollout_corr/rollout_is_seq_min:0.9878296852111816 - rollout_corr/rollout_is_seq_max_deviation:0.02765977382659912 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.131755315233022) - rollout_corr/training_log_ppl:np.float64(0.11852792226272868) - rollout_corr/kl:np.float64(0.0003325382051393433) - rollout_corr/k3_kl:np.float64(0.00034916359861636437) - rollout_corr/rollout_ppl:np.float64(1.1313399029895663) - rollout_corr/rollout_log_ppl:np.float64(0.11819538294548693) - rollout_corr/log_ppl_diff:np.float64(0.00033253931724175345) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026639311381586595) - rollout_corr/log_ppl_diff_max:np.float64(0.013242244720458984) - rollout_corr/log_ppl_diff_min:np.float64(-0.016467127948999405) - rollout_corr/ppl_ratio:np.float64(1.0003395683597773) - rollout_corr/chi2_token:np.float64(0.0007266292814165354) - rollout_corr/chi2_seq:np.float64(0.01648964500054717) - actor/pg_loss:np.float64(0.00048130599316209555) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033832383451226633) - actor/ppo_kl:np.float64(-6.58408303486624e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(1.0383088923990726) - perf/mfu/actor:np.float64(0.058279019834465894) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.49875259399414) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01177155040204525 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.01177155040204525 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:61.265625 - response_length/max:188.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:61.265625 - response_length_non_aborted/max:188.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:564.84375 - prompt_length/max:925.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011989101767539978 - timing_s/agent_loop/generate_sequences/min:np.float64(0.36225566640496254) - timing_s/agent_loop/generate_sequences/max:np.float64(1.4305716436356306) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7251358349603834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.4305716436356306) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:685 - timing_s/agent_loop/slowest/response_length:163 - timing_s/gen:7.379183633252978 - timing_s/reward:0.06530339270830154 - timing_s/old_log_prob:2.3417293541133404 - timing_s/adv:0.12139510363340378 - timing_s/update_actor:10.008013004437089 - timing_s/step:19.973069543018937 - timing_s/stop_profile:0.00011491402983665466 - timing_per_token_ms/update_actor:0.062439251606130924 - timing_per_token_ms/adv:0.0007573750569826295 - timing_per_token_ms/gen:0.47049117784066424 - perf/total_num_tokens:160284 - perf/time_per_step:19.973069543018937 - perf/throughput:1003.1257317182318 (TaskRunner pid=2186821) Training Progress: 59%|█████▉ | 59/100 [31:26<18:10, 26.60s/it] (TaskRunner pid=2186821) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2186821) validation generation end (TaskRunner pid=2186821) [prompt] user (TaskRunner pid=2186821) Your task is to answer the user's question using available tools. (TaskRunner pid=2186821) You have access to the following tools: (TaskRunner pid=2186821) Name: Axolotl (TaskRunner pid=2186821) Description: Collection of axolotl pictures and facts (TaskRunner pid=2186821) Documentation: (TaskRunner pid=2186821) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2186821) Parameters: {} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{url, source, description} (TaskRunner pid=2186821) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2186821) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2186821) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2186821) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Array[Object{fact, source}] (TaskRunner pid=2186821) (TaskRunner pid=2186821) Use the following format: (TaskRunner pid=2186821) Thought: you should always think about what to do (TaskRunner pid=2186821) Action: the action to take, should be one of the tool names. (TaskRunner pid=2186821) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2186821) (TaskRunner pid=2186821) Begin! (TaskRunner pid=2186821) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2186821) assistant (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: I need to retrieve a random axolotl image. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2187230) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2186821) step:60 - global_seqlen/min:19655 - global_seqlen/max:26725 - global_seqlen/minmax_diff:7070 - global_seqlen/balanced_min:22283 - global_seqlen/balanced_max:22285 - global_seqlen/mean:22283.75 - actor/entropy:0.11737333983182907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12435013055801392 - training/rollout_probs_diff_mean:0.002071123570203781 - training/rollout_probs_diff_std:0.007533193100243807 - training/rollout_actor_probs_pearson_corr:0.9990726113319397 - rollout_corr/rollout_is_mean:1.0002330541610718 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6179546117782593 - rollout_corr/rollout_is_min:0.6648016571998596 - rollout_corr/rollout_is_std:0.02260114625096321 - rollout_corr/rollout_is_eff_sample_size:0.9994896871650066 - rollout_corr/rollout_is_seq_mean:1.000064730644226 - rollout_corr/rollout_is_seq_std:0.0031728087924420834 - rollout_corr/rollout_is_seq_max:1.0129413604736328 - rollout_corr/rollout_is_seq_min:0.9875838756561279 - rollout_corr/rollout_is_seq_max_deviation:0.012941360473632812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.138518909458071) - rollout_corr/training_log_ppl:np.float64(0.12504120650191908) - rollout_corr/kl:np.float64(0.0003483210107191326) - rollout_corr/k3_kl:np.float64(0.0003858008813590885) - rollout_corr/rollout_ppl:np.float64(1.1381008280441165) - rollout_corr/rollout_log_ppl:np.float64(0.12469288447391591) - rollout_corr/log_ppl_diff:np.float64(0.00034832202800316736) - rollout_corr/log_ppl_abs_diff:np.float64(0.002807611926982645) - rollout_corr/log_ppl_diff_max:np.float64(0.03661584481596947) - rollout_corr/log_ppl_diff_min:np.float64(-0.012226477265357971) - rollout_corr/ppl_ratio:np.float64(1.0003584378864616) - rollout_corr/chi2_token:np.float64(0.0008182665333151817) - rollout_corr/chi2_seq:np.float64(0.06787867774255574) - actor/pg_loss:np.float64(0.0001881817588582635) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023437499476131052) - actor/ppo_kl:np.float64(0.00015049402761491137) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3214338298421353) - perf/mfu/actor:np.float64(0.06495085722775226) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.53596496582031) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5698529411764706) - val-aux/tooluse/reward/std@16:np.float64(0.07615620935902813) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5996470588235294) - val-aux/tooluse/reward/best@2/std:np.float64(0.05626862345835548) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5401617647058823) - val-aux/tooluse/reward/worst@2/std:np.float64(0.07223619122582384) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5701323529411765) - val-aux/tooluse/reward/maj@2/std:np.float64(0.07498509960962543) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6169411764705881) - val-aux/tooluse/reward/best@4/std:np.float64(0.037178902351910376) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5078382352941176) - val-aux/tooluse/reward/worst@4/std:np.float64(0.06205494807295318) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5772941176470588) - val-aux/tooluse/reward/maj@4/std:np.float64(0.058742657752740526) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6303382352941176) - val-aux/tooluse/reward/best@8/std:np.float64(0.029464398852667613) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.48097058823529415) - val-aux/tooluse/reward/worst@8/std:np.float64(0.04337998279844502) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5849852941176471) - val-aux/tooluse/reward/maj@8/std:np.float64(0.04201769762389744) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6438235294117647) - val-aux/tooluse/reward/best@16/std:np.float64(0.025943153321102023) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.4653088235294117) - val-aux/tooluse/reward/worst@16/std:np.float64(0.02175179044551583) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5912941176470587) - val-aux/tooluse/reward/maj@16/std:np.float64(0.026493333036556047) - val-aux/tooluse/score/mean@16:np.float64(0.5698529411764706) - val-aux/tooluse/score/std@16:np.float64(0.07615620935902813) - val-aux/tooluse/score/best@2/mean:np.float64(0.5996470588235294) - val-aux/tooluse/score/best@2/std:np.float64(0.05626862345835548) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5401617647058823) - val-aux/tooluse/score/worst@2/std:np.float64(0.07223619122582384) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5701323529411765) - val-aux/tooluse/score/maj@2/std:np.float64(0.07498509960962543) - val-aux/tooluse/score/best@4/mean:np.float64(0.6169411764705881) - val-aux/tooluse/score/best@4/std:np.float64(0.037178902351910376) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5078382352941176) - val-aux/tooluse/score/worst@4/std:np.float64(0.06205494807295318) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5772941176470588) - val-aux/tooluse/score/maj@4/std:np.float64(0.058742657752740526) - val-aux/tooluse/score/best@8/mean:np.float64(0.6303382352941176) - val-aux/tooluse/score/best@8/std:np.float64(0.029464398852667613) - val-aux/tooluse/score/worst@8/mean:np.float64(0.48097058823529415) - val-aux/tooluse/score/worst@8/std:np.float64(0.04337998279844502) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5849852941176471) - val-aux/tooluse/score/maj@8/std:np.float64(0.04201769762389744) - val-aux/tooluse/score/best@16/mean:np.float64(0.6438235294117647) - val-aux/tooluse/score/best@16/std:np.float64(0.025943153321102023) - val-aux/tooluse/score/worst@16/mean:np.float64(0.4653088235294117) - val-aux/tooluse/score/worst@16/std:np.float64(0.02175179044551583) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5912941176470587) - val-aux/tooluse/score/maj@16/std:np.float64(0.026493333036556047) - val-core/tooluse/acc/mean@16:np.float64(0.5698529411764706) - val-aux/tooluse/acc/std@16:np.float64(0.07615620935902813) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5996470588235294) - val-aux/tooluse/acc/best@2/std:np.float64(0.05626862345835548) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5401617647058823) - val-aux/tooluse/acc/worst@2/std:np.float64(0.07223619122582384) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5701323529411765) - val-aux/tooluse/acc/maj@2/std:np.float64(0.07498509960962543) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6169411764705881) - val-aux/tooluse/acc/best@4/std:np.float64(0.037178902351910376) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5078382352941176) - val-aux/tooluse/acc/worst@4/std:np.float64(0.06205494807295318) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5772941176470588) - val-aux/tooluse/acc/maj@4/std:np.float64(0.058742657752740526) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6303382352941176) - val-aux/tooluse/acc/best@8/std:np.float64(0.029464398852667613) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.48097058823529415) - val-aux/tooluse/acc/worst@8/std:np.float64(0.04337998279844502) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5849852941176471) - val-aux/tooluse/acc/maj@8/std:np.float64(0.04201769762389744) - val-core/tooluse/acc/best@16/mean:np.float64(0.6438235294117647) - val-core/tooluse/acc/best@16/std:np.float64(0.025943153321102023) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.4653088235294117) - val-aux/tooluse/acc/worst@16/std:np.float64(0.02175179044551583) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5912941176470587) - val-core/tooluse/acc/maj@16/std:np.float64(0.026493333036556047) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-8.344459638465196e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-8.344459638465196e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:64.3671875 - response_length/max:302.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:64.3671875 - response_length_non_aborted/max:302.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:632.0 - prompt_length/max:1266.0 - prompt_length/min:231.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001582745462656021 - timing_s/agent_loop/generate_sequences/min:np.float64(0.30406797118484974) - timing_s/agent_loop/generate_sequences/max:np.float64(1.9876642487943172) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7035536118710297) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.9876642487943172) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:822 - timing_s/agent_loop/slowest/response_length:302 - timing_s/gen:8.254790261387825 - timing_s/reward:0.07222243957221508 - timing_s/old_log_prob:2.394257690757513 - timing_s/adv:0.12101082503795624 - timing_s/update_actor:9.813037015497684 - timing_s/step:20.742113484069705 - timing_s/testing:19.449192071333528 - timing_s/save_checkpoint:6.7468203362077475 - timing_s/stop_profile:0.00011053122580051422 - timing_per_token_ms/update_actor:0.0550459248078627 - timing_per_token_ms/adv:0.0006788064454925464 - timing_per_token_ms/gen:0.5009582632229533 - perf/total_num_tokens:178270 - perf/time_per_step:20.742113484069705 - perf/throughput:1074.323984251378 (TaskRunner pid=2186821) Training Progress: 60%|██████ | 60/100 [32:13<21:49, 32.73s/it] (TaskRunner pid=2186821) step:61 - global_seqlen/min:16305 - global_seqlen/max:24774 - global_seqlen/minmax_diff:8469 - global_seqlen/balanced_min:21085 - global_seqlen/balanced_max:21086 - global_seqlen/mean:21085.625 - actor/entropy:0.1431281566619873 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23160694539546967 - training/rollout_probs_diff_mean:0.0026310908142477274 - training/rollout_probs_diff_std:0.008698221296072006 - training/rollout_actor_probs_pearson_corr:0.9989337921142578 - rollout_corr/rollout_is_mean:0.9998639225959778 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.236553549068049e-05 - rollout_corr/rollout_is_max:1.3905707597732544 - rollout_corr/rollout_is_min:0.1387927532196045 - rollout_corr/rollout_is_std:0.0252092182636261 - rollout_corr/rollout_is_eff_sample_size:0.9993646607713826 - rollout_corr/rollout_is_seq_mean:0.9999908208847046 - rollout_corr/rollout_is_seq_std:0.0036935685202479362 - rollout_corr/rollout_is_seq_max:1.015987515449524 - rollout_corr/rollout_is_seq_min:0.9856516718864441 - rollout_corr/rollout_is_seq_max_deviation:0.015987515449523926 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.149116420187056) - rollout_corr/training_log_ppl:np.float64(0.1328091084415064) - rollout_corr/kl:np.float64(0.00041618762830708533) - rollout_corr/k3_kl:np.float64(0.0006400519962772577) - rollout_corr/rollout_ppl:np.float64(1.148430076893419) - rollout_corr/rollout_log_ppl:np.float64(0.13239292024445604) - rollout_corr/log_ppl_diff:np.float64(0.00041618819705036003) - rollout_corr/log_ppl_abs_diff:np.float64(0.0039976443422347074) - rollout_corr/log_ppl_diff_max:np.float64(0.02162143588066101) - rollout_corr/log_ppl_diff_min:np.float64(-0.022468186914920807) - rollout_corr/ppl_ratio:np.float64(1.0004322663880885) - rollout_corr/chi2_token:np.float64(0.0016511671710759401) - rollout_corr/chi2_seq:np.float64(0.07527914713136852) - actor/pg_loss:np.float64(-0.0002816558117046952) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006089745438657701) - actor/ppo_kl:np.float64(4.5401107612974556e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.534139946103096) - perf/mfu/actor:np.float64(0.061207876711523376) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.92566299438477) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:0 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0026356971357017756 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0026356971357017756 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:47.42578125 - response_length/max:201.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:47.42578125 - response_length_non_aborted/max:201.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:611.5 - prompt_length/max:1107.0 - prompt_length/min:294.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.182173430919647e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.42757853493094444) - timing_s/agent_loop/generate_sequences/max:np.float64(1.3821868281811476) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6472933545228443) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.3821868281811476) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:658 - timing_s/agent_loop/slowest/response_length:201 - timing_s/gen:7.37207162566483 - timing_s/reward:0.06708627194166183 - timing_s/old_log_prob:2.511333314701915 - timing_s/adv:0.12446930445730686 - timing_s/update_actor:9.942053593695164 - timing_s/step:20.10321070998907 - timing_s/stop_profile:0.00011691451072692871 - timing_per_token_ms/update_actor:0.05893857541390855 - timing_per_token_ms/adv:0.0007378800987480028 - timing_per_token_ms/gen:0.6072046475302554 - perf/total_num_tokens:168685 - perf/time_per_step:20.10321070998907 - perf/throughput:1048.868526733532 (TaskRunner pid=2186821) Training Progress: 61%|██████ | 61/100 [32:33<18:49, 28.96s/it] (TaskRunner pid=2186821) step:62 - global_seqlen/min:17565 - global_seqlen/max:24751 - global_seqlen/minmax_diff:7186 - global_seqlen/balanced_min:21903 - global_seqlen/balanced_max:21905 - global_seqlen/mean:21904.25 - actor/entropy:0.11886042356491089 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.168881356716156 - training/rollout_probs_diff_mean:0.0024704409297555685 - training/rollout_probs_diff_std:0.008923884481191635 - training/rollout_actor_probs_pearson_corr:0.9986943602561951 - rollout_corr/rollout_is_mean:0.9996351003646851 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.574698567390442 - rollout_corr/rollout_is_min:0.6108553409576416 - rollout_corr/rollout_is_std:0.025602193549275398 - rollout_corr/rollout_is_eff_sample_size:0.9993445403979695 - rollout_corr/rollout_is_seq_mean:0.9997009038925171 - rollout_corr/rollout_is_seq_std:0.004095304757356644 - rollout_corr/rollout_is_seq_max:1.026859164237976 - rollout_corr/rollout_is_seq_min:0.9848344326019287 - rollout_corr/rollout_is_seq_max_deviation:0.026859164237976074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1315844510681927) - rollout_corr/training_log_ppl:np.float64(0.12044897437590407) - rollout_corr/kl:np.float64(0.0010513766884443498) - rollout_corr/k3_kl:np.float64(0.0007216799451670256) - rollout_corr/rollout_ppl:np.float64(1.1302654375322163) - rollout_corr/rollout_log_ppl:np.float64(0.11939759689266793) - rollout_corr/log_ppl_diff:np.float64(0.0010513774832361378) - rollout_corr/log_ppl_abs_diff:np.float64(0.004160821372352075) - rollout_corr/log_ppl_diff_max:np.float64(0.03678739070892334) - rollout_corr/log_ppl_diff_min:np.float64(-0.019756071269512177) - rollout_corr/ppl_ratio:np.float64(1.0010713175870478) - rollout_corr/chi2_token:np.float64(0.0008089393377304077) - rollout_corr/chi2_seq:np.float64(0.062061256961897016) - actor/pg_loss:np.float64(0.00010176748037338257) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017361111531499773) - actor/ppo_kl:np.float64(0.00040047723710046057) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.059641916304826736) - perf/mfu/actor:np.float64(0.06312725657210846) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.84144592285156) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:0 - critic/score/mean:0.33984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004315203696023673 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004315203696023673 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:55.4453125 - response_length/max:161.0 - response_length/min:20.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:55.4453125 - response_length_non_aborted/max:161.0 - response_length_non_aborted/min:20.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:629.0625 - prompt_length/max:1067.0 - prompt_length/min:387.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001513417810201645 - timing_s/agent_loop/generate_sequences/min:np.float64(0.32755867578089237) - timing_s/agent_loop/generate_sequences/max:np.float64(1.3986550010740757) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6741413225681754) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.3986550010740757) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1067 - timing_s/agent_loop/slowest/response_length:153 - timing_s/gen:7.271084057167172 - timing_s/reward:0.06812154874205589 - timing_s/old_log_prob:2.371399072930217 - timing_s/adv:0.11961814016103745 - timing_s/update_actor:10.081531595438719 - timing_s/step:19.994628623127937 - timing_s/stop_profile:0.0001157708466053009 - timing_per_token_ms/update_actor:0.057531823706807574 - timing_per_token_ms/adv:0.0006826194697435283 - timing_per_token_ms/gen:0.5122646228805955 - perf/total_num_tokens:175234 - perf/time_per_step:19.994628623127937 - perf/throughput:1095.506718972674 (TaskRunner pid=2186821) Training Progress: 62%|██████▏ | 62/100 [32:53<16:38, 26.28s/it] (TaskRunner pid=2186821) step:63 - global_seqlen/min:14803 - global_seqlen/max:30176 - global_seqlen/minmax_diff:15373 - global_seqlen/balanced_min:21791 - global_seqlen/balanced_max:27273 - global_seqlen/mean:22478.375 - actor/entropy:0.08528505265712738 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47441259026527405 - training/rollout_probs_diff_mean:0.0017882190877571702 - training/rollout_probs_diff_std:0.008737008087337017 - training/rollout_actor_probs_pearson_corr:0.9983723163604736 - rollout_corr/rollout_is_mean:1.0001095533370972 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.209640246699564e-05 - rollout_corr/rollout_is_max:1.9363701343536377 - rollout_corr/rollout_is_min:0.44314664602279663 - rollout_corr/rollout_is_std:0.022925831377506256 - rollout_corr/rollout_is_eff_sample_size:0.9994746823460503 - rollout_corr/rollout_is_seq_mean:0.999945878982544 - rollout_corr/rollout_is_seq_std:0.0032501474488526583 - rollout_corr/rollout_is_seq_max:1.0091886520385742 - rollout_corr/rollout_is_seq_min:0.9875873923301697 - rollout_corr/rollout_is_seq_max_deviation:0.012412607669830322 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.116357317660004) - rollout_corr/training_log_ppl:np.float64(0.10595428082888247) - rollout_corr/kl:np.float64(0.0006757602736904289) - rollout_corr/k3_kl:np.float64(0.000400009842309354) - rollout_corr/rollout_ppl:np.float64(1.115600806195289) - rollout_corr/rollout_log_ppl:np.float64(0.10527851938422828) - rollout_corr/log_ppl_diff:np.float64(0.0006757614446541993) - rollout_corr/log_ppl_abs_diff:np.float64(0.002838110469383537) - rollout_corr/log_ppl_diff_max:np.float64(0.01647457294166088) - rollout_corr/log_ppl_diff_min:np.float64(-0.013881832361221313) - rollout_corr/ppl_ratio:np.float64(1.0006836995016783) - rollout_corr/chi2_token:np.float64(0.00024244561791419983) - rollout_corr/chi2_seq:np.float64(0.15850739646703005) - actor/pg_loss:np.float64(0.00010369345545768738) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0003014751285945749) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24026023596525192) - perf/mfu/actor:np.float64(0.0622725889165461) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.67060089111328) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:0 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003715007333084941 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.003715007333084941 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:92.79296875 - response_length/max:6339.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:92.79296875 - response_length_non_aborted/max:6339.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:609.65625 - prompt_length/max:1090.0 - prompt_length/min:334.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013208016753196716 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3095095120370388) - timing_s/agent_loop/generate_sequences/max:np.float64(30.474849607795477) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8861954648018582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.474849607795477) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:658 - timing_s/agent_loop/slowest/response_length:6339 - timing_s/gen:36.32377944327891 - timing_s/reward:0.10488219559192657 - timing_s/old_log_prob:2.598118981346488 - timing_s/adv:0.12427417747676373 - timing_s/update_actor:10.530107263475657 - timing_s/step:49.77264612726867 - timing_s/stop_profile:0.00011328421533107758 - timing_per_token_ms/update_actor:0.05855687557194224 - timing_per_token_ms/adv:0.0006910762982019592 - timing_per_token_ms/gen:1.529100376479853 - perf/total_num_tokens:179827 - perf/time_per_step:49.77264612726867 - perf/throughput:451.6210559214149 (TaskRunner pid=2186821) Training Progress: 63%|██████▎ | 63/100 [33:43<20:33, 33.34s/it] (TaskRunner pid=2186821) step:64 - global_seqlen/min:18306 - global_seqlen/max:27418 - global_seqlen/minmax_diff:9112 - global_seqlen/balanced_min:21535 - global_seqlen/balanced_max:21540 - global_seqlen/mean:21538.0 - actor/entropy:0.11135164648294449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19308286905288696 - training/rollout_probs_diff_mean:0.0025100891944020987 - training/rollout_probs_diff_std:0.009406065568327904 - training/rollout_actor_probs_pearson_corr:0.9984923601150513 - rollout_corr/rollout_is_mean:0.9999561309814453 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8682419061660767 - rollout_corr/rollout_is_min:0.574765682220459 - rollout_corr/rollout_is_std:0.02667955495417118 - rollout_corr/rollout_is_eff_sample_size:0.9992887076290544 - rollout_corr/rollout_is_seq_mean:1.0001846551895142 - rollout_corr/rollout_is_seq_std:0.0035255998373031616 - rollout_corr/rollout_is_seq_max:1.0185474157333374 - rollout_corr/rollout_is_seq_min:0.9897435307502747 - rollout_corr/rollout_is_seq_max_deviation:0.018547415733337402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1150092212483287) - rollout_corr/training_log_ppl:np.float64(0.10501038596521539) - rollout_corr/kl:np.float64(-9.648062473388563e-07) - rollout_corr/k3_kl:np.float64(0.0003767458835342019) - rollout_corr/rollout_ppl:np.float64(1.1150037543848157) - rollout_corr/rollout_log_ppl:np.float64(0.10501134932383138) - rollout_corr/log_ppl_diff:np.float64(-9.633586159907281e-07) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025862482798402198) - rollout_corr/log_ppl_diff_max:np.float64(0.009196415543556213) - rollout_corr/log_ppl_diff_min:np.float64(-0.015879735350608826) - rollout_corr/ppl_ratio:np.float64(1.0000058086588979) - rollout_corr/chi2_token:np.float64(0.0015549135860055685) - rollout_corr/chi2_seq:np.float64(0.0646910578943789) - actor/pg_loss:np.float64(3.539444878697395e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.084302291739732e-05) - actor/ppo_kl:np.float64(-0.00014173465454092593) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20763375610113144) - perf/mfu/actor:np.float64(0.06317185527502119) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.65876388549805) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:0 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003250253794249147 - critic/advantages/max:0.625 - critic/advantages/min:-0.375 - critic/returns/mean:0.0003250253794249147 - critic/returns/max:0.625 - critic/returns/min:-0.375 - response_length/mean:61.59375 - response_length/max:314.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:61.59375 - response_length_non_aborted/max:314.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:611.46875 - prompt_length/max:1282.0 - prompt_length/min:313.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012438185513019562 - timing_s/agent_loop/generate_sequences/min:np.float64(0.198879212141037) - timing_s/agent_loop/generate_sequences/max:np.float64(2.0189621057361364) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6909046956207021) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.0189621057361364) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1166 - timing_s/agent_loop/slowest/response_length:314 - timing_s/gen:8.027167243883014 - timing_s/reward:0.07141768746078014 - timing_s/old_log_prob:2.329207956790924 - timing_s/adv:0.12327419221401215 - timing_s/update_actor:9.960799409076571 - timing_s/step:20.59943986684084 - timing_s/stop_profile:0.00011242926120758057 - timing_per_token_ms/update_actor:0.057809449630168604 - timing_per_token_ms/adv:0.0007154459107972662 - timing_per_token_ms/gen:0.5090796070448386 - perf/total_num_tokens:172304 - perf/time_per_step:20.59943986684084 - perf/throughput:1045.5624103968948 (TaskRunner pid=2186821) Training Progress: 64%|██████▍ | 64/100 [34:04<17:43, 29.53s/it] (TaskRunner pid=2186821) step:65 - global_seqlen/min:14600 - global_seqlen/max:24241 - global_seqlen/minmax_diff:9641 - global_seqlen/balanced_min:19847 - global_seqlen/balanced_max:19849 - global_seqlen/mean:19848.5 - actor/entropy:0.10534079372882843 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17359209060668945 - training/rollout_probs_diff_mean:0.002186333294957876 - training/rollout_probs_diff_std:0.008365694433450699 - training/rollout_actor_probs_pearson_corr:0.9987361431121826 - rollout_corr/rollout_is_mean:1.000185251235962 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.27352565061301e-05 - rollout_corr/rollout_is_max:1.5868637561798096 - rollout_corr/rollout_is_min:0.4158308506011963 - rollout_corr/rollout_is_std:0.024988362565636635 - rollout_corr/rollout_is_eff_sample_size:0.9993763285779826 - rollout_corr/rollout_is_seq_mean:1.0001742839813232 - rollout_corr/rollout_is_seq_std:0.003700182307511568 - rollout_corr/rollout_is_seq_max:1.0183687210083008 - rollout_corr/rollout_is_seq_min:0.9882088899612427 - rollout_corr/rollout_is_seq_max_deviation:0.01836872100830078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1083924761041999) - rollout_corr/training_log_ppl:np.float64(0.09991783006626065) - rollout_corr/kl:np.float64(0.0007350788401048192) - rollout_corr/k3_kl:np.float64(0.0006820513079526247) - rollout_corr/rollout_ppl:np.float64(1.1075742891989648) - rollout_corr/rollout_log_ppl:np.float64(0.0991827499892679) - rollout_corr/log_ppl_diff:np.float64(0.0007350800769927446) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031701580846856814) - rollout_corr/log_ppl_diff_max:np.float64(0.06005068123340607) - rollout_corr/log_ppl_diff_min:np.float64(-0.014482289552688599) - rollout_corr/ppl_ratio:np.float64(1.0007509994320571) - rollout_corr/chi2_token:np.float64(0.0008519587572664022) - rollout_corr/chi2_seq:np.float64(0.05666199978441) - actor/pg_loss:np.float64(-0.00026839436031877995) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005075112803751836) - actor/ppo_kl:np.float64(0.0005840546220881038) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25221172720193863) - perf/mfu/actor:np.float64(0.0586596987849775) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.70203018188477) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:0 - critic/score/mean:0.30859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0015370137989521027 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:0.0015370137989521027 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:62.265625 - response_length/max:367.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:62.265625 - response_length_non_aborted/max:367.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:558.0 - prompt_length/max:1098.0 - prompt_length/min:239.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001517515629529953 - timing_s/agent_loop/generate_sequences/min:np.float64(0.289266187697649) - timing_s/agent_loop/generate_sequences/max:np.float64(2.294645592570305) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6458261714797118) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.294645592570305) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:843 - timing_s/agent_loop/slowest/response_length:351 - timing_s/gen:8.031295906752348 - timing_s/reward:0.06731517426669598 - timing_s/old_log_prob:2.30041185580194 - timing_s/adv:0.11953642405569553 - timing_s/update_actor:9.628395600244403 - timing_s/step:20.233721885830164 - timing_s/stop_profile:0.00011358223855495453 - timing_per_token_ms/update_actor:0.06063679623299244 - timing_per_token_ms/adv:0.0007528051493544571 - timing_per_token_ms/gen:0.5038454144763079 - perf/total_num_tokens:158788 - perf/time_per_step:20.233721885830164 - perf/throughput:980.9613926689416 (TaskRunner pid=2186821) Training Progress: 65%|██████▌ | 65/100 [34:24<15:36, 26.76s/it] (TaskRunner pid=2186821) step:66 - global_seqlen/min:16279 - global_seqlen/max:28442 - global_seqlen/minmax_diff:12163 - global_seqlen/balanced_min:21770 - global_seqlen/balanced_max:21780 - global_seqlen/mean:21775.0 - actor/entropy:0.11566649377346039 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1477210521697998 - training/rollout_probs_diff_mean:0.0023267834912985563 - training/rollout_probs_diff_std:0.008381574414670467 - training/rollout_actor_probs_pearson_corr:0.9988042712211609 - rollout_corr/rollout_is_mean:1.0002785921096802 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8682419061660767 - rollout_corr/rollout_is_min:0.6180638670921326 - rollout_corr/rollout_is_std:0.025324810296297073 - rollout_corr/rollout_is_eff_sample_size:0.9993596603662409 - rollout_corr/rollout_is_seq_mean:1.0002113580703735 - rollout_corr/rollout_is_seq_std:0.003810943104326725 - rollout_corr/rollout_is_seq_max:1.024300456047058 - rollout_corr/rollout_is_seq_min:0.9887398481369019 - rollout_corr/rollout_is_seq_max_deviation:0.024300456047058105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1119779921136796) - rollout_corr/training_log_ppl:np.float64(0.10315267260011751) - rollout_corr/kl:np.float64(0.00011603397713599861) - rollout_corr/k3_kl:np.float64(0.000378654260745237) - rollout_corr/rollout_ppl:np.float64(1.111866609659046) - rollout_corr/rollout_log_ppl:np.float64(0.10303663826198317) - rollout_corr/log_ppl_diff:np.float64(0.00011603433813434094) - rollout_corr/log_ppl_abs_diff:np.float64(0.002915127668529749) - rollout_corr/log_ppl_diff_max:np.float64(0.010114789009094238) - rollout_corr/log_ppl_diff_min:np.float64(-0.019321218132972717) - rollout_corr/ppl_ratio:np.float64(1.000124613987282) - rollout_corr/chi2_token:np.float64(0.001349401194602251) - rollout_corr/chi2_seq:np.float64(0.13869083835743368) - actor/pg_loss:np.float64(0.0004653114592656493) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038026632682885975) - actor/ppo_kl:np.float64(1.4615076821655748e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.47829057462513447) - perf/mfu/actor:np.float64(0.06351485054670433) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.66472244262695) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036742330994457006 - critic/advantages/max:0.25 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036742330994457006 - critic/returns/max:0.25 - critic/returns/min:-0.875 - response_length/mean:64.1875 - response_length/max:263.0 - response_length/min:21.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:64.1875 - response_length_non_aborted/max:263.0 - response_length_non_aborted/min:21.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:616.28125 - prompt_length/max:1266.0 - prompt_length/min:301.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.888783097267151e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3741486594080925) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7456177342683077) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7557336804748047) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7456177342683077) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1266 - timing_s/agent_loop/slowest/response_length:263 - timing_s/gen:8.006275936961174 - timing_s/reward:0.06622691079974174 - timing_s/old_log_prob:2.4285263549536467 - timing_s/adv:0.14698189496994019 - timing_s/update_actor:10.01797054335475 - timing_s/step:20.735631804913282 - timing_s/stop_profile:0.00013743899762630463 - timing_per_token_ms/update_actor:0.05750844169549225 - timing_per_token_ms/adv:0.0008437537024680837 - timing_per_token_ms/gen:0.4872368510808894 - perf/total_num_tokens:174200 - perf/time_per_step:20.735631804913282 - perf/throughput:1050.1247420317543 (TaskRunner pid=2186821) Training Progress: 66%|██████▌ | 66/100 [34:45<14:08, 24.97s/it] (TaskRunner pid=2186821) step:67 - global_seqlen/min:18309 - global_seqlen/max:27022 - global_seqlen/minmax_diff:8713 - global_seqlen/balanced_min:21570 - global_seqlen/balanced_max:21571 - global_seqlen/mean:21570.375 - actor/entropy:0.1386716067790985 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12738272547721863 - training/rollout_probs_diff_mean:0.002327083144336939 - training/rollout_probs_diff_std:0.007890698499977589 - training/rollout_actor_probs_pearson_corr:0.9991012215614319 - rollout_corr/rollout_is_mean:1.0001142024993896 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.4434194564819336 - rollout_corr/rollout_is_min:0.662186324596405 - rollout_corr/rollout_is_std:0.023212624713778496 - rollout_corr/rollout_is_eff_sample_size:0.999461583269263 - rollout_corr/rollout_is_seq_mean:1.000009536743164 - rollout_corr/rollout_is_seq_std:0.0034106788225471973 - rollout_corr/rollout_is_seq_max:1.0132274627685547 - rollout_corr/rollout_is_seq_min:0.9887202978134155 - rollout_corr/rollout_is_seq_max_deviation:0.013227462768554688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1490855081938207) - rollout_corr/training_log_ppl:np.float64(0.13255107589975523) - rollout_corr/kl:np.float64(0.0006250192089369833) - rollout_corr/k3_kl:np.float64(0.0009286665210046507) - rollout_corr/rollout_ppl:np.float64(1.1483208891004324) - rollout_corr/rollout_log_ppl:np.float64(0.13192605532276502) - rollout_corr/log_ppl_diff:np.float64(0.0006250205769902095) - rollout_corr/log_ppl_abs_diff:np.float64(0.003652114450233057) - rollout_corr/log_ppl_diff_max:np.float64(0.030259907245635986) - rollout_corr/log_ppl_diff_min:np.float64(-0.0346045047044754) - rollout_corr/ppl_ratio:np.float64(1.0006418467964977) - rollout_corr/chi2_token:np.float64(0.004107039421796799) - rollout_corr/chi2_seq:np.float64(0.19155026762746274) - actor/pg_loss:np.float64(-6.401096470654011e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009699253059807234) - actor/ppo_kl:np.float64(0.00035800067613322994) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5406112484633923) - perf/mfu/actor:np.float64(0.06365219355712572) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.68170166015625) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004099372308701277 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004099372308701277 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:55.38671875 - response_length/max:280.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:55.38671875 - response_length_non_aborted/max:280.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:618.6875 - prompt_length/max:974.0 - prompt_length/min:259.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001452229917049408 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3767722398042679) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7952134013175964) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6910717051578104) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7952134013175964) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:928 - timing_s/agent_loop/slowest/response_length:280 - timing_s/gen:7.702375374734402 - timing_s/reward:0.07097534090280533 - timing_s/old_log_prob:2.3013424053788185 - timing_s/adv:0.11997343227267265 - timing_s/update_actor:9.759083239361644 - timing_s/step:20.038989229127765 - timing_s/stop_profile:0.00011456012725830078 - timing_per_token_ms/update_actor:0.056553741180679774 - timing_per_token_ms/adv:0.0006952442428137703 - timing_per_token_ms/gen:0.5432241607119263 - perf/total_num_tokens:172563 - perf/time_per_step:20.038989229127765 - perf/throughput:1076.4203100945972 (TaskRunner pid=2186821) Training Progress: 67%|██████▋ | 67/100 [35:05<12:55, 23.50s/it] (TaskRunner pid=2186821) step:68 - global_seqlen/min:14517 - global_seqlen/max:25266 - global_seqlen/minmax_diff:10749 - global_seqlen/balanced_min:20501 - global_seqlen/balanced_max:20870 - global_seqlen/mean:20573.125 - actor/entropy:0.10662759095430374 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39484357833862305 - training/rollout_probs_diff_mean:0.002211408456787467 - training/rollout_probs_diff_std:0.008915318176150322 - training/rollout_actor_probs_pearson_corr:0.9985798001289368 - rollout_corr/rollout_is_mean:0.9998282790184021 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.957700341241434e-05 - rollout_corr/rollout_is_max:1.3801378011703491 - rollout_corr/rollout_is_min:0.45617926120758057 - rollout_corr/rollout_is_std:0.022121349349617958 - rollout_corr/rollout_is_eff_sample_size:0.9995107661240452 - rollout_corr/rollout_is_seq_mean:0.9997799396514893 - rollout_corr/rollout_is_seq_std:0.0030584156047552824 - rollout_corr/rollout_is_seq_max:1.0100442171096802 - rollout_corr/rollout_is_seq_min:0.9878214001655579 - rollout_corr/rollout_is_seq_max_deviation:0.012178599834442139 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.123645728919655) - rollout_corr/training_log_ppl:np.float64(0.11266848256309459) - rollout_corr/kl:np.float64(0.0007945385357366885) - rollout_corr/k3_kl:np.float64(0.0005963715714711049) - rollout_corr/rollout_ppl:np.float64(1.1226090472191572) - rollout_corr/rollout_log_ppl:np.float64(0.11187394349508395) - rollout_corr/log_ppl_diff:np.float64(0.0007945390680106357) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034221571258967742) - rollout_corr/log_ppl_diff_max:np.float64(0.02311807870864868) - rollout_corr/log_ppl_diff_min:np.float64(-0.01669842004776001) - rollout_corr/ppl_ratio:np.float64(1.0008066503796726) - rollout_corr/chi2_token:np.float64(0.0006675580516457558) - rollout_corr/chi2_seq:np.float64(0.024267845088616014) - actor/pg_loss:np.float64(-0.00029652018565684557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012710814189631492) - actor/ppo_kl:np.float64(0.00031021224599214747) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18935751169919968) - perf/mfu/actor:np.float64(0.059948751668886927) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.80745315551758) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:0 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003358653513714671 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.003358653513714671 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:65.56640625 - response_length/max:1069.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:65.56640625 - response_length_non_aborted/max:1069.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:577.34375 - prompt_length/max:943.0 - prompt_length/min:243.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014813058078289032 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3193749114871025) - timing_s/agent_loop/generate_sequences/max:np.float64(5.3926901537925005) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6515206723779556) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.3926901537925005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:655 - timing_s/agent_loop/slowest/response_length:1069 - timing_s/gen:11.28217645920813 - timing_s/reward:0.06865314394235611 - timing_s/old_log_prob:2.355808090418577 - timing_s/adv:0.11834297701716423 - timing_s/update_actor:9.789116937667131 - timing_s/step:23.687935914844275 - timing_s/stop_profile:0.00011312216520309448 - timing_per_token_ms/update_actor:0.05947757655720225 - timing_per_token_ms/adv:0.0007190386549027203 - timing_per_token_ms/gen:0.6721582638789473 - perf/total_num_tokens:164585 - perf/time_per_step:23.687935914844275 - perf/throughput:868.5064445445266 (TaskRunner pid=2186821) Training Progress: 68%|██████▊ | 68/100 [35:29<12:34, 23.58s/it] (TaskRunner pid=2186821) step:69 - global_seqlen/min:19851 - global_seqlen/max:28305 - global_seqlen/minmax_diff:8454 - global_seqlen/balanced_min:22312 - global_seqlen/balanced_max:22314 - global_seqlen/mean:22313.25 - actor/entropy:0.07839744538068771 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12435317039489746 - training/rollout_probs_diff_mean:0.0019370357040315866 - training/rollout_probs_diff_std:0.008107616566121578 - training/rollout_actor_probs_pearson_corr:0.99845290184021 - rollout_corr/rollout_is_mean:0.9996426105499268 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5068966150283813 - rollout_corr/rollout_is_min:0.5352832674980164 - rollout_corr/rollout_is_std:0.022748352959752083 - rollout_corr/rollout_is_eff_sample_size:0.9994824228709994 - rollout_corr/rollout_is_seq_mean:0.9995903372764587 - rollout_corr/rollout_is_seq_std:0.0035978280939161777 - rollout_corr/rollout_is_seq_max:1.0198397636413574 - rollout_corr/rollout_is_seq_min:0.9852287769317627 - rollout_corr/rollout_is_seq_max_deviation:0.019839763641357422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0926828631199896) - rollout_corr/training_log_ppl:np.float64(0.08574950812908355) - rollout_corr/kl:np.float64(0.0014661419390655794) - rollout_corr/k3_kl:np.float64(0.0013016935036729155) - rollout_corr/rollout_ppl:np.float64(1.0909414659254253) - rollout_corr/rollout_log_ppl:np.float64(0.0842833658980453) - rollout_corr/log_ppl_diff:np.float64(0.0014661422092103749) - rollout_corr/log_ppl_abs_diff:np.float64(0.00403502545759693) - rollout_corr/log_ppl_diff_max:np.float64(0.07710502296686172) - rollout_corr/log_ppl_diff_min:np.float64(-0.021180443465709686) - rollout_corr/ppl_ratio:np.float64(1.0015123777557164) - rollout_corr/chi2_token:np.float64(0.0008243031334131956) - rollout_corr/chi2_seq:np.float64(0.021936802892014384) - actor/pg_loss:np.float64(-0.0004004109650850296) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038476591726066545) - actor/ppo_kl:np.float64(0.000767709411054085) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5201289840042591) - perf/mfu/actor:np.float64(0.06520100913875665) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.80792617797852) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003168202703818679 - critic/advantages/max:0.75 - critic/advantages/min:-0.625 - critic/returns/mean:-0.003168202703818679 - critic/returns/max:0.75 - critic/returns/min:-0.625 - response_length/mean:62.7265625 - response_length/max:349.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:62.7265625 - response_length_non_aborted/max:349.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:634.5625 - prompt_length/max:1228.0 - prompt_length/min:238.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012427940964698792 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3433664795011282) - timing_s/agent_loop/generate_sequences/max:np.float64(2.108884857967496) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7166416698164539) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.108884857967496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:690 - timing_s/agent_loop/slowest/response_length:349 - timing_s/gen:7.866490313783288 - timing_s/reward:0.0744597427546978 - timing_s/old_log_prob:2.4035667441785336 - timing_s/adv:0.11865086294710636 - timing_s/update_actor:9.976531768217683 - timing_s/step:20.526024479418993 - timing_s/stop_profile:0.00011589750647544861 - timing_per_token_ms/update_actor:0.05588905565200992 - timing_per_token_ms/adv:0.000664688374324148 - timing_per_token_ms/gen:0.48987983022688303 - perf/total_num_tokens:178506 - perf/time_per_step:20.526024479418993 - perf/throughput:1087.0711969759668 (TaskRunner pid=2186821) Training Progress: 69%|██████▉ | 69/100 [35:49<11:42, 22.68s/it] (TaskRunner pid=2186821) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2186821) validation generation end (TaskRunner pid=2186821) [prompt] user (TaskRunner pid=2186821) Your task is to answer the user's question using available tools. (TaskRunner pid=2186821) You have access to the following tools: (TaskRunner pid=2186821) Name: Axolotl (TaskRunner pid=2186821) Description: Collection of axolotl pictures and facts (TaskRunner pid=2186821) Documentation: (TaskRunner pid=2186821) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2186821) Parameters: {} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{url, source, description} (TaskRunner pid=2186821) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2186821) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2186821) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2186821) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Array[Object{fact, source}] (TaskRunner pid=2186821) (TaskRunner pid=2186821) Use the following format: (TaskRunner pid=2186821) Thought: you should always think about what to do (TaskRunner pid=2186821) Action: the action to take, should be one of the tool names. (TaskRunner pid=2186821) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2186821) (TaskRunner pid=2186821) Begin! (TaskRunner pid=2186821) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2186821) assistant (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: I need to retrieve a random axolotl image. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2187230) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2186821) step:70 - global_seqlen/min:16187 - global_seqlen/max:23570 - global_seqlen/minmax_diff:7383 - global_seqlen/balanced_min:19682 - global_seqlen/balanced_max:19683 - global_seqlen/mean:19682.375 - actor/entropy:0.08844967186450958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12239843606948853 - training/rollout_probs_diff_mean:0.0023390837013721466 - training/rollout_probs_diff_std:0.009418622590601444 - training/rollout_actor_probs_pearson_corr:0.9980756044387817 - rollout_corr/rollout_is_mean:1.0003266334533691 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7544522285461426 - rollout_corr/rollout_is_min:0.6534404754638672 - rollout_corr/rollout_is_std:0.026394300162792206 - rollout_corr/rollout_is_eff_sample_size:0.9993041830381005 - rollout_corr/rollout_is_seq_mean:1.0003859996795654 - rollout_corr/rollout_is_seq_std:0.0044074165634810925 - rollout_corr/rollout_is_seq_max:1.0315426588058472 - rollout_corr/rollout_is_seq_min:0.9861540198326111 - rollout_corr/rollout_is_seq_max_deviation:0.03154265880584717 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.093177287839353) - rollout_corr/training_log_ppl:np.float64(0.08628692819002026) - rollout_corr/kl:np.float64(2.0917364807360173e-05) - rollout_corr/k3_kl:np.float64(0.0005525162554344265) - rollout_corr/rollout_ppl:np.float64(1.093022387009114) - rollout_corr/rollout_log_ppl:np.float64(0.08626601043215487) - rollout_corr/log_ppl_diff:np.float64(2.091775786539074e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.003551566836904385) - rollout_corr/log_ppl_diff_max:np.float64(0.021391361951828003) - rollout_corr/log_ppl_diff_min:np.float64(-0.023619428277015686) - rollout_corr/ppl_ratio:np.float64(1.0000345066655427) - rollout_corr/chi2_token:np.float64(0.002114285482093692) - rollout_corr/chi2_seq:np.float64(0.10350201418623328) - actor/pg_loss:np.float64(7.6336320489645e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016534391761524603) - actor/ppo_kl:np.float64(6.120576780332954e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.46551600471138954) - perf/mfu/actor:np.float64(0.05773728614515245) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.7386589050293) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5496323529411765) - val-aux/tooluse/reward/std@16:np.float64(0.05755633789461379) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5771029411764705) - val-aux/tooluse/reward/best@2/std:np.float64(0.04356311363344626) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5239264705882354) - val-aux/tooluse/reward/worst@2/std:np.float64(0.0539913882678391) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5508088235294119) - val-aux/tooluse/reward/maj@2/std:np.float64(0.05740345453811504) - val-aux/tooluse/reward/best@4/mean:np.float64(0.5936764705882352) - val-aux/tooluse/reward/best@4/std:np.float64(0.024360798637160536) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5010294117647058) - val-aux/tooluse/reward/worst@4/std:np.float64(0.040826515564984504) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5522941176470589) - val-aux/tooluse/reward/maj@4/std:np.float64(0.05159972446889591) - val-aux/tooluse/reward/best@8/mean:np.float64(0.601264705882353) - val-aux/tooluse/reward/best@8/std:np.float64(0.007976372811305546) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.4865) - val-aux/tooluse/reward/worst@8/std:np.float64(0.024224663795531426) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.55475) - val-aux/tooluse/reward/maj@8/std:np.float64(0.04285407992446189) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6028088235294118) - val-aux/tooluse/reward/best@16/std:np.float64(0.0013888302563556346) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.4780147058823529) - val-aux/tooluse/reward/worst@16/std:np.float64(0.012815658443131544) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5555588235294118) - val-aux/tooluse/reward/maj@16/std:np.float64(0.03514016343124551) - val-aux/tooluse/score/mean@16:np.float64(0.5496323529411765) - val-aux/tooluse/score/std@16:np.float64(0.05755633789461379) - val-aux/tooluse/score/best@2/mean:np.float64(0.5771029411764705) - val-aux/tooluse/score/best@2/std:np.float64(0.04356311363344626) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5239264705882354) - val-aux/tooluse/score/worst@2/std:np.float64(0.0539913882678391) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5508088235294119) - val-aux/tooluse/score/maj@2/std:np.float64(0.05740345453811504) - val-aux/tooluse/score/best@4/mean:np.float64(0.5936764705882352) - val-aux/tooluse/score/best@4/std:np.float64(0.024360798637160536) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5010294117647058) - val-aux/tooluse/score/worst@4/std:np.float64(0.040826515564984504) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5522941176470589) - val-aux/tooluse/score/maj@4/std:np.float64(0.05159972446889591) - val-aux/tooluse/score/best@8/mean:np.float64(0.601264705882353) - val-aux/tooluse/score/best@8/std:np.float64(0.007976372811305546) - val-aux/tooluse/score/worst@8/mean:np.float64(0.4865) - val-aux/tooluse/score/worst@8/std:np.float64(0.024224663795531426) - val-aux/tooluse/score/maj@8/mean:np.float64(0.55475) - val-aux/tooluse/score/maj@8/std:np.float64(0.04285407992446189) - val-aux/tooluse/score/best@16/mean:np.float64(0.6028088235294118) - val-aux/tooluse/score/best@16/std:np.float64(0.0013888302563556346) - val-aux/tooluse/score/worst@16/mean:np.float64(0.4780147058823529) - val-aux/tooluse/score/worst@16/std:np.float64(0.012815658443131544) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5555588235294118) - val-aux/tooluse/score/maj@16/std:np.float64(0.03514016343124551) - val-core/tooluse/acc/mean@16:np.float64(0.5496323529411765) - val-aux/tooluse/acc/std@16:np.float64(0.05755633789461379) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5771029411764705) - val-aux/tooluse/acc/best@2/std:np.float64(0.04356311363344626) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5239264705882354) - val-aux/tooluse/acc/worst@2/std:np.float64(0.0539913882678391) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5508088235294119) - val-aux/tooluse/acc/maj@2/std:np.float64(0.05740345453811504) - val-aux/tooluse/acc/best@4/mean:np.float64(0.5936764705882352) - val-aux/tooluse/acc/best@4/std:np.float64(0.024360798637160536) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5010294117647058) - val-aux/tooluse/acc/worst@4/std:np.float64(0.040826515564984504) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5522941176470589) - val-aux/tooluse/acc/maj@4/std:np.float64(0.05159972446889591) - val-aux/tooluse/acc/best@8/mean:np.float64(0.601264705882353) - val-aux/tooluse/acc/best@8/std:np.float64(0.007976372811305546) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.4865) - val-aux/tooluse/acc/worst@8/std:np.float64(0.024224663795531426) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.55475) - val-aux/tooluse/acc/maj@8/std:np.float64(0.04285407992446189) - val-core/tooluse/acc/best@16/mean:np.float64(0.6028088235294118) - val-core/tooluse/acc/best@16/std:np.float64(0.0013888302563556346) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.4780147058823529) - val-aux/tooluse/acc/worst@16/std:np.float64(0.012815658443131544) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5555588235294118) - val-core/tooluse/acc/maj@16/std:np.float64(0.03514016343124551) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012956895865499973 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:0.012956895865499973 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:49.48046875 - response_length/max:211.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:49.48046875 - response_length_non_aborted/max:211.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:565.59375 - prompt_length/max:837.0 - prompt_length/min:300.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010338053107261658 - timing_s/agent_loop/generate_sequences/min:np.float64(0.33458391577005386) - timing_s/agent_loop/generate_sequences/max:np.float64(1.469787733629346) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6181605459933053) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.469787733629346) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:688 - timing_s/agent_loop/slowest/response_length:211 - timing_s/gen:7.314110724255443 - timing_s/reward:0.06778864376246929 - timing_s/old_log_prob:2.4042498599737883 - timing_s/adv:0.1323879435658455 - timing_s/update_actor:9.814626293256879 - timing_s/step:19.81720365397632 - timing_s/testing:19.376502443104982 - timing_s/save_checkpoint:6.761790065094829 - timing_s/stop_profile:0.00010353513062000275 - timing_per_token_ms/update_actor:0.06233131350546414 - timing_per_token_ms/adv:0.0008407772408426669 - timing_per_token_ms/gen:0.5774145988991428 - perf/total_num_tokens:157459 - perf/time_per_step:19.81720365397632 - perf/throughput:993.1963834892887 (TaskRunner pid=2186821) Training Progress: 70%|███████ | 70/100 [36:35<14:50, 29.67s/it] (TaskRunner pid=2186821) step:71 - global_seqlen/min:13409 - global_seqlen/max:23022 - global_seqlen/minmax_diff:9613 - global_seqlen/balanced_min:18415 - global_seqlen/balanced_max:18416 - global_seqlen/mean:18415.375 - actor/entropy:0.07598459720611572 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1816185712814331 - training/rollout_probs_diff_mean:0.002127055311575532 - training/rollout_probs_diff_std:0.009174765087664127 - training/rollout_actor_probs_pearson_corr:0.9978766441345215 - rollout_corr/rollout_is_mean:1.0004253387451172 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.517364263534546 - rollout_corr/rollout_is_min:0.5686201453208923 - rollout_corr/rollout_is_std:0.023850927129387856 - rollout_corr/rollout_is_eff_sample_size:0.999431813917092 - rollout_corr/rollout_is_seq_mean:1.0004338026046753 - rollout_corr/rollout_is_seq_std:0.0037776771932840347 - rollout_corr/rollout_is_seq_max:1.019316554069519 - rollout_corr/rollout_is_seq_min:0.9883419275283813 - rollout_corr/rollout_is_seq_max_deviation:0.019316554069519043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0782841937616467) - rollout_corr/training_log_ppl:np.float64(0.0727233948268804) - rollout_corr/kl:np.float64(0.0007001423157078079) - rollout_corr/k3_kl:np.float64(0.0006122129584039959) - rollout_corr/rollout_ppl:np.float64(1.0775115815922618) - rollout_corr/rollout_log_ppl:np.float64(0.07202325187677161) - rollout_corr/log_ppl_diff:np.float64(0.0007001429501087841) - rollout_corr/log_ppl_abs_diff:np.float64(0.003077924083072503) - rollout_corr/log_ppl_diff_max:np.float64(0.0801437720656395) - rollout_corr/log_ppl_diff_min:np.float64(-0.019086822867393494) - rollout_corr/ppl_ratio:np.float64(1.0007220127154142) - rollout_corr/chi2_token:np.float64(0.0006950090173631907) - rollout_corr/chi2_seq:np.float64(0.05971434875391424) - actor/pg_loss:np.float64(-0.0004017227329313755) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032617346732877195) - actor/ppo_kl:np.float64(0.0008463677952992299) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22777396067976952) - perf/mfu/actor:np.float64(0.053802508036654435) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.9806900024414) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:0 - critic/score/mean:0.3046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006540523376315832 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006540523376315832 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:48.82421875 - response_length/max:167.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:48.82421875 - response_length_non_aborted/max:167.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:526.65625 - prompt_length/max:919.0 - prompt_length/min:244.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011532381176948547 - timing_s/agent_loop/generate_sequences/min:np.float64(0.2941615004092455) - timing_s/agent_loop/generate_sequences/max:np.float64(1.1457315776497126) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6272039481991669) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.1457315776497126) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:683 - timing_s/agent_loop/slowest/response_length:167 - timing_s/gen:6.983486615121365 - timing_s/reward:0.06309150531888008 - timing_s/old_log_prob:2.5102628357708454 - timing_s/adv:0.14555940963327885 - timing_s/update_actor:9.723819429054856 - timing_s/step:19.520865296944976 - timing_s/stop_profile:0.00011177174746990204 - timing_per_token_ms/update_actor:0.06600340360333998 - timing_per_token_ms/adv:0.0009880290900489324 - timing_per_token_ms/gen:0.5587236270998771 - perf/total_num_tokens:147323 - perf/time_per_step:19.520865296944976 - perf/throughput:943.368786161442 (TaskRunner pid=2186821) Training Progress: 71%|███████ | 71/100 [36:55<12:52, 26.64s/it] (TaskRunner pid=2186821) step:72 - global_seqlen/min:13730 - global_seqlen/max:24731 - global_seqlen/minmax_diff:11001 - global_seqlen/balanced_min:19496 - global_seqlen/balanced_max:19497 - global_seqlen/mean:19496.375 - actor/entropy:0.08788894861936569 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2175697386264801 - training/rollout_probs_diff_mean:0.002134342212229967 - training/rollout_probs_diff_std:0.008528187870979309 - training/rollout_actor_probs_pearson_corr:0.9983265995979309 - rollout_corr/rollout_is_mean:1.0001068115234375 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.4946010112762451 - rollout_corr/rollout_is_min:0.6110516786575317 - rollout_corr/rollout_is_std:0.02497643232345581 - rollout_corr/rollout_is_eff_sample_size:0.9993766857600336 - rollout_corr/rollout_is_seq_mean:1.0001248121261597 - rollout_corr/rollout_is_seq_std:0.003666416509076953 - rollout_corr/rollout_is_seq_max:1.0181126594543457 - rollout_corr/rollout_is_seq_min:0.9863795042037964 - rollout_corr/rollout_is_seq_max_deviation:0.018112659454345703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0840384075418115) - rollout_corr/training_log_ppl:np.float64(0.07686574042963912) - rollout_corr/kl:np.float64(0.001449307150600987) - rollout_corr/k3_kl:np.float64(0.0014701985435344733) - rollout_corr/rollout_ppl:np.float64(1.0819846582598984) - rollout_corr/rollout_log_ppl:np.float64(0.07541643293734523) - rollout_corr/log_ppl_diff:np.float64(0.0014493074922938831) - rollout_corr/log_ppl_abs_diff:np.float64(0.003871039501973428) - rollout_corr/log_ppl_diff_max:np.float64(0.3335646390914917) - rollout_corr/log_ppl_diff_min:np.float64(-0.015902407467365265) - rollout_corr/ppl_ratio:np.float64(1.0017005749978125) - rollout_corr/chi2_token:np.float64(0.0008249098900705576) - rollout_corr/chi2_seq:np.float64(0.024600058794021606) - actor/pg_loss:np.float64(0.00016359053552150726) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003124999930150807) - actor/ppo_kl:np.float64(0.0012726991783864605) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17507495637983084) - perf/mfu/actor:np.float64(0.05781006943102812) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.94901180267334) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:0 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002651009417604655 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:-0.0002651009417604655 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:49.73046875 - response_length/max:222.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:49.73046875 - response_length_non_aborted/max:222.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:559.53125 - prompt_length/max:1219.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010812468826770782 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3474149573594332) - timing_s/agent_loop/generate_sequences/max:np.float64(1.670287849381566) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.647487770569569) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.670287849381566) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:509 - timing_s/agent_loop/slowest/response_length:222 - timing_s/gen:7.3059077095240355 - timing_s/reward:0.06727459281682968 - timing_s/old_log_prob:2.3987455554306507 - timing_s/adv:0.12765621207654476 - timing_s/update_actor:9.73313213326037 - timing_s/step:19.718283392488956 - timing_s/stop_profile:0.00011084787547588348 - timing_per_token_ms/update_actor:0.06240347329478153 - timing_per_token_ms/adv:0.000818461201611484 - timing_per_token_ms/gen:0.573867544538845 - perf/total_num_tokens:155971 - perf/time_per_step:19.718283392488956 - perf/throughput:988.7460592754496 (TaskRunner pid=2186821) Training Progress: 72%|███████▏ | 72/100 [37:14<11:28, 24.58s/it] (TaskRunner pid=2186821) step:73 - global_seqlen/min:16594 - global_seqlen/max:26431 - global_seqlen/minmax_diff:9837 - global_seqlen/balanced_min:20634 - global_seqlen/balanced_max:20636 - global_seqlen/mean:20635.0 - actor/entropy:0.10029152035713196 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24422436952590942 - training/rollout_probs_diff_mean:0.0025965215172618628 - training/rollout_probs_diff_std:0.009902778081595898 - training/rollout_actor_probs_pearson_corr:0.9981914758682251 - rollout_corr/rollout_is_mean:1.0002070665359497 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6127688884735107 - rollout_corr/rollout_is_min:0.5366321206092834 - rollout_corr/rollout_is_std:0.028644677251577377 - rollout_corr/rollout_is_eff_sample_size:0.9991806312129362 - rollout_corr/rollout_is_seq_mean:1.0000232458114624 - rollout_corr/rollout_is_seq_std:0.004117417614907026 - rollout_corr/rollout_is_seq_max:1.0113325119018555 - rollout_corr/rollout_is_seq_min:0.9814764857292175 - rollout_corr/rollout_is_seq_max_deviation:0.01852351427078247 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.111888203304261) - rollout_corr/training_log_ppl:np.float64(0.10169768709329219) - rollout_corr/kl:np.float64(0.0012584341527173137) - rollout_corr/k3_kl:np.float64(0.0010015971820018787) - rollout_corr/rollout_ppl:np.float64(1.1103733917698264) - rollout_corr/rollout_log_ppl:np.float64(0.10043925159288847) - rollout_corr/log_ppl_diff:np.float64(0.0012584355004037207) - rollout_corr/log_ppl_abs_diff:np.float64(0.00436706487516858) - rollout_corr/log_ppl_diff_max:np.float64(0.052030131220817566) - rollout_corr/log_ppl_diff_min:np.float64(-0.027246102690696716) - rollout_corr/ppl_ratio:np.float64(1.0012871085200459) - rollout_corr/chi2_token:np.float64(0.001259434036910534) - rollout_corr/chi2_seq:np.float64(0.1000730162486434) - actor/pg_loss:np.float64(6.404274608939886e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010925927708740346) - actor/ppo_kl:np.float64(0.000891916221860356) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.6096062213182449) - perf/mfu/actor:np.float64(0.0602215301028761) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.81703186035156) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:0 - critic/score/mean:0.33984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0039369272999465466 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0039369272999465466 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:47.75 - response_length/max:126.0 - response_length/min:21.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:47.75 - response_length_non_aborted/max:126.0 - response_length_non_aborted/min:21.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:597.09375 - prompt_length/max:958.0 - prompt_length/min:254.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016604550182819366 - timing_s/agent_loop/generate_sequences/min:np.float64(0.30985935032367706) - timing_s/agent_loop/generate_sequences/max:np.float64(1.0095258690416813) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6258649334413349) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.0095258690416813) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:538 - timing_s/agent_loop/slowest/response_length:126 - timing_s/gen:6.8417515736073256 - timing_s/reward:0.06724212318658829 - timing_s/old_log_prob:2.348432609811425 - timing_s/adv:0.12957127206027508 - timing_s/update_actor:9.955212401226163 - timing_s/step:19.38772290572524 - timing_s/stop_profile:0.00017177686095237732 - timing_per_token_ms/update_actor:0.06030538164057525 - timing_per_token_ms/adv:0.0007848998792117463 - timing_per_token_ms/gen:0.5596982635477197 - perf/total_num_tokens:165080 - perf/time_per_step:19.38772290572524 - perf/throughput:1064.3333464347397 (TaskRunner pid=2186821) Training Progress: 73%|███████▎ | 73/100 [37:34<10:21, 23.04s/it] (TaskRunner pid=2186821) step:74 - global_seqlen/min:15206 - global_seqlen/max:25544 - global_seqlen/minmax_diff:10338 - global_seqlen/balanced_min:21038 - global_seqlen/balanced_max:21039 - global_seqlen/mean:21038.25 - actor/entropy:0.08587076514959335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1548398733139038 - training/rollout_probs_diff_mean:0.0017919654492288828 - training/rollout_probs_diff_std:0.0072895316407084465 - training/rollout_actor_probs_pearson_corr:0.9988563656806946 - rollout_corr/rollout_is_mean:0.9998028874397278 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.362675666809082 - rollout_corr/rollout_is_min:0.5060909390449524 - rollout_corr/rollout_is_std:0.021432988345623016 - rollout_corr/rollout_is_eff_sample_size:0.9995406592796418 - rollout_corr/rollout_is_seq_mean:0.9998031258583069 - rollout_corr/rollout_is_seq_std:0.0029136750381439924 - rollout_corr/rollout_is_seq_max:1.011588215827942 - rollout_corr/rollout_is_seq_min:0.9849720001220703 - rollout_corr/rollout_is_seq_max_deviation:0.015027999877929688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0892230179160833) - rollout_corr/training_log_ppl:np.float64(0.08277007905599021) - rollout_corr/kl:np.float64(0.001339519181669857) - rollout_corr/k3_kl:np.float64(0.0009751809513094678) - rollout_corr/rollout_ppl:np.float64(1.087687789928168) - rollout_corr/rollout_log_ppl:np.float64(0.08143055971868307) - rollout_corr/log_ppl_diff:np.float64(0.001339519337307138) - rollout_corr/log_ppl_abs_diff:np.float64(0.003112707860964292) - rollout_corr/log_ppl_diff_max:np.float64(0.09869062900543213) - rollout_corr/log_ppl_diff_min:np.float64(-0.03295694664120674) - rollout_corr/ppl_ratio:np.float64(1.0013818354345858) - rollout_corr/chi2_token:np.float64(0.0008573848754167557) - rollout_corr/chi2_seq:np.float64(0.021732635563239455) - actor/pg_loss:np.float64(0.00011195079423487186) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006596584862563759) - actor/ppo_kl:np.float64(0.0009261008932259784) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4043646901845932) - perf/mfu/actor:np.float64(0.060771806873964065) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.83388137817383) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008082267013378441 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008082267013378441 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:56.7890625 - response_length/max:221.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:56.7890625 - response_length_non_aborted/max:221.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:600.65625 - prompt_length/max:1001.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012590736150741577 - timing_s/agent_loop/generate_sequences/min:np.float64(0.32374397851526737) - timing_s/agent_loop/generate_sequences/max:np.float64(1.6478642672300339) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6905667082828586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.6478642672300339) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:680 - timing_s/agent_loop/slowest/response_length:221 - timing_s/gen:7.5468333549797535 - timing_s/reward:0.06799736805260181 - timing_s/old_log_prob:2.3390706460922956 - timing_s/adv:0.1624999325722456 - timing_s/update_actor:10.031886177137494 - timing_s/step:20.233424052596092 - timing_s/stop_profile:0.0001115705817937851 - timing_per_token_ms/update_actor:0.059605041870981985 - timing_per_token_ms/adv:0.0009655029088222975 - timing_per_token_ms/gen:0.5191108374590558 - perf/total_num_tokens:168306 - perf/time_per_step:20.233424052596092 - perf/throughput:1039.777051343944 (TaskRunner pid=2186821) Training Progress: 74%|███████▍ | 74/100 [37:54<09:37, 22.21s/it] (TaskRunner pid=2186821) step:75 - global_seqlen/min:17763 - global_seqlen/max:23903 - global_seqlen/minmax_diff:6140 - global_seqlen/balanced_min:21364 - global_seqlen/balanced_max:21365 - global_seqlen/mean:21364.625 - actor/entropy:0.09424465894699097 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28349238634109497 - training/rollout_probs_diff_mean:0.0022053334396332502 - training/rollout_probs_diff_std:0.009037191979587078 - training/rollout_actor_probs_pearson_corr:0.9983837008476257 - rollout_corr/rollout_is_mean:1.0001094341278076 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6245036125183105 - rollout_corr/rollout_is_min:0.6104828715324402 - rollout_corr/rollout_is_std:0.026128750294446945 - rollout_corr/rollout_is_eff_sample_size:0.9993179922693084 - rollout_corr/rollout_is_seq_mean:1.0002636909484863 - rollout_corr/rollout_is_seq_std:0.0038344203494489193 - rollout_corr/rollout_is_seq_max:1.0156431198120117 - rollout_corr/rollout_is_seq_min:0.9888671636581421 - rollout_corr/rollout_is_seq_max_deviation:0.01564311981201172 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0989727941341698) - rollout_corr/training_log_ppl:np.float64(0.09086760537320515) - rollout_corr/kl:np.float64(0.0012431366648200104) - rollout_corr/k3_kl:np.float64(0.0007976194634600193) - rollout_corr/rollout_ppl:np.float64(1.0974521660245955) - rollout_corr/rollout_log_ppl:np.float64(0.08962446804980573) - rollout_corr/log_ppl_diff:np.float64(0.0012431373233994236) - rollout_corr/log_ppl_abs_diff:np.float64(0.003943783563954639) - rollout_corr/log_ppl_diff_max:np.float64(0.042537301778793335) - rollout_corr/log_ppl_diff_min:np.float64(-0.018140852451324463) - rollout_corr/ppl_ratio:np.float64(1.0012638620100915) - rollout_corr/chi2_token:np.float64(0.0004901017528027296) - rollout_corr/chi2_seq:np.float64(0.04358635330572724) - actor/pg_loss:np.float64(-0.00031686725560575724) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005913507084187586) - actor/ppo_kl:np.float64(0.0011816173359875393) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.33969270810484886) - perf/mfu/actor:np.float64(0.06230916106850206) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.67224884033203) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007051126565784216 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007051126565784216 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:64.33203125 - response_length/max:362.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:64.33203125 - response_length_non_aborted/max:362.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:603.3125 - prompt_length/max:972.0 - prompt_length/min:259.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.686127305030823e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.35497486032545567) - timing_s/agent_loop/generate_sequences/max:np.float64(2.223156375810504) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7597961314386339) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.223156375810504) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:660 - timing_s/agent_loop/slowest/response_length:362 - timing_s/gen:7.947652503848076 - timing_s/reward:0.07020808570086956 - timing_s/old_log_prob:2.369071690365672 - timing_s/adv:0.15120256878435612 - timing_s/update_actor:9.995859671384096 - timing_s/step:20.617115888744593 - timing_s/stop_profile:0.00012504681944847107 - timing_per_token_ms/update_actor:0.05848370654401901 - timing_per_token_ms/adv:0.000884654942365921 - timing_per_token_ms/gen:0.48258257962523987 - perf/total_num_tokens:170917 - perf/time_per_step:20.617115888744593 - perf/throughput:1036.2567254939615 (TaskRunner pid=2186821) Training Progress: 75%|███████▌ | 75/100 [38:15<09:03, 21.74s/it] (TaskRunner pid=2186821) step:76 - global_seqlen/min:15894 - global_seqlen/max:27968 - global_seqlen/minmax_diff:12074 - global_seqlen/balanced_min:20871 - global_seqlen/balanced_max:20873 - global_seqlen/mean:20872.0 - actor/entropy:0.09563218057155609 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2428145408630371 - training/rollout_probs_diff_mean:0.002500321017578244 - training/rollout_probs_diff_std:0.009985673241317272 - training/rollout_actor_probs_pearson_corr:0.9980630874633789 - rollout_corr/rollout_is_mean:1.0002797842025757 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6452488899230957 - rollout_corr/rollout_is_min:0.5504816770553589 - rollout_corr/rollout_is_std:0.028827209025621414 - rollout_corr/rollout_is_eff_sample_size:0.9991701580892315 - rollout_corr/rollout_is_seq_mean:1.000307559967041 - rollout_corr/rollout_is_seq_std:0.0038258926942944527 - rollout_corr/rollout_is_seq_max:1.0182652473449707 - rollout_corr/rollout_is_seq_min:0.9867900609970093 - rollout_corr/rollout_is_seq_max_deviation:0.018265247344970703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1047835033386946) - rollout_corr/training_log_ppl:np.float64(0.09592312493350619) - rollout_corr/kl:np.float64(0.0007892963960358657) - rollout_corr/k3_kl:np.float64(0.0007510980486694803) - rollout_corr/rollout_ppl:np.float64(1.1038377983495593) - rollout_corr/rollout_log_ppl:np.float64(0.09513382830664341) - rollout_corr/log_ppl_diff:np.float64(0.0007892966268627788) - rollout_corr/log_ppl_abs_diff:np.float64(0.003325715279970609) - rollout_corr/log_ppl_diff_max:np.float64(0.02627423405647278) - rollout_corr/log_ppl_diff_min:np.float64(-0.01872362196445465) - rollout_corr/ppl_ratio:np.float64(1.0008032659534365) - rollout_corr/chi2_token:np.float64(0.0013040690682828426) - rollout_corr/chi2_seq:np.float64(0.08589521795511246) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0006820129431659439) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.06129451581277368) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.69346618652344) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:0 - critic/score/mean:0.25 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:65.75 - response_length/max:218.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:65.75 - response_length_non_aborted/max:218.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:586.5 - prompt_length/max:911.0 - prompt_length/min:244.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015643984079360962 - timing_s/agent_loop/generate_sequences/min:np.float64(0.2203538529574871) - timing_s/agent_loop/generate_sequences/max:np.float64(1.6224692575633526) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7573042559452006) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.6224692575633526) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:772 - timing_s/agent_loop/slowest/response_length:218 - timing_s/gen:7.584711737930775 - timing_s/reward:0.0697365403175354 - timing_s/old_log_prob:2.4581129793077707 - timing_s/adv:0.15203558094799519 - timing_s/update_actor:9.899470403790474 - timing_s/step:20.248617127537727 - timing_s/stop_profile:0.00011800229549407959 - timing_per_token_ms/update_actor:0.0592867861476528 - timing_per_token_ms/adv:0.0009105235539717994 - timing_per_token_ms/gen:0.45061262701584925 - perf/total_num_tokens:166976 - perf/time_per_step:20.248617127537727 - perf/throughput:1030.7864417869052 (TaskRunner pid=2186821) Training Progress: 76%|███████▌ | 76/100 [38:35<08:31, 21.32s/it] (TaskRunner pid=2186821) step:77 - global_seqlen/min:16741 - global_seqlen/max:27058 - global_seqlen/minmax_diff:10317 - global_seqlen/balanced_min:22037 - global_seqlen/balanced_max:22041 - global_seqlen/mean:22038.875 - actor/entropy:0.09047998487949371 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2712806463241577 - training/rollout_probs_diff_mean:0.002367057604715228 - training/rollout_probs_diff_std:0.009614572860300541 - training/rollout_actor_probs_pearson_corr:0.9980126619338989 - rollout_corr/rollout_is_mean:0.9998871684074402 - rollout_corr/rollout_is_ratio_fraction_high:6.801333074690774e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.1073689460754395 - rollout_corr/rollout_is_min:0.6189932227134705 - rollout_corr/rollout_is_std:0.027166610583662987 - rollout_corr/rollout_is_eff_sample_size:0.9992622815001809 - rollout_corr/rollout_is_seq_mean:0.9998594522476196 - rollout_corr/rollout_is_seq_std:0.0037069327663630247 - rollout_corr/rollout_is_seq_max:1.0115433931350708 - rollout_corr/rollout_is_seq_min:0.9865688681602478 - rollout_corr/rollout_is_seq_max_deviation:0.013431131839752197 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0869557205587626) - rollout_corr/training_log_ppl:np.float64(0.08091833497701373) - rollout_corr/kl:np.float64(0.0009423998322053428) - rollout_corr/k3_kl:np.float64(0.0005074112005607617) - rollout_corr/rollout_ppl:np.float64(1.0859127631410956) - rollout_corr/rollout_log_ppl:np.float64(0.07997593424943261) - rollout_corr/log_ppl_diff:np.float64(0.0009424007275811164) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029916646772107924) - rollout_corr/log_ppl_diff_max:np.float64(0.02207048237323761) - rollout_corr/log_ppl_diff_min:np.float64(-0.012270539999008179) - rollout_corr/ppl_ratio:np.float64(1.0009525569621474) - rollout_corr/chi2_token:np.float64(7.578637450933456e-05) - rollout_corr/chi2_seq:np.float64(0.028896737843751907) - actor/pg_loss:np.float64(5.2629504352808e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011549270348041318) - actor/ppo_kl:np.float64(0.00047376988089808947) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5290998741984367) - perf/mfu/actor:np.float64(0.06473549993612905) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.73304462432861) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:0 - critic/score/mean:0.30078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007821532897651196 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007821532897651196 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:57.43359375 - response_length/max:280.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:57.43359375 - response_length_non_aborted/max:280.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:631.28125 - prompt_length/max:1518.0 - prompt_length/min:340.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.976141154766083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3634605035185814) - timing_s/agent_loop/generate_sequences/max:np.float64(1.8110489174723625) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7062395122848102) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.8110489174723625) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:690 - timing_s/agent_loop/slowest/response_length:280 - timing_s/gen:7.904487172141671 - timing_s/reward:0.07453422620892525 - timing_s/old_log_prob:2.3450540006160736 - timing_s/adv:0.10765969008207321 - timing_s/update_actor:9.802180843427777 - timing_s/step:20.31804470717907 - timing_s/stop_profile:0.00011210888624191284 - timing_per_token_ms/update_actor:0.055595968733815684 - timing_per_token_ms/adv:0.0006106237845742648 - timing_per_token_ms/gen:0.5376104993635089 - perf/total_num_tokens:176311 - perf/time_per_step:20.31804470717907 - perf/throughput:1084.694679907506 (TaskRunner pid=2186821) Training Progress: 77%|███████▋ | 77/100 [38:56<08:03, 21.03s/it] (TaskRunner pid=2186821) step:78 - global_seqlen/min:15229 - global_seqlen/max:27646 - global_seqlen/minmax_diff:12417 - global_seqlen/balanced_min:20330 - global_seqlen/balanced_max:20337 - global_seqlen/mean:20333.5 - actor/entropy:0.09211444109678268 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25539231300354004 - training/rollout_probs_diff_mean:0.002834741724655032 - training/rollout_probs_diff_std:0.010969211347401142 - training/rollout_actor_probs_pearson_corr:0.9975700974464417 - rollout_corr/rollout_is_mean:1.000118374824524 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0001239771954715252 - rollout_corr/rollout_is_max:1.7932195663452148 - rollout_corr/rollout_is_min:0.28206202387809753 - rollout_corr/rollout_is_std:0.030543934553861618 - rollout_corr/rollout_is_eff_sample_size:0.999068294612318 - rollout_corr/rollout_is_seq_mean:0.9999359250068665 - rollout_corr/rollout_is_seq_std:0.004063135012984276 - rollout_corr/rollout_is_seq_max:1.0116618871688843 - rollout_corr/rollout_is_seq_min:0.9871120452880859 - rollout_corr/rollout_is_seq_max_deviation:0.012887954711914062 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0924286842346191) - rollout_corr/training_log_ppl:np.float64(0.08598786131642555) - rollout_corr/kl:np.float64(0.0013388191907868219) - rollout_corr/k3_kl:np.float64(0.0014105520158788493) - rollout_corr/rollout_ppl:np.float64(1.0909363836981356) - rollout_corr/rollout_log_ppl:np.float64(0.08464904180709709) - rollout_corr/log_ppl_diff:np.float64(0.0013388195093284594) - rollout_corr/log_ppl_abs_diff:np.float64(0.004615726518750307) - rollout_corr/log_ppl_diff_max:np.float64(0.1926252543926239) - rollout_corr/log_ppl_diff_min:np.float64(-0.014810949563980103) - rollout_corr/ppl_ratio:np.float64(1.0014323338400573) - rollout_corr/chi2_token:np.float64(0.0025046388618648052) - rollout_corr/chi2_seq:np.float64(0.40550557360984385) - actor/pg_loss:np.float64(-0.0005510339979082346) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009495893318671733) - actor/ppo_kl:np.float64(0.0008352402429423478) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4624008312821388) - perf/mfu/actor:np.float64(0.06025145373132497) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.88408279418945) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005850173532962799 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.005850173532962799 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:63.015625 - response_length/max:186.0 - response_length/min:19.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:63.015625 - response_length_non_aborted/max:186.0 - response_length_non_aborted/min:19.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:572.40625 - prompt_length/max:1522.0 - prompt_length/min:271.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011169351637363434 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3336565066128969) - timing_s/agent_loop/generate_sequences/max:np.float64(1.449223866686225) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7560998645349173) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.449223866686225) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1522 - timing_s/agent_loop/slowest/response_length:161 - timing_s/gen:7.1584425289183855 - timing_s/reward:0.06962324865162373 - timing_s/old_log_prob:2.3300754353404045 - timing_s/adv:0.1247237641364336 - timing_s/update_actor:9.814557055011392 - timing_s/step:19.58255380950868 - timing_s/stop_profile:2.9744580388069153e-05 - timing_per_token_ms/update_actor:0.06033489718328984 - timing_per_token_ms/adv:0.0007667381669193301 - timing_per_token_ms/gen:0.443741788303892 - perf/total_num_tokens:162668 - perf/time_per_step:19.58255380950868 - perf/throughput:1038.3477148995082 (TaskRunner pid=2186821) Training Progress: 78%|███████▊ | 78/100 [39:15<07:33, 20.61s/it] (TaskRunner pid=2186821) step:79 - global_seqlen/min:19237 - global_seqlen/max:26306 - global_seqlen/minmax_diff:7069 - global_seqlen/balanced_min:22631 - global_seqlen/balanced_max:22633 - global_seqlen/mean:22632.5 - actor/entropy:0.08841986209154129 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917463183403015 - training/rollout_probs_diff_mean:0.002064367989078164 - training/rollout_probs_diff_std:0.008159906603395939 - training/rollout_actor_probs_pearson_corr:0.9985878467559814 - rollout_corr/rollout_is_mean:1.0001155138015747 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9583549499511719 - rollout_corr/rollout_is_min:0.5474395155906677 - rollout_corr/rollout_is_std:0.026558639481663704 - rollout_corr/rollout_is_eff_sample_size:0.9992953739003395 - rollout_corr/rollout_is_seq_mean:1.0000557899475098 - rollout_corr/rollout_is_seq_std:0.003652038751170039 - rollout_corr/rollout_is_seq_max:1.015695571899414 - rollout_corr/rollout_is_seq_min:0.982600212097168 - rollout_corr/rollout_is_seq_max_deviation:0.01739978790283203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0943387090228498) - rollout_corr/training_log_ppl:np.float64(0.08645597107533831) - rollout_corr/kl:np.float64(0.0005531878948588087) - rollout_corr/k3_kl:np.float64(0.0005682264083631816) - rollout_corr/rollout_ppl:np.float64(1.0938276601955295) - rollout_corr/rollout_log_ppl:np.float64(0.08590278181873146) - rollout_corr/log_ppl_diff:np.float64(0.0005531892566068564) - rollout_corr/log_ppl_abs_diff:np.float64(0.002929982034402201) - rollout_corr/log_ppl_diff_max:np.float64(0.022015169262886047) - rollout_corr/log_ppl_diff_min:np.float64(-0.03195255994796753) - rollout_corr/ppl_ratio:np.float64(1.0005638275761157) - rollout_corr/chi2_token:np.float64(0.001123278634622693) - rollout_corr/chi2_seq:np.float64(0.2852943616453558) - actor/pg_loss:np.float64(-8.514593355357647e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031819743526284583) - actor/ppo_kl:np.float64(0.0002573904631617552) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08233014866709709) - perf/mfu/actor:np.float64(0.06633609144533846) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.6417646408081) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006074009579606354 - critic/advantages/max:0.75 - critic/advantages/min:-0.25 - critic/returns/mean:0.0006074009579606354 - critic/returns/max:0.75 - critic/returns/min:-0.25 - response_length/mean:62.703125 - response_length/max:472.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:62.703125 - response_length_non_aborted/max:472.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:644.5625 - prompt_length/max:1100.0 - prompt_length/min:386.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7393922209739685e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.2934191357344389) - timing_s/agent_loop/generate_sequences/max:np.float64(2.760852999985218) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6941356540701236) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.760852999985218) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:602 - timing_s/agent_loop/slowest/response_length:472 - timing_s/gen:8.772417327389121 - timing_s/reward:0.06808440387248993 - timing_s/old_log_prob:2.3338895328342915 - timing_s/adv:0.12887953221797943 - timing_s/update_actor:9.745721759274602 - timing_s/step:21.132244227454066 - timing_s/stop_profile:5.026906728744507e-05 - timing_per_token_ms/update_actor:0.05382592377816526 - timing_per_token_ms/adv:0.0007118056567876915 - timing_per_token_ms/gen:0.5464999580980016 - perf/total_num_tokens:181060 - perf/time_per_step:21.132244227454066 - perf/throughput:1070.9936794406754 (TaskRunner pid=2186821) Training Progress: 79%|███████▉ | 79/100 [39:36<07:16, 20.77s/it] (TaskRunner pid=2186821) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2186821) validation generation end (TaskRunner pid=2186821) [prompt] user (TaskRunner pid=2186821) Your task is to answer the user's question using available tools. (TaskRunner pid=2186821) You have access to the following tools: (TaskRunner pid=2186821) Name: Axolotl (TaskRunner pid=2186821) Description: Collection of axolotl pictures and facts (TaskRunner pid=2186821) Documentation: (TaskRunner pid=2186821) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2186821) Parameters: {} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{url, source, description} (TaskRunner pid=2186821) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2186821) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2186821) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2186821) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Array[Object{fact, source}] (TaskRunner pid=2186821) (TaskRunner pid=2186821) Use the following format: (TaskRunner pid=2186821) Thought: you should always think about what to do (TaskRunner pid=2186821) Action: the action to take, should be one of the tool names. (TaskRunner pid=2186821) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2186821) (TaskRunner pid=2186821) Begin! (TaskRunner pid=2186821) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2186821) assistant (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: I need to retrieve a random axolotl image. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (TaskRunner pid=2186821) step:80 - global_seqlen/min:18406 - global_seqlen/max:22706 - global_seqlen/minmax_diff:4300 - global_seqlen/balanced_min:20600 - global_seqlen/balanced_max:20601 - global_seqlen/mean:20600.125 - actor/entropy:0.12976038455963135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18435516953468323 - training/rollout_probs_diff_mean:0.002813642146065831 - training/rollout_probs_diff_std:0.009766236878931522 - training/rollout_actor_probs_pearson_corr:0.9985457062721252 - rollout_corr/rollout_is_mean:0.9999985694885254 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5287047624588013 - rollout_corr/rollout_is_min:0.6208714842796326 - rollout_corr/rollout_is_std:0.02707870677113533 - rollout_corr/rollout_is_eff_sample_size:0.999267280930876 - rollout_corr/rollout_is_seq_mean:0.9998317956924438 - rollout_corr/rollout_is_seq_std:0.0038929362781345844 - rollout_corr/rollout_is_seq_max:1.0152685642242432 - rollout_corr/rollout_is_seq_min:0.9854781031608582 - rollout_corr/rollout_is_seq_max_deviation:0.015268564224243164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1238618027418852) - rollout_corr/training_log_ppl:np.float64(0.11240091156651033) - rollout_corr/kl:np.float64(0.0006160001859236708) - rollout_corr/k3_kl:np.float64(0.000960965013966808) - rollout_corr/rollout_ppl:np.float64(1.123094170819968) - rollout_corr/rollout_log_ppl:np.float64(0.11178491113969358) - rollout_corr/log_ppl_diff:np.float64(0.000616000426816754) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036759572685696185) - rollout_corr/log_ppl_diff_max:np.float64(0.0396907702088356) - rollout_corr/log_ppl_diff_min:np.float64(-0.044073186814785004) - rollout_corr/ppl_ratio:np.float64(1.0006406886968762) - rollout_corr/chi2_token:np.float64(0.003454043762758374) - rollout_corr/chi2_seq:np.float64(0.1775191102642566) - actor/pg_loss:np.float64(-0.0003176091704517603) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006510417006211355) - actor/ppo_kl:np.float64(8.020458579416356e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20610717311501503) - perf/mfu/actor:np.float64(0.060404280826569484) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.66295623779297) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5597426470588235) - val-aux/tooluse/reward/std@16:np.float64(0.06027308069341465) - val-aux/tooluse/reward/best@2/mean:np.float64(0.58325) - val-aux/tooluse/reward/best@2/std:np.float64(0.05767316773843347) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5348088235294117) - val-aux/tooluse/reward/worst@2/std:np.float64(0.04252208837282093) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5584705882352942) - val-aux/tooluse/reward/maj@2/std:np.float64(0.05945960492079213) - val-aux/tooluse/reward/best@4/mean:np.float64(0.60625) - val-aux/tooluse/reward/best@4/std:np.float64(0.050050801921357184) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5196470588235295) - val-aux/tooluse/reward/worst@4/std:np.float64(0.027303020755291393) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5566176470588236) - val-aux/tooluse/reward/maj@4/std:np.float64(0.047827757116740637) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6264411764705882) - val-aux/tooluse/reward/best@8/std:np.float64(0.03883579024249038) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5103676470588235) - val-aux/tooluse/reward/worst@8/std:np.float64(0.014080194938761274) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5556764705882353) - val-aux/tooluse/reward/maj@8/std:np.float64(0.03705070094690515) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6432058823529412) - val-aux/tooluse/reward/best@16/std:np.float64(0.029778395317154094) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5053970588235294) - val-aux/tooluse/reward/worst@16/std:np.float64(0.008638214461898126) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5546029411764706) - val-aux/tooluse/reward/maj@16/std:np.float64(0.02922536679396373) - val-aux/tooluse/score/mean@16:np.float64(0.5597426470588235) - val-aux/tooluse/score/std@16:np.float64(0.06027308069341465) - val-aux/tooluse/score/best@2/mean:np.float64(0.58325) - val-aux/tooluse/score/best@2/std:np.float64(0.05767316773843347) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5348088235294117) - val-aux/tooluse/score/worst@2/std:np.float64(0.04252208837282093) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5584705882352942) - val-aux/tooluse/score/maj@2/std:np.float64(0.05945960492079213) - val-aux/tooluse/score/best@4/mean:np.float64(0.60625) - val-aux/tooluse/score/best@4/std:np.float64(0.050050801921357184) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5196470588235295) - val-aux/tooluse/score/worst@4/std:np.float64(0.027303020755291393) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5566176470588236) - val-aux/tooluse/score/maj@4/std:np.float64(0.047827757116740637) - val-aux/tooluse/score/best@8/mean:np.float64(0.6264411764705882) - val-aux/tooluse/score/best@8/std:np.float64(0.03883579024249038) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5103676470588235) - val-aux/tooluse/score/worst@8/std:np.float64(0.014080194938761274) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5556764705882353) - val-aux/tooluse/score/maj@8/std:np.float64(0.03705070094690515) - val-aux/tooluse/score/best@16/mean:np.float64(0.6432058823529412) - val-aux/tooluse/score/best@16/std:np.float64(0.029778395317154094) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5053970588235294) - val-aux/tooluse/score/worst@16/std:np.float64(0.008638214461898126) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5546029411764706) - val-aux/tooluse/score/maj@16/std:np.float64(0.02922536679396373) - val-core/tooluse/acc/mean@16:np.float64(0.5597426470588235) - val-aux/tooluse/acc/std@16:np.float64(0.06027308069341465) - val-aux/tooluse/acc/best@2/mean:np.float64(0.58325) - val-aux/tooluse/acc/best@2/std:np.float64(0.05767316773843347) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5348088235294117) - val-aux/tooluse/acc/worst@2/std:np.float64(0.04252208837282093) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5584705882352942) - val-aux/tooluse/acc/maj@2/std:np.float64(0.05945960492079213) - val-aux/tooluse/acc/best@4/mean:np.float64(0.60625) - val-aux/tooluse/acc/best@4/std:np.float64(0.050050801921357184) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5196470588235295) - val-aux/tooluse/acc/worst@4/std:np.float64(0.027303020755291393) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5566176470588236) - val-aux/tooluse/acc/maj@4/std:np.float64(0.047827757116740637) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6264411764705882) - val-aux/tooluse/acc/best@8/std:np.float64(0.03883579024249038) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5103676470588235) - val-aux/tooluse/acc/worst@8/std:np.float64(0.014080194938761274) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5556764705882353) - val-aux/tooluse/acc/maj@8/std:np.float64(0.03705070094690515) - val-core/tooluse/acc/best@16/mean:np.float64(0.6432058823529412) - val-core/tooluse/acc/best@16/std:np.float64(0.029778395317154094) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5053970588235294) - val-aux/tooluse/acc/worst@16/std:np.float64(0.008638214461898126) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5546029411764706) - val-core/tooluse/acc/maj@16/std:np.float64(0.02922536679396373) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:0 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004508312267716974 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004508312267716974 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:69.31640625 - response_length/max:365.0 - response_length/min:21.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:69.31640625 - response_length_non_aborted/max:365.0 - response_length_non_aborted/min:21.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:574.4375 - prompt_length/max:921.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.6212226152420044e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3052533492445946) - timing_s/agent_loop/generate_sequences/max:np.float64(2.2596486750990152) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7736131100464263) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.2596486750990152) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:666 - timing_s/agent_loop/slowest/response_length:365 - timing_s/gen:8.555581303313375 - timing_s/reward:0.06965361349284649 - timing_s/old_log_prob:2.5024388916790485 - timing_s/adv:0.1247616782784462 - timing_s/update_actor:9.808789655566216 - timing_s/step:21.147849468514323 - timing_s/testing:19.541524754837155 - timing_s/save_checkpoint:6.565726824104786 - timing_s/stop_profile:0.0002165045589208603 - timing_per_token_ms/update_actor:0.05951899354716425 - timing_per_token_ms/adv:0.0007570444249637211 - timing_per_token_ms/gen:0.48214039466403913 - perf/total_num_tokens:164801 - perf/time_per_step:21.147849468514323 - perf/throughput:974.1002285206448 (TaskRunner pid=2186821) Training Progress: 80%|████████ | 80/100 [40:24<09:34, 28.73s/it] (WorkerDict pid=2187230) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2186821) step:81 - global_seqlen/min:15709 - global_seqlen/max:24214 - global_seqlen/minmax_diff:8505 - global_seqlen/balanced_min:19546 - global_seqlen/balanced_max:19562 - global_seqlen/mean:19548.875 - actor/entropy:0.12183531373739243 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413543164730072 - training/rollout_probs_diff_mean:0.003104869043454528 - training/rollout_probs_diff_std:0.011046580038964748 - training/rollout_actor_probs_pearson_corr:0.9980646371841431 - rollout_corr/rollout_is_mean:0.9999661445617676 - rollout_corr/rollout_is_ratio_fraction_high:5.9527352277655154e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.9527352277655154e-05 - rollout_corr/rollout_is_max:2.3468480110168457 - rollout_corr/rollout_is_min:0.48356470465660095 - rollout_corr/rollout_is_std:0.03255961090326309 - rollout_corr/rollout_is_eff_sample_size:0.9989407565558476 - rollout_corr/rollout_is_seq_mean:1.0000340938568115 - rollout_corr/rollout_is_seq_std:0.0044234455563127995 - rollout_corr/rollout_is_seq_max:1.0211315155029297 - rollout_corr/rollout_is_seq_min:0.9853703379631042 - rollout_corr/rollout_is_seq_max_deviation:0.021131515502929688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.114231202751398) - rollout_corr/training_log_ppl:np.float64(0.10455811901556444) - rollout_corr/kl:np.float64(0.00035691297584072856) - rollout_corr/k3_kl:np.float64(0.0005542375498328589) - rollout_corr/rollout_ppl:np.float64(1.1137656113132834) - rollout_corr/rollout_log_ppl:np.float64(0.10420120506023522) - rollout_corr/log_ppl_diff:np.float64(0.0003569139553292189) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032176758868445177) - rollout_corr/log_ppl_diff_max:np.float64(0.020464837551116943) - rollout_corr/log_ppl_diff_min:np.float64(-0.016885340213775635) - rollout_corr/ppl_ratio:np.float64(1.0003678859211504) - rollout_corr/chi2_token:np.float64(0.0015489794313907623) - rollout_corr/chi2_seq:np.float64(0.10837135906331241) - actor/pg_loss:np.float64(-0.00018352107144892216) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024383243362535723) - actor/ppo_kl:np.float64(-5.674431687019421e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15610348992049694) - perf/mfu/actor:np.float64(0.057665724406500404) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.97675323486328) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:0 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003735341364517808 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003735341364517808 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:65.62109375 - response_length/max:466.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:65.62109375 - response_length_non_aborted/max:466.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:545.28125 - prompt_length/max:785.0 - prompt_length/min:308.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017671845853328705 - timing_s/agent_loop/generate_sequences/min:np.float64(0.2570110373198986) - timing_s/agent_loop/generate_sequences/max:np.float64(2.7513653449714184) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6718395547577529) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.7513653449714184) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:332 - timing_s/agent_loop/slowest/response_length:466 - timing_s/gen:8.28086070343852 - timing_s/reward:0.06363525055348873 - timing_s/old_log_prob:2.385941695421934 - timing_s/adv:0.17470956407487392 - timing_s/update_actor:9.8357517067343 - timing_s/step:20.834580784663558 - timing_s/stop_profile:0.00011775083839893341 - timing_per_token_ms/update_actor:0.06289205713074474 - timing_per_token_ms/adv:0.0011171331091614858 - timing_per_token_ms/gen:0.4929377167354319 - perf/total_num_tokens:156391 - perf/time_per_step:20.834580784663558 - perf/throughput:938.2898173977193 (TaskRunner pid=2186821) Training Progress: 81%|████████ | 81/100 [40:45<08:21, 26.38s/it] (TaskRunner pid=2186821) step:82 - global_seqlen/min:14319 - global_seqlen/max:29122 - global_seqlen/minmax_diff:14803 - global_seqlen/balanced_min:20471 - global_seqlen/balanced_max:20473 - global_seqlen/mean:20472.0 - actor/entropy:0.14132733643054962 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15450960397720337 - training/rollout_probs_diff_mean:0.00274678529240191 - training/rollout_probs_diff_std:0.009343286976218224 - training/rollout_actor_probs_pearson_corr:0.9987658858299255 - rollout_corr/rollout_is_mean:1.000356674194336 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.4836255311965942 - rollout_corr/rollout_is_min:0.6087498068809509 - rollout_corr/rollout_is_std:0.027225784957408905 - rollout_corr/rollout_is_eff_sample_size:0.9992597818035928 - rollout_corr/rollout_is_seq_mean:1.0003483295440674 - rollout_corr/rollout_is_seq_std:0.0035101997200399637 - rollout_corr/rollout_is_seq_max:1.0170772075653076 - rollout_corr/rollout_is_seq_min:0.9881327748298645 - rollout_corr/rollout_is_seq_max_deviation:0.017077207565307617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.127422840334475) - rollout_corr/training_log_ppl:np.float64(0.11179355848798878) - rollout_corr/kl:np.float64(0.0003650335591132148) - rollout_corr/k3_kl:np.float64(0.0006114917770651471) - rollout_corr/rollout_ppl:np.float64(1.1270018997602165) - rollout_corr/rollout_log_ppl:np.float64(0.11142852477769338) - rollout_corr/log_ppl_diff:np.float64(0.00036503371029539267) - rollout_corr/log_ppl_abs_diff:np.float64(0.003178495336214837) - rollout_corr/log_ppl_diff_max:np.float64(0.026302799582481384) - rollout_corr/log_ppl_diff_min:np.float64(-0.016005054116249084) - rollout_corr/ppl_ratio:np.float64(1.0003765302244574) - rollout_corr/chi2_token:np.float64(0.001770994160324335) - rollout_corr/chi2_seq:np.float64(0.11267242371104658) - actor/pg_loss:np.float64(0.00021281628869473934) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023436875562765636) - actor/ppo_kl:np.float64(0.00039072491987646174) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20249015092849731) - perf/mfu/actor:np.float64(0.061011891003283095) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.89033126831055) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0013274053344503045 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:0.0013274053344503045 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:61.0625 - response_length/max:328.0 - response_length/min:20.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:61.0625 - response_length_non_aborted/max:328.0 - response_length_non_aborted/min:20.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:578.6875 - prompt_length/max:1283.0 - prompt_length/min:220.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.892508387565613e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.2626740951091051) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7200956121087074) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6172774978549569) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7200956121087074) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:450 - timing_s/agent_loop/slowest/response_length:328 - timing_s/gen:7.608151499181986 - timing_s/reward:0.06692511402070522 - timing_s/old_log_prob:2.3546895757317543 - timing_s/adv:0.1626171823590994 - timing_s/update_actor:9.743294956162572 - timing_s/step:20.014716116711497 - timing_s/stop_profile:2.7166679501533508e-05 - timing_per_token_ms/update_actor:0.05949159190701062 - timing_per_token_ms/adv:0.000992924374506029 - timing_per_token_ms/gen:0.48670365271123245 - perf/total_num_tokens:163776 - perf/time_per_step:20.014716116711497 - perf/throughput:1022.84738292674 (TaskRunner pid=2186821) Training Progress: 82%|████████▏ | 82/100 [41:05<07:20, 24.47s/it] (TaskRunner pid=2186821) step:83 - global_seqlen/min:15723 - global_seqlen/max:28474 - global_seqlen/minmax_diff:12751 - global_seqlen/balanced_min:19478 - global_seqlen/balanced_max:19481 - global_seqlen/mean:19479.875 - actor/entropy:0.12773916125297546 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846359670162201 - training/rollout_probs_diff_mean:0.0031572936568409204 - training/rollout_probs_diff_std:0.011177166365087032 - training/rollout_actor_probs_pearson_corr:0.9981591105461121 - rollout_corr/rollout_is_mean:0.999937891960144 - rollout_corr/rollout_is_ratio_fraction_high:5.465974390972406e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.6873650550842285 - rollout_corr/rollout_is_min:0.5365312099456787 - rollout_corr/rollout_is_std:0.0337815098464489 - rollout_corr/rollout_is_eff_sample_size:0.9988598724271667 - rollout_corr/rollout_is_seq_mean:0.9997518062591553 - rollout_corr/rollout_is_seq_std:0.004658383317291737 - rollout_corr/rollout_is_seq_max:1.0205543041229248 - rollout_corr/rollout_is_seq_min:0.984998881816864 - rollout_corr/rollout_is_seq_max_deviation:0.020554304122924805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1387000745162368) - rollout_corr/training_log_ppl:np.float64(0.123117197486863) - rollout_corr/kl:np.float64(0.0007611578072452829) - rollout_corr/k3_kl:np.float64(0.0006818295798380891) - rollout_corr/rollout_ppl:np.float64(1.1376577280461788) - rollout_corr/rollout_log_ppl:np.float64(0.12235603810586326) - rollout_corr/log_ppl_diff:np.float64(0.000761159380999743) - rollout_corr/log_ppl_abs_diff:np.float64(0.003798962627115543) - rollout_corr/log_ppl_diff_max:np.float64(0.02187219262123108) - rollout_corr/log_ppl_diff_min:np.float64(-0.018547192215919495) - rollout_corr/ppl_ratio:np.float64(1.0007750277873129) - rollout_corr/chi2_token:np.float64(0.0012700711376965046) - rollout_corr/chi2_seq:np.float64(0.059739051619544625) - actor/pg_loss:np.float64(-0.00010679836850613356) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.867574196192436e-05) - actor/ppo_kl:np.float64(-5.442824385681888e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1069959169253707) - perf/mfu/actor:np.float64(0.05553081442074319) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.74485397338867) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:0 - critic/score/mean:0.25390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001421153312548995 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:-0.001421153312548995 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:71.46484375 - response_length/max:354.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:71.46484375 - response_length_non_aborted/max:354.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:537.28125 - prompt_length/max:1090.0 - prompt_length/min:269.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.565048635005951e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.26000942289829254) - timing_s/agent_loop/generate_sequences/max:np.float64(1.9914382919669151) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7026713288360042) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.9914382919669151) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:537 - timing_s/agent_loop/slowest/response_length:286 - timing_s/gen:7.084146520122886 - timing_s/reward:0.06496298499405384 - timing_s/old_log_prob:2.3972960226237774 - timing_s/adv:0.12237579561769962 - timing_s/update_actor:10.069500276818871 - timing_s/step:19.813385935500264 - timing_s/stop_profile:0.00012416020035743713 - timing_per_token_ms/update_actor:0.06461476444804491 - timing_per_token_ms/adv:0.0007852706679181696 - timing_per_token_ms/gen:0.38721762886706124 - perf/total_num_tokens:155839 - perf/time_per_step:19.813385935500264 - perf/throughput:983.167393166117 (TaskRunner pid=2186821) Training Progress: 83%|████████▎ | 83/100 [41:24<06:32, 23.09s/it] (TaskRunner pid=2186821) step:84 - global_seqlen/min:16704 - global_seqlen/max:25951 - global_seqlen/minmax_diff:9247 - global_seqlen/balanced_min:21022 - global_seqlen/balanced_max:21024 - global_seqlen/mean:21023.25 - actor/entropy:0.09461385011672974 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15355414152145386 - training/rollout_probs_diff_mean:0.0021508035715669394 - training/rollout_probs_diff_std:0.008979055099189281 - training/rollout_actor_probs_pearson_corr:0.9984091520309448 - rollout_corr/rollout_is_mean:0.9999325275421143 - rollout_corr/rollout_is_ratio_fraction_high:6.416014366550371e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2561583518981934 - rollout_corr/rollout_is_min:0.6136392951011658 - rollout_corr/rollout_is_std:0.025258822366595268 - rollout_corr/rollout_is_eff_sample_size:0.999362398677144 - rollout_corr/rollout_is_seq_mean:1.000151515007019 - rollout_corr/rollout_is_seq_std:0.0034036580473184586 - rollout_corr/rollout_is_seq_max:1.0197176933288574 - rollout_corr/rollout_is_seq_min:0.9908979535102844 - rollout_corr/rollout_is_seq_max_deviation:0.019717693328857422 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.092458315193653) - rollout_corr/training_log_ppl:np.float64(0.08484213713200006) - rollout_corr/kl:np.float64(0.000503509940610769) - rollout_corr/k3_kl:np.float64(0.0008734361738819141) - rollout_corr/rollout_ppl:np.float64(1.0919238738715649) - rollout_corr/rollout_log_ppl:np.float64(0.08433862631864031) - rollout_corr/log_ppl_diff:np.float64(0.0005035108206357108) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031829194267629646) - rollout_corr/log_ppl_diff_max:np.float64(0.16107693314552307) - rollout_corr/log_ppl_diff_min:np.float64(-0.02159041166305542) - rollout_corr/ppl_ratio:np.float64(1.0005644736811519) - rollout_corr/chi2_token:np.float64(0.00162202725186944) - rollout_corr/chi2_seq:np.float64(0.09504595655016601) - actor/pg_loss:np.float64(-0.0001654154621064663) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00023871528537711129) - actor/ppo_kl:np.float64(0.0003587856638356257) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29741760343313217) - perf/mfu/actor:np.float64(0.061329542521180766) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.6827654838562) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:0 - critic/score/mean:0.46875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007354356348514557 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007354356348514557 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:60.8828125 - response_length/max:197.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:60.8828125 - response_length_non_aborted/max:197.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:596.09375 - prompt_length/max:1521.0 - prompt_length/min:252.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.805895388126373e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.260761933401227) - timing_s/agent_loop/generate_sequences/max:np.float64(1.4389296304434538) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6682899079678464) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.4389296304434538) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:445 - timing_s/agent_loop/slowest/response_length:175 - timing_s/gen:7.425006395205855 - timing_s/reward:0.07034909538924694 - timing_s/old_log_prob:2.4274944588541985 - timing_s/adv:0.1527370996773243 - timing_s/update_actor:9.91138636879623 - timing_s/step:20.07283789664507 - timing_s/stop_profile:0.00011865980923175812 - timing_per_token_ms/update_actor:0.058931102284353214 - timing_per_token_ms/adv:0.000908143957745141 - timing_per_token_ms/gen:0.47638947742883714 - perf/total_num_tokens:168186 - perf/time_per_step:20.07283789664507 - perf/throughput:1047.3481681189573 (TaskRunner pid=2186821) Training Progress: 84%|████████▍ | 84/100 [41:45<05:55, 22.21s/it] (TaskRunner pid=2186821) step:85 - global_seqlen/min:16325 - global_seqlen/max:25483 - global_seqlen/minmax_diff:9158 - global_seqlen/balanced_min:20378 - global_seqlen/balanced_max:20380 - global_seqlen/mean:20378.5 - actor/entropy:0.0961875319480896 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12435293197631836 - training/rollout_probs_diff_mean:0.0022104678209871054 - training/rollout_probs_diff_std:0.008986894972622395 - training/rollout_actor_probs_pearson_corr:0.9984658360481262 - rollout_corr/rollout_is_mean:1.000191569328308 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5252803564071655 - rollout_corr/rollout_is_min:0.6941801905632019 - rollout_corr/rollout_is_std:0.025225764140486717 - rollout_corr/rollout_is_eff_sample_size:0.9993643035979273 - rollout_corr/rollout_is_seq_mean:1.0000587701797485 - rollout_corr/rollout_is_seq_std:0.003474070457741618 - rollout_corr/rollout_is_seq_max:1.012987732887268 - rollout_corr/rollout_is_seq_min:0.987943172454834 - rollout_corr/rollout_is_seq_max_deviation:0.012987732887268066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0985006871633232) - rollout_corr/training_log_ppl:np.float64(0.09045715395768639) - rollout_corr/kl:np.float64(0.0004510803064325941) - rollout_corr/k3_kl:np.float64(0.0004701950260463106) - rollout_corr/rollout_ppl:np.float64(1.097953483927995) - rollout_corr/rollout_log_ppl:np.float64(0.0900060733747523) - rollout_corr/log_ppl_diff:np.float64(0.0004510805829340825) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029007207358517917) - rollout_corr/log_ppl_diff_max:np.float64(0.029350727796554565) - rollout_corr/log_ppl_diff_min:np.float64(-0.017659738659858704) - rollout_corr/ppl_ratio:np.float64(1.000461032614112) - rollout_corr/chi2_token:np.float64(0.0010218755342066288) - rollout_corr/chi2_seq:np.float64(0.05891488725319505) - actor/pg_loss:np.float64(-0.000380878453142941) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005472565171658061) - actor/ppo_kl:np.float64(0.0002119349483313826) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26748207956552505) - perf/mfu/actor:np.float64(0.05966993150048007) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.61484146118164) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:0 - critic/score/mean:0.390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006996875745244324 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006996875745244324 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:60.015625 - response_length/max:271.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:60.015625 - response_length_non_aborted/max:271.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:576.8125 - prompt_length/max:888.0 - prompt_length/min:259.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.520856499671936e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.26062049344182014) - timing_s/agent_loop/generate_sequences/max:np.float64(1.6837278790771961) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6221279379824409) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.6837278790771961) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:704 - timing_s/agent_loop/slowest/response_length:271 - timing_s/gen:7.81646647118032 - timing_s/reward:0.06752477213740349 - timing_s/old_log_prob:2.2825107518583536 - timing_s/adv:0.12067915126681328 - timing_s/update_actor:9.907612515613437 - timing_s/step:20.27846842445433 - timing_s/stop_profile:0.0001116488128900528 - timing_per_token_ms/update_actor:0.06077245942791077 - timing_per_token_ms/adv:0.0007402357341488166 - timing_per_token_ms/gen:0.5087520483715386 - perf/total_num_tokens:163028 - perf/time_per_step:20.27846842445433 - perf/throughput:1004.9328959885864 (TaskRunner pid=2186821) Training Progress: 85%|████████▌ | 85/100 [42:05<05:24, 21.64s/it] (TaskRunner pid=2186821) step:86 - global_seqlen/min:16938 - global_seqlen/max:24621 - global_seqlen/minmax_diff:7683 - global_seqlen/balanced_min:20224 - global_seqlen/balanced_max:20225 - global_seqlen/mean:20224.375 - actor/entropy:0.09503393620252609 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.14452630281448364 - training/rollout_probs_diff_mean:0.0022688054013997316 - training/rollout_probs_diff_std:0.00910940021276474 - training/rollout_actor_probs_pearson_corr:0.9983958005905151 - rollout_corr/rollout_is_mean:1.0000323057174683 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5526758432388306 - rollout_corr/rollout_is_min:0.6120901107788086 - rollout_corr/rollout_is_std:0.025166623294353485 - rollout_corr/rollout_is_eff_sample_size:0.9993670419342761 - rollout_corr/rollout_is_seq_mean:1.0000190734863281 - rollout_corr/rollout_is_seq_std:0.0037850497756153345 - rollout_corr/rollout_is_seq_max:1.0154569149017334 - rollout_corr/rollout_is_seq_min:0.9778366088867188 - rollout_corr/rollout_is_seq_max_deviation:0.02216339111328125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0937399892136455) - rollout_corr/training_log_ppl:np.float64(0.08565731679118471) - rollout_corr/kl:np.float64(0.0009681722051269048) - rollout_corr/k3_kl:np.float64(0.0005242457798748587) - rollout_corr/rollout_ppl:np.float64(1.0925875343382359) - rollout_corr/rollout_log_ppl:np.float64(0.08468914415425388) - rollout_corr/log_ppl_diff:np.float64(0.0009681726369308308) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035082436515949667) - rollout_corr/log_ppl_diff_max:np.float64(0.026906728744506836) - rollout_corr/log_ppl_diff_min:np.float64(-0.013941187411546707) - rollout_corr/ppl_ratio:np.float64(1.0009810361079872) - rollout_corr/chi2_token:np.float64(8.188444189727306e-05) - rollout_corr/chi2_seq:np.float64(0.01394746988080442) - actor/pg_loss:np.float64(-8.437945507466793e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.512019510613754e-05) - actor/ppo_kl:np.float64(0.0006607537204104119) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.05828133225440979) - perf/mfu/actor:np.float64(0.05844908477560541) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.68750762939453) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:0 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007979866350069642 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007979866350069642 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:63.63671875 - response_length/max:269.0 - response_length/min:21.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:63.63671875 - response_length_non_aborted/max:269.0 - response_length_non_aborted/min:21.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:568.375 - prompt_length/max:912.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.68719071149826e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.2710339520126581) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7292438801378012) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6599176314484794) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7292438801378012) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:720 - timing_s/agent_loop/slowest/response_length:269 - timing_s/gen:7.54965215921402 - timing_s/reward:0.0682990700006485 - timing_s/old_log_prob:2.378750892356038 - timing_s/adv:0.16500810906291008 - timing_s/update_actor:10.076906379312277 - timing_s/step:20.323747577145696 - timing_s/stop_profile:6.505101919174194e-05 - timing_per_token_ms/update_actor:0.06228193936346783 - timing_per_token_ms/adv:0.0010198591369505244 - timing_per_token_ms/gen:0.4634247228048628 - perf/total_num_tokens:161795 - perf/time_per_step:20.323747577145696 - perf/throughput:995.1105190236941 (TaskRunner pid=2186821) Training Progress: 86%|████████▌ | 86/100 [42:25<04:57, 21.25s/it] (TaskRunner pid=2186821) step:87 - global_seqlen/min:15515 - global_seqlen/max:28894 - global_seqlen/minmax_diff:13379 - global_seqlen/balanced_min:21002 - global_seqlen/balanced_max:21011 - global_seqlen/mean:21009.0 - actor/entropy:0.10146483778953552 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5099416971206665 - training/rollout_probs_diff_mean:0.0027698653284460306 - training/rollout_probs_diff_std:0.012136919423937798 - training/rollout_actor_probs_pearson_corr:0.9972761273384094 - rollout_corr/rollout_is_mean:0.9996400475502014 - rollout_corr/rollout_is_ratio_fraction_high:6.406971078831702e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.406971078831702e-05 - rollout_corr/rollout_is_max:3.188687324523926 - rollout_corr/rollout_is_min:0.28876039385795593 - rollout_corr/rollout_is_std:0.03163576498627663 - rollout_corr/rollout_is_eff_sample_size:0.998999405740853 - rollout_corr/rollout_is_seq_mean:0.9997050762176514 - rollout_corr/rollout_is_seq_std:0.0037543082144111395 - rollout_corr/rollout_is_seq_max:1.0146812200546265 - rollout_corr/rollout_is_seq_min:0.9810764789581299 - rollout_corr/rollout_is_seq_max_deviation:0.018923521041870117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0893349526450038) - rollout_corr/training_log_ppl:np.float64(0.08257730866216662) - rollout_corr/kl:np.float64(0.00041104497830524167) - rollout_corr/k3_kl:np.float64(0.0006256297465644656) - rollout_corr/rollout_ppl:np.float64(1.088805693667382) - rollout_corr/rollout_log_ppl:np.float64(0.08216626255216397) - rollout_corr/log_ppl_diff:np.float64(0.00041104611000264413) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032669340203028696) - rollout_corr/log_ppl_diff_max:np.float64(0.02910405397415161) - rollout_corr/log_ppl_diff_min:np.float64(-0.013232894241809845) - rollout_corr/ppl_ratio:np.float64(1.0004246074240655) - rollout_corr/chi2_token:np.float64(0.0015545454807579517) - rollout_corr/chi2_seq:np.float64(0.07278866553679109) - actor/pg_loss:np.float64(0.00019642815459519625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(-0.000310074345634348) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.33189913630485535) - perf/mfu/actor:np.float64(0.06174830397591233) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.67496109008789) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:0 - critic/score/mean:0.4375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00131342897657305 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00131342897657305 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:60.96875 - response_length/max:416.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:60.96875 - response_length_non_aborted/max:416.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:595.5625 - prompt_length/max:1087.0 - prompt_length/min:256.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.604404628276825e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.36136968061327934) - timing_s/agent_loop/generate_sequences/max:np.float64(2.472342634573579) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7344769983319566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.472342634573579) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1007 - timing_s/agent_loop/slowest/response_length:416 - timing_s/gen:8.50223089568317 - timing_s/reward:0.07095866091549397 - timing_s/old_log_prob:2.3183823972940445 - timing_s/adv:0.12597763910889626 - timing_s/update_actor:9.843376768752933 - timing_s/step:20.94783579558134 - timing_s/stop_profile:0.00011215172708034515 - timing_per_token_ms/update_actor:0.0585664284875109 - timing_per_token_ms/adv:0.0007495456656010297 - timing_per_token_ms/gen:0.544735449492771 - perf/total_num_tokens:168072 - perf/time_per_step:20.94783579558134 - perf/throughput:1002.9198340590182 (TaskRunner pid=2186821) Training Progress: 87%|████████▋ | 87/100 [42:46<04:35, 21.17s/it] (TaskRunner pid=2186821) step:88 - global_seqlen/min:18306 - global_seqlen/max:26944 - global_seqlen/minmax_diff:8638 - global_seqlen/balanced_min:22623 - global_seqlen/balanced_max:22628 - global_seqlen/mean:22625.125 - actor/entropy:0.07744886726140976 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.13839662075042725 - training/rollout_probs_diff_mean:0.002282948000356555 - training/rollout_probs_diff_std:0.009653711691498756 - training/rollout_actor_probs_pearson_corr:0.9977383613586426 - rollout_corr/rollout_is_mean:0.999604344367981 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6470028162002563 - rollout_corr/rollout_is_min:0.5352633595466614 - rollout_corr/rollout_is_std:0.024999093264341354 - rollout_corr/rollout_is_eff_sample_size:0.9993750189126472 - rollout_corr/rollout_is_seq_mean:0.9996408820152283 - rollout_corr/rollout_is_seq_std:0.0034014969132840633 - rollout_corr/rollout_is_seq_max:1.015273928642273 - rollout_corr/rollout_is_seq_min:0.9880582690238953 - rollout_corr/rollout_is_seq_max_deviation:0.01527392864227295 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0637829122133553) - rollout_corr/training_log_ppl:np.float64(0.05952696956592263) - rollout_corr/kl:np.float64(0.00043886838502515957) - rollout_corr/k3_kl:np.float64(0.0004149295411513855) - rollout_corr/rollout_ppl:np.float64(1.0632442012429237) - rollout_corr/rollout_log_ppl:np.float64(0.059088100748340366) - rollout_corr/log_ppl_diff:np.float64(0.0004388688175822608) - rollout_corr/log_ppl_abs_diff:np.float64(0.002738780007348396) - rollout_corr/log_ppl_diff_max:np.float64(0.023479610681533813) - rollout_corr/log_ppl_diff_min:np.float64(-0.012138411402702332) - rollout_corr/ppl_ratio:np.float64(1.0004479619674385) - rollout_corr/chi2_token:np.float64(0.0008232854306697845) - rollout_corr/chi2_seq:np.float64(0.058355293702334166) - actor/pg_loss:np.float64(-2.8157373890280724e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000274344263743842) - actor/ppo_kl:np.float64(-0.00021105582402825007) - actor/pg_clipfrac_lower:np.float64(6.403688166756183e-05) - actor/grad_norm:np.float64(0.39451638609170914) - perf/mfu/actor:np.float64(0.06620109311592431) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.60878276824951) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:0 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007506116759032011 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.007506116759032011 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:55.87890625 - response_length/max:242.0 - response_length/min:31.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:55.87890625 - response_length_non_aborted/max:242.0 - response_length_non_aborted/min:31.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:651.15625 - prompt_length/max:1233.0 - prompt_length/min:346.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.97720456123352e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.42946951650083065) - timing_s/agent_loop/generate_sequences/max:np.float64(1.6336607318371534) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7271359804581152) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.6336607318371534) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:401 - timing_s/agent_loop/slowest/response_length:242 - timing_s/gen:7.497651543468237 - timing_s/reward:0.07208896428346634 - timing_s/old_log_prob:2.465151036158204 - timing_s/adv:0.12461346387863159 - timing_s/update_actor:9.95025759935379 - timing_s/step:20.194737285375595 - timing_s/stop_profile:0.00011076033115386963 - timing_per_token_ms/update_actor:0.05497349517048961 - timing_per_token_ms/adv:0.0006884683724323711 - timing_per_token_ms/gen:0.524128035195263 - perf/total_num_tokens:181001 - perf/time_per_step:20.194737285375595 - perf/throughput:1120.347577702059 (TaskRunner pid=2186821) Training Progress: 88%|████████▊ | 88/100 [43:06<04:10, 20.89s/it] (TaskRunner pid=2186821) step:89 - global_seqlen/min:17296 - global_seqlen/max:27541 - global_seqlen/minmax_diff:10245 - global_seqlen/balanced_min:22486 - global_seqlen/balanced_max:22488 - global_seqlen/mean:22487.375 - actor/entropy:0.10038791596889496 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22949323058128357 - training/rollout_probs_diff_mean:0.003009300446137786 - training/rollout_probs_diff_std:0.011635876260697842 - training/rollout_actor_probs_pearson_corr:0.9974488615989685 - rollout_corr/rollout_is_mean:0.999867856502533 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.584578659385443e-05 - rollout_corr/rollout_is_max:1.6244648694992065 - rollout_corr/rollout_is_min:0.3921358287334442 - rollout_corr/rollout_is_std:0.03237418830394745 - rollout_corr/rollout_is_eff_sample_size:0.9989526523855053 - rollout_corr/rollout_is_seq_mean:0.9998213648796082 - rollout_corr/rollout_is_seq_std:0.004119370598345995 - rollout_corr/rollout_is_seq_max:1.014970064163208 - rollout_corr/rollout_is_seq_min:0.9827616810798645 - rollout_corr/rollout_is_seq_max_deviation:0.017238318920135498 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0928037841804326) - rollout_corr/training_log_ppl:np.float64(0.08605340984649956) - rollout_corr/kl:np.float64(0.00034215472579823825) - rollout_corr/k3_kl:np.float64(0.0006300729410405048) - rollout_corr/rollout_ppl:np.float64(1.0924206171184778) - rollout_corr/rollout_log_ppl:np.float64(0.08571125372145616) - rollout_corr/log_ppl_diff:np.float64(0.0003421561250434024) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032413370445283363) - rollout_corr/log_ppl_diff_max:np.float64(0.022987104952335358) - rollout_corr/log_ppl_diff_min:np.float64(-0.015995271503925323) - rollout_corr/ppl_ratio:np.float64(1.0003535056021065) - rollout_corr/chi2_token:np.float64(0.0018907524645328522) - rollout_corr/chi2_seq:np.float64(0.12052021478302777) - actor/pg_loss:np.float64(-6.640632636845112e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(3.9457070670323446e-05) - actor/ppo_kl:np.float64(-0.0002827961003930568) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.06062762439250946) - perf/mfu/actor:np.float64(0.06635115220467382) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.77862930297852) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:0 - critic/score/mean:0.30859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00283959973603487 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00283959973603487 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:59.32421875 - response_length/max:291.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:59.32421875 - response_length_non_aborted/max:291.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:643.40625 - prompt_length/max:1261.0 - prompt_length/min:356.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.530410170555115e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3594407197088003) - timing_s/agent_loop/generate_sequences/max:np.float64(2.0016611702740192) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.6736538480763556) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.0016611702740192) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:659 - timing_s/agent_loop/slowest/response_length:291 - timing_s/gen:7.680245589464903 - timing_s/reward:0.06961960345506668 - timing_s/old_log_prob:2.482805097475648 - timing_s/adv:0.1313204113394022 - timing_s/update_actor:9.601976839825511 - timing_s/step:20.053274491801858 - timing_s/stop_profile:0.00011432357132434845 - timing_per_token_ms/update_actor:0.05337426466976198 - timing_per_token_ms/adv:0.0007299674336122057 - timing_per_token_ms/gen:0.5057118317946205 - perf/total_num_tokens:179899 - perf/time_per_step:20.053274491801858 - perf/throughput:1121.3816979961675 (TaskRunner pid=2186821) Training Progress: 89%|████████▉ | 89/100 [43:27<03:47, 20.66s/it] (TaskRunner pid=2186821) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2186821) validation generation end (TaskRunner pid=2186821) [prompt] user (TaskRunner pid=2186821) Your task is to answer the user's question using available tools. (TaskRunner pid=2186821) You have access to the following tools: (TaskRunner pid=2186821) Name: Axolotl (TaskRunner pid=2186821) Description: Collection of axolotl pictures and facts (TaskRunner pid=2186821) Documentation: (TaskRunner pid=2186821) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2186821) Parameters: {} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{url, source, description} (TaskRunner pid=2186821) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2186821) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2186821) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2186821) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Array[Object{fact, source}] (TaskRunner pid=2186821) (TaskRunner pid=2186821) Use the following format: (TaskRunner pid=2186821) Thought: you should always think about what to do (TaskRunner pid=2186821) Action: the action to take, should be one of the tool names. (TaskRunner pid=2186821) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2186821) (TaskRunner pid=2186821) Begin! (TaskRunner pid=2186821) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2186821) assistant (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: I need to retrieve a random axolotl image. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2187230) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2186821) step:90 - global_seqlen/min:18901 - global_seqlen/max:29580 - global_seqlen/minmax_diff:10679 - global_seqlen/balanced_min:23921 - global_seqlen/balanced_max:23923 - global_seqlen/mean:23922.25 - actor/entropy:0.0823388621211052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2793269753456116 - training/rollout_probs_diff_mean:0.002548050833866 - training/rollout_probs_diff_std:0.010876636952161789 - training/rollout_actor_probs_pearson_corr:0.9972853064537048 - rollout_corr/rollout_is_mean:1.0003397464752197 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7267858982086182 - rollout_corr/rollout_is_min:0.6176038384437561 - rollout_corr/rollout_is_std:0.03031475655734539 - rollout_corr/rollout_is_eff_sample_size:0.9990825732872448 - rollout_corr/rollout_is_seq_mean:1.0003252029418945 - rollout_corr/rollout_is_seq_std:0.003927420359104872 - rollout_corr/rollout_is_seq_max:1.0202091932296753 - rollout_corr/rollout_is_seq_min:0.991267204284668 - rollout_corr/rollout_is_seq_max_deviation:0.020209193229675293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0736785060726106) - rollout_corr/training_log_ppl:np.float64(0.06932337003763678) - rollout_corr/kl:np.float64(0.0005910541265166103) - rollout_corr/k3_kl:np.float64(0.0011513489333899313) - rollout_corr/rollout_ppl:np.float64(1.0729992659762502) - rollout_corr/rollout_log_ppl:np.float64(0.06873231557074178) - rollout_corr/log_ppl_diff:np.float64(0.0005910544668950024) - rollout_corr/log_ppl_abs_diff:np.float64(0.003654166425803851) - rollout_corr/log_ppl_diff_max:np.float64(0.05813804268836975) - rollout_corr/log_ppl_diff_min:np.float64(-0.017090462148189545) - rollout_corr/ppl_ratio:np.float64(1.000619990983978) - rollout_corr/chi2_token:np.float64(0.002650438342243433) - rollout_corr/chi2_seq:np.float64(0.18175277928821743) - actor/pg_loss:np.float64(-0.000767852645367384) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004610119649441913) - actor/ppo_kl:np.float64(0.0005058494558696225) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.8131211251020432) - perf/mfu/actor:np.float64(0.07026519311187743) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.84130477905273) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5643382352941176) - val-aux/tooluse/reward/std@16:np.float64(0.025200550760176754) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5753088235294117) - val-aux/tooluse/reward/best@2/std:np.float64(0.019423727416340034) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5534705882352942) - val-aux/tooluse/reward/worst@2/std:np.float64(0.023525176219296687) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.564485294117647) - val-aux/tooluse/reward/maj@2/std:np.float64(0.02500577736990771) - val-aux/tooluse/reward/best@4/mean:np.float64(0.5827205882352942) - val-aux/tooluse/reward/best@4/std:np.float64(0.011070854979983632) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5434705882352941) - val-aux/tooluse/reward/worst@4/std:np.float64(0.019089348620445867) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5658676470588235) - val-aux/tooluse/reward/maj@4/std:np.float64(0.021602139398553266) - val-aux/tooluse/reward/best@8/mean:np.float64(0.5868235294117646) - val-aux/tooluse/reward/best@8/std:np.float64(0.004948607833743041) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.535235294117647) - val-aux/tooluse/reward/worst@8/std:np.float64(0.012780132161577052) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5661911764705883) - val-aux/tooluse/reward/maj@8/std:np.float64(0.015651275248249153) - val-aux/tooluse/reward/best@16/mean:np.float64(0.5881323529411765) - val-aux/tooluse/reward/best@16/std:np.float64(0.0012260684910055711) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5304411764705882) - val-aux/tooluse/reward/worst@16/std:np.float64(0.005383773933771195) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5664117647058824) - val-aux/tooluse/reward/maj@16/std:np.float64(0.010779063665948978) - val-aux/tooluse/score/mean@16:np.float64(0.5643382352941176) - val-aux/tooluse/score/std@16:np.float64(0.025200550760176754) - val-aux/tooluse/score/best@2/mean:np.float64(0.5753088235294117) - val-aux/tooluse/score/best@2/std:np.float64(0.019423727416340034) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5534705882352942) - val-aux/tooluse/score/worst@2/std:np.float64(0.023525176219296687) - val-aux/tooluse/score/maj@2/mean:np.float64(0.564485294117647) - val-aux/tooluse/score/maj@2/std:np.float64(0.02500577736990771) - val-aux/tooluse/score/best@4/mean:np.float64(0.5827205882352942) - val-aux/tooluse/score/best@4/std:np.float64(0.011070854979983632) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5434705882352941) - val-aux/tooluse/score/worst@4/std:np.float64(0.019089348620445867) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5658676470588235) - val-aux/tooluse/score/maj@4/std:np.float64(0.021602139398553266) - val-aux/tooluse/score/best@8/mean:np.float64(0.5868235294117646) - val-aux/tooluse/score/best@8/std:np.float64(0.004948607833743041) - val-aux/tooluse/score/worst@8/mean:np.float64(0.535235294117647) - val-aux/tooluse/score/worst@8/std:np.float64(0.012780132161577052) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5661911764705883) - val-aux/tooluse/score/maj@8/std:np.float64(0.015651275248249153) - val-aux/tooluse/score/best@16/mean:np.float64(0.5881323529411765) - val-aux/tooluse/score/best@16/std:np.float64(0.0012260684910055711) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5304411764705882) - val-aux/tooluse/score/worst@16/std:np.float64(0.005383773933771195) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5664117647058824) - val-aux/tooluse/score/maj@16/std:np.float64(0.010779063665948978) - val-core/tooluse/acc/mean@16:np.float64(0.5643382352941176) - val-aux/tooluse/acc/std@16:np.float64(0.025200550760176754) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5753088235294117) - val-aux/tooluse/acc/best@2/std:np.float64(0.019423727416340034) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5534705882352942) - val-aux/tooluse/acc/worst@2/std:np.float64(0.023525176219296687) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.564485294117647) - val-aux/tooluse/acc/maj@2/std:np.float64(0.02500577736990771) - val-aux/tooluse/acc/best@4/mean:np.float64(0.5827205882352942) - val-aux/tooluse/acc/best@4/std:np.float64(0.011070854979983632) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5434705882352941) - val-aux/tooluse/acc/worst@4/std:np.float64(0.019089348620445867) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5658676470588235) - val-aux/tooluse/acc/maj@4/std:np.float64(0.021602139398553266) - val-aux/tooluse/acc/best@8/mean:np.float64(0.5868235294117646) - val-aux/tooluse/acc/best@8/std:np.float64(0.004948607833743041) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.535235294117647) - val-aux/tooluse/acc/worst@8/std:np.float64(0.012780132161577052) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5661911764705883) - val-aux/tooluse/acc/maj@8/std:np.float64(0.015651275248249153) - val-core/tooluse/acc/best@16/mean:np.float64(0.5881323529411765) - val-core/tooluse/acc/best@16/std:np.float64(0.0012260684910055711) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5304411764705882) - val-aux/tooluse/acc/worst@16/std:np.float64(0.005383773933771195) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5664117647058824) - val-core/tooluse/acc/maj@16/std:np.float64(0.010779063665948978) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:0 - critic/score/mean:0.27734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.27734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008656180463731289 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.008656180463731289 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:74.0078125 - response_length/max:547.0 - response_length/min:21.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:74.0078125 - response_length_non_aborted/max:547.0 - response_length_non_aborted/min:21.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:673.5625 - prompt_length/max:1261.0 - prompt_length/min:300.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.24970954656601e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3906703069806099) - timing_s/agent_loop/generate_sequences/max:np.float64(3.188204074278474) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8403847993176896) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.188204074278474) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:600 - timing_s/agent_loop/slowest/response_length:547 - timing_s/gen:8.993232371285558 - timing_s/reward:0.07457084581255913 - timing_s/old_log_prob:2.4410028103739023 - timing_s/adv:0.1360334400087595 - timing_s/update_actor:9.921312434598804 - timing_s/step:21.65125297009945 - timing_s/testing:19.563337489962578 - timing_s/save_checkpoint:6.556817794218659 - timing_s/stop_profile:3.8759782910346985e-05 - timing_per_token_ms/update_actor:0.05184144695105395 - timing_per_token_ms/adv:0.0007108102290167078 - timing_per_token_ms/gen:0.4746771018307589 - perf/total_num_tokens:191378 - perf/time_per_step:21.65125297009945 - perf/throughput:1104.889866329529 (TaskRunner pid=2186821) Training Progress: 90%|█████████ | 90/100 [44:14<04:48, 28.80s/it] (TaskRunner pid=2186821) step:91 - global_seqlen/min:17844 - global_seqlen/max:28544 - global_seqlen/minmax_diff:10700 - global_seqlen/balanced_min:22571 - global_seqlen/balanced_max:22573 - global_seqlen/mean:22571.625 - actor/entropy:0.053729720413684845 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102105498313904 - training/rollout_probs_diff_mean:0.001740073668770492 - training/rollout_probs_diff_std:0.009415044449269772 - training/rollout_actor_probs_pearson_corr:0.9968917369842529 - rollout_corr/rollout_is_mean:0.9998834133148193 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013608527660835534 - rollout_corr/rollout_is_max:1.6952793598175049 - rollout_corr/rollout_is_min:0.3830607831478119 - rollout_corr/rollout_is_std:0.02560102939605713 - rollout_corr/rollout_is_eff_sample_size:0.9993450166104645 - rollout_corr/rollout_is_seq_mean:0.9998332262039185 - rollout_corr/rollout_is_seq_std:0.003369073150679469 - rollout_corr/rollout_is_seq_max:1.0201692581176758 - rollout_corr/rollout_is_seq_min:0.9857123494148254 - rollout_corr/rollout_is_seq_max_deviation:0.02016925811767578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.05597510561347) - rollout_corr/training_log_ppl:np.float64(0.05267521727523672) - rollout_corr/kl:np.float64(0.0016971310389592986) - rollout_corr/k3_kl:np.float64(0.0010454924462020877) - rollout_corr/rollout_ppl:np.float64(1.0541398110799491) - rollout_corr/rollout_log_ppl:np.float64(0.05097808571508722) - rollout_corr/log_ppl_diff:np.float64(0.0016971315601495007) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037543022483532695) - rollout_corr/log_ppl_diff_max:np.float64(0.047784220427274704) - rollout_corr/log_ppl_diff_min:np.float64(-0.022862013429403305) - rollout_corr/ppl_ratio:np.float64(1.0017294795252383) - rollout_corr/chi2_token:np.float64(0.000754273496568203) - rollout_corr/chi2_seq:np.float64(0.01849907636642456) - actor/pg_loss:np.float64(1.2909062206745148e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028437291985028423) - actor/ppo_kl:np.float64(0.0011705174980214395) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2050250507891178) - perf/mfu/actor:np.float64(0.06486334440804689) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.9461784362793) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0033114084508270025 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0033114084508270025 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:86.11328125 - response_length/max:741.0 - response_length/min:24.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.11328125 - response_length_non_aborted/max:741.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:619.25 - prompt_length/max:1504.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.5114044547080994e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3167601749300957) - timing_s/agent_loop/generate_sequences/max:np.float64(4.152940118685365) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7976723746323842) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.152940118685365) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:604 - timing_s/agent_loop/slowest/response_length:741 - timing_s/gen:9.910437446087599 - timing_s/reward:0.06484177894890308 - timing_s/old_log_prob:2.4560873862355947 - timing_s/adv:0.140070715919137 - timing_s/update_actor:10.038492850959301 - timing_s/step:22.702763283625245 - timing_s/stop_profile:0.00011825188994407654 - timing_per_token_ms/update_actor:0.05559243547462412 - timing_per_token_ms/adv:0.0007757013281007515 - timing_per_token_ms/gen:0.4495548852840825 - perf/total_num_tokens:180573 - perf/time_per_step:22.702763283625245 - perf/throughput:994.2236862540944 (TaskRunner pid=2186821) Training Progress: 91%|█████████ | 91/100 [44:37<04:02, 26.99s/it] (TaskRunner pid=2186821) step:92 - global_seqlen/min:15644 - global_seqlen/max:21371 - global_seqlen/minmax_diff:5727 - global_seqlen/balanced_min:19781 - global_seqlen/balanced_max:19783 - global_seqlen/mean:19782.25 - actor/entropy:0.10824279487133026 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31746214628219604 - training/rollout_probs_diff_mean:0.003058228176087141 - training/rollout_probs_diff_std:0.012305145151913166 - training/rollout_actor_probs_pearson_corr:0.997308075428009 - rollout_corr/rollout_is_mean:1.0001909732818604 - rollout_corr/rollout_is_ratio_fraction_high:6.432522786781192e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.5125865936279297 - rollout_corr/rollout_is_min:0.541451632976532 - rollout_corr/rollout_is_std:0.033562012016773224 - rollout_corr/rollout_is_eff_sample_size:0.9988750966741704 - rollout_corr/rollout_is_seq_mean:1.0000405311584473 - rollout_corr/rollout_is_seq_std:0.004703574348241091 - rollout_corr/rollout_is_seq_max:1.0411250591278076 - rollout_corr/rollout_is_seq_min:0.9804712533950806 - rollout_corr/rollout_is_seq_max_deviation:0.04112505912780762 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1021919082850218) - rollout_corr/training_log_ppl:np.float64(0.09181874391106248) - rollout_corr/kl:np.float64(0.0009127749330870571) - rollout_corr/k3_kl:np.float64(0.0010845346354813046) - rollout_corr/rollout_ppl:np.float64(1.101098203100264) - rollout_corr/rollout_log_ppl:np.float64(0.09090596910573367) - rollout_corr/log_ppl_diff:np.float64(0.0009127748053288087) - rollout_corr/log_ppl_abs_diff:np.float64(0.004611733738784096) - rollout_corr/log_ppl_diff_max:np.float64(0.03188742697238922) - rollout_corr/log_ppl_diff_min:np.float64(-0.019275113940238953) - rollout_corr/ppl_ratio:np.float64(1.0009348115418106) - rollout_corr/chi2_token:np.float64(0.0029305326752364635) - rollout_corr/chi2_seq:np.float64(0.17545894416980445) - actor/pg_loss:np.float64(-0.0005388801218941808) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011998453192063607) - actor/ppo_kl:np.float64(0.00043901222511522064) - actor/pg_clipfrac_lower:np.float64(1.6208507076953538e-05) - actor/grad_norm:np.float64(0.3320599216967821) - perf/mfu/actor:np.float64(0.05800264479099754) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.99492645263672) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01101569551974535 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01101569551974535 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:60.7265625 - response_length/max:477.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:60.7265625 - response_length_non_aborted/max:477.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:557.46875 - prompt_length/max:967.0 - prompt_length/min:253.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001497473567724228 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4306583758443594) - timing_s/agent_loop/generate_sequences/max:np.float64(2.7337815277278423) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.754132783957175) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.7337815277278423) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:357 - timing_s/agent_loop/slowest/response_length:477 - timing_s/gen:8.714579444378614 - timing_s/reward:0.09057171642780304 - timing_s/old_log_prob:2.2890512384474277 - timing_s/adv:0.11212765984237194 - timing_s/update_actor:9.872397037222981 - timing_s/step:21.184980154037476 - timing_s/stop_profile:0.00011058151721954346 - timing_per_token_ms/update_actor:0.06238166182577173 - timing_per_token_ms/adv:0.0007085117961959076 - timing_per_token_ms/gen:0.5605673127736147 - perf/total_num_tokens:158258 - perf/time_per_step:21.184980154037476 - perf/throughput:933.7865721922736 (TaskRunner pid=2186821) Training Progress: 92%|█████████▏| 92/100 [44:58<03:22, 25.27s/it] (TaskRunner pid=2186821) step:93 - global_seqlen/min:17198 - global_seqlen/max:24466 - global_seqlen/minmax_diff:7268 - global_seqlen/balanced_min:21416 - global_seqlen/balanced_max:21511 - global_seqlen/mean:21429.875 - actor/entropy:0.10560684651136398 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413555383682251 - training/rollout_probs_diff_mean:0.0030651416163891554 - training/rollout_probs_diff_std:0.0121381226927042 - training/rollout_actor_probs_pearson_corr:0.9973388910293579 - rollout_corr/rollout_is_mean:0.999578058719635 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7523056268692017 - rollout_corr/rollout_is_min:0.5087061524391174 - rollout_corr/rollout_is_std:0.032223787158727646 - rollout_corr/rollout_is_eff_sample_size:0.9989619313293221 - rollout_corr/rollout_is_seq_mean:0.9993093013763428 - rollout_corr/rollout_is_seq_std:0.004702479112893343 - rollout_corr/rollout_is_seq_max:1.0194453001022339 - rollout_corr/rollout_is_seq_min:0.9826031923294067 - rollout_corr/rollout_is_seq_max_deviation:0.019445300102233887 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.107296394649893) - rollout_corr/training_log_ppl:np.float64(0.09752401821242529) - rollout_corr/kl:np.float64(0.0007962529997911361) - rollout_corr/k3_kl:np.float64(0.001236870685897451) - rollout_corr/rollout_ppl:np.float64(1.10622769407928) - rollout_corr/rollout_log_ppl:np.float64(0.09672776410661754) - rollout_corr/log_ppl_diff:np.float64(0.0007962540985317901) - rollout_corr/log_ppl_abs_diff:np.float64(0.0046425195469055325) - rollout_corr/log_ppl_diff_max:np.float64(0.034155428409576416) - rollout_corr/log_ppl_diff_min:np.float64(-0.027831733226776123) - rollout_corr/ppl_ratio:np.float64(1.000820759916678) - rollout_corr/chi2_token:np.float64(0.003672647988423705) - rollout_corr/chi2_seq:np.float64(0.1993567522149533) - actor/pg_loss:np.float64(-0.0003916305722668767) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013478272157954052) - actor/ppo_kl:np.float64(-0.0004308259566130346) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5551838800311089) - perf/mfu/actor:np.float64(0.06318009598319979) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.9970817565918) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:0 - critic/score/mean:0.32421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.32421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009326752042397857 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009326752042397857 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:68.05859375 - response_length/max:882.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:68.05859375 - response_length_non_aborted/max:882.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:601.625 - prompt_length/max:1127.0 - prompt_length/min:365.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012545660138130188 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3932042345404625) - timing_s/agent_loop/generate_sequences/max:np.float64(4.70703068934381) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.79028696558089) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.70703068934381) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:671 - timing_s/agent_loop/slowest/response_length:882 - timing_s/gen:10.571364218369126 - timing_s/reward:0.06661546789109707 - timing_s/old_log_prob:2.3541376926004887 - timing_s/adv:0.11667812429368496 - timing_s/update_actor:9.75522230565548 - timing_s/step:22.952574487775564 - timing_s/stop_profile:7.41463154554367e-05 - timing_per_token_ms/update_actor:0.05690200191120737 - timing_per_token_ms/adv:0.0006805809897029553 - timing_per_token_ms/gen:0.606747644973261 - perf/total_num_tokens:171439 - perf/time_per_step:22.952574487775564 - perf/throughput:933.6588804629935 (TaskRunner pid=2186821) Training Progress: 93%|█████████▎| 93/100 [45:21<02:52, 24.58s/it] (TaskRunner pid=2186821) step:94 - global_seqlen/min:17949 - global_seqlen/max:25338 - global_seqlen/minmax_diff:7389 - global_seqlen/balanced_min:21996 - global_seqlen/balanced_max:21998 - global_seqlen/mean:21996.875 - actor/entropy:0.08015390485525131 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311316251754761 - training/rollout_probs_diff_mean:0.0025048493407666683 - training/rollout_probs_diff_std:0.014926725998520851 - training/rollout_actor_probs_pearson_corr:0.9947695136070251 - rollout_corr/rollout_is_mean:1.0001863241195679 - rollout_corr/rollout_is_ratio_fraction_high:0.0001792650145944208 - rollout_corr/rollout_is_ratio_fraction_low:0.0004182850243523717 - rollout_corr/rollout_is_max:3.7225520610809326 - rollout_corr/rollout_is_min:0.21798011660575867 - rollout_corr/rollout_is_std:0.0327676385641098 - rollout_corr/rollout_is_eff_sample_size:0.9989276714704524 - rollout_corr/rollout_is_seq_mean:1.000191330909729 - rollout_corr/rollout_is_seq_std:0.004530466161668301 - rollout_corr/rollout_is_seq_max:1.0286916494369507 - rollout_corr/rollout_is_seq_min:0.9870480298995972 - rollout_corr/rollout_is_seq_max_deviation:0.028691649436950684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0838712938129902) - rollout_corr/training_log_ppl:np.float64(0.07754150877644861) - rollout_corr/kl:np.float64(0.0024726516294357737) - rollout_corr/k3_kl:np.float64(0.002506558561928423) - rollout_corr/rollout_ppl:np.float64(1.080967173911631) - rollout_corr/rollout_log_ppl:np.float64(0.07506885696875543) - rollout_corr/log_ppl_diff:np.float64(0.0024726517931412673) - rollout_corr/log_ppl_abs_diff:np.float64(0.005323115996361594) - rollout_corr/log_ppl_diff_max:np.float64(0.23773008584976196) - rollout_corr/log_ppl_diff_min:np.float64(-0.022450793534517288) - rollout_corr/ppl_ratio:np.float64(1.002725764643401) - rollout_corr/chi2_token:np.float64(0.0018759856466203928) - rollout_corr/chi2_seq:np.float64(0.11286435835063457) - actor/pg_loss:np.float64(-0.00023683113977313042) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006263627110456582) - actor/ppo_kl:np.float64(0.002028622540912295) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.7879590741358697) - perf/mfu/actor:np.float64(0.06505628583795689) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.9808235168457) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:0 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00318942335434258 - critic/advantages/max:0.625 - critic/advantages/min:-0.625 - critic/returns/mean:-0.00318942335434258 - critic/returns/max:0.625 - critic/returns/min:-0.625 - response_length/mean:65.37109375 - response_length/max:357.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:65.37109375 - response_length_non_aborted/max:357.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:622.03125 - prompt_length/max:1068.0 - prompt_length/min:309.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.832903742790222e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3352525942027569) - timing_s/agent_loop/generate_sequences/max:np.float64(2.417144061997533) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7498065964246052) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.417144061997533) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:681 - timing_s/agent_loop/slowest/response_length:357 - timing_s/gen:8.31571107916534 - timing_s/reward:0.08572586812078953 - timing_s/old_log_prob:2.361599260941148 - timing_s/adv:0.13315275683999062 - timing_s/update_actor:9.824275519698858 - timing_s/step:20.809847692027688 - timing_s/stop_profile:0.00012519583106040955 - timing_per_token_ms/update_actor:0.05582767733882005 - timing_per_token_ms/adv:0.0007566572344934827 - timing_per_token_ms/gen:0.49690535280342635 - perf/total_num_tokens:175975 - perf/time_per_step:20.809847692027688 - perf/throughput:1057.041614409656 (TaskRunner pid=2186821) Training Progress: 94%|█████████▍| 94/100 [45:42<02:20, 23.47s/it] (TaskRunner pid=2186821) step:95 - global_seqlen/min:15649 - global_seqlen/max:30549 - global_seqlen/minmax_diff:14900 - global_seqlen/balanced_min:20834 - global_seqlen/balanced_max:28123 - global_seqlen/mean:21748.875 - actor/entropy:0.07513740658760071 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35350093245506287 - training/rollout_probs_diff_mean:0.0019117791671305895 - training/rollout_probs_diff_std:0.009358747862279415 - training/rollout_actor_probs_pearson_corr:0.9978646636009216 - rollout_corr/rollout_is_mean:0.9996581673622131 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.939343777252361e-05 - rollout_corr/rollout_is_max:1.897936224937439 - rollout_corr/rollout_is_min:0.4058622717857361 - rollout_corr/rollout_is_std:0.02562313713133335 - rollout_corr/rollout_is_eff_sample_size:0.9993434689215152 - rollout_corr/rollout_is_seq_mean:0.9994479417800903 - rollout_corr/rollout_is_seq_std:0.0035836442839354277 - rollout_corr/rollout_is_seq_max:1.0104641914367676 - rollout_corr/rollout_is_seq_min:0.9822297096252441 - rollout_corr/rollout_is_seq_max_deviation:0.01777029037475586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.105578055139631) - rollout_corr/training_log_ppl:np.float64(0.09638552920841903) - rollout_corr/kl:np.float64(0.0012761271027748045) - rollout_corr/k3_kl:np.float64(0.0010636132939431664) - rollout_corr/rollout_ppl:np.float64(1.1041361936368048) - rollout_corr/rollout_log_ppl:np.float64(0.09510940151540126) - rollout_corr/log_ppl_diff:np.float64(0.0012761276957462542) - rollout_corr/log_ppl_abs_diff:np.float64(0.003952755214413628) - rollout_corr/log_ppl_diff_max:np.float64(0.030952416360378265) - rollout_corr/log_ppl_diff_min:np.float64(-0.03871774673461914) - rollout_corr/ppl_ratio:np.float64(1.001297430600971) - rollout_corr/chi2_token:np.float64(0.002524543320760131) - rollout_corr/chi2_seq:np.float64(0.059393760515376925) - actor/pg_loss:np.float64(-0.000198688474483788) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00042582985770422965) - actor/ppo_kl:np.float64(0.0003791285014335122) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.319208649918437) - perf/mfu/actor:np.float64(0.060077570529742295) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.80831146240234) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:0 - critic/score/mean:0.30859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0010023420909419656 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0010023420909419656 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:98.40234375 - response_length/max:8192.0 - response_length/min:24.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:98.40234375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:24.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:581.25 - prompt_length/max:1109.0 - prompt_length/min:226.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001066494733095169 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3445663843303919) - timing_s/agent_loop/generate_sequences/max:np.float64(46.47439414821565) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9527361490545445) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(46.47439414821565) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:540 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.214822486042976 - timing_s/reward:0.07029260881245136 - timing_s/old_log_prob:2.7041114643216133 - timing_s/adv:0.12733351811766624 - timing_s/update_actor:10.45735114440322 - timing_s/step:65.6581646706909 - timing_s/stop_profile:0.0001132376492023468 - timing_per_token_ms/update_actor:0.06010282798767304 - timing_per_token_ms/adv:0.0007318396820391069 - timing_per_token_ms/gen:2.0727570356890546 - perf/total_num_tokens:173991 - perf/time_per_step:65.6581646706909 - perf/throughput:331.24402896550146 (TaskRunner pid=2186821) Training Progress: 95%|█████████▌| 95/100 [46:48<03:00, 36.14s/it] (TaskRunner pid=2186821) step:96 - global_seqlen/min:16944 - global_seqlen/max:31773 - global_seqlen/minmax_diff:14829 - global_seqlen/balanced_min:22802 - global_seqlen/balanced_max:22810 - global_seqlen/mean:22807.375 - actor/entropy:0.12361276894807816 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846313774585724 - training/rollout_probs_diff_mean:0.0026221766602247953 - training/rollout_probs_diff_std:0.009845267049968243 - training/rollout_actor_probs_pearson_corr:0.9984864592552185 - rollout_corr/rollout_is_mean:1.0001460313796997 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.564319133758545 - rollout_corr/rollout_is_min:0.5897137522697449 - rollout_corr/rollout_is_std:0.02817048504948616 - rollout_corr/rollout_is_eff_sample_size:0.9992072910640203 - rollout_corr/rollout_is_seq_mean:1.0002440214157104 - rollout_corr/rollout_is_seq_std:0.0036513926461338997 - rollout_corr/rollout_is_seq_max:1.0137042999267578 - rollout_corr/rollout_is_seq_min:0.9876942038536072 - rollout_corr/rollout_is_seq_max_deviation:0.013704299926757812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1153161800466478) - rollout_corr/training_log_ppl:np.float64(0.10447345185093582) - rollout_corr/kl:np.float64(0.0008286499134086966) - rollout_corr/k3_kl:np.float64(0.0013676498009410065) - rollout_corr/rollout_ppl:np.float64(1.1144337393343449) - rollout_corr/rollout_log_ppl:np.float64(0.10364480025054945) - rollout_corr/log_ppl_diff:np.float64(0.0008286516003863653) - rollout_corr/log_ppl_abs_diff:np.float64(0.003987944810432964) - rollout_corr/log_ppl_diff_max:np.float64(0.06060691922903061) - rollout_corr/log_ppl_diff_min:np.float64(-0.02798895537853241) - rollout_corr/ppl_ratio:np.float64(1.0008571883663535) - rollout_corr/chi2_token:np.float64(0.004126191139221191) - rollout_corr/chi2_seq:np.float64(0.25006804475560784) - actor/pg_loss:np.float64(-0.00043539959006011486) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007866308542361367) - actor/ppo_kl:np.float64(0.0007061706304796189) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22839521430432796) - perf/mfu/actor:np.float64(0.06751675628877768) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.74926376342773) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008263812400400639 - critic/advantages/max:0.75 - critic/advantages/min:-0.75 - critic/returns/mean:-0.008263812400400639 - critic/returns/max:0.75 - critic/returns/min:-0.75 - response_length/mean:74.44921875 - response_length/max:662.0 - response_length/min:28.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:74.44921875 - response_length_non_aborted/max:662.0 - response_length_non_aborted/min:28.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:638.28125 - prompt_length/max:1499.0 - prompt_length/min:275.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.089894592761993e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.46803602017462254) - timing_s/agent_loop/generate_sequences/max:np.float64(3.733808971941471) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8648341324442299) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.733808971941471) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:512 - timing_s/agent_loop/slowest/response_length:662 - timing_s/gen:9.566982191056013 - timing_s/reward:0.07348695024847984 - timing_s/old_log_prob:2.3647974394261837 - timing_s/adv:0.11896724626421928 - timing_s/update_actor:9.706341752782464 - timing_s/step:21.91598841547966 - timing_s/stop_profile:0.0001126658171415329 - timing_per_token_ms/update_actor:0.05319738545526646 - timing_per_token_ms/adv:0.0006520218036063953 - timing_per_token_ms/gen:0.5019666399630628 - perf/total_num_tokens:182459 - perf/time_per_step:21.91598841547966 - perf/throughput:1040.672889929561 (TaskRunner pid=2186821) Training Progress: 96%|█████████▌| 96/100 [47:10<02:07, 31.88s/it] (TaskRunner pid=2186821) step:97 - global_seqlen/min:16727 - global_seqlen/max:23205 - global_seqlen/minmax_diff:6478 - global_seqlen/balanced_min:21097 - global_seqlen/balanced_max:21231 - global_seqlen/mean:21137.75 - actor/entropy:0.1040182113647461 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.269555002450943 - training/rollout_probs_diff_mean:0.002123801503330469 - training/rollout_probs_diff_std:0.008524525910615921 - training/rollout_actor_probs_pearson_corr:0.9986470937728882 - rollout_corr/rollout_is_mean:0.9998640418052673 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5506705045700073 - rollout_corr/rollout_is_min:0.6108541488647461 - rollout_corr/rollout_is_std:0.02370302565395832 - rollout_corr/rollout_is_eff_sample_size:0.9994382439470041 - rollout_corr/rollout_is_seq_mean:0.9998148679733276 - rollout_corr/rollout_is_seq_std:0.0030276936013251543 - rollout_corr/rollout_is_seq_max:1.012966513633728 - rollout_corr/rollout_is_seq_min:0.9873403906822205 - rollout_corr/rollout_is_seq_max_deviation:0.012966513633728027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1056180871091783) - rollout_corr/training_log_ppl:np.float64(0.09563198406249285) - rollout_corr/kl:np.float64(0.0005883509425608224) - rollout_corr/k3_kl:np.float64(0.0005984544800219371) - rollout_corr/rollout_ppl:np.float64(1.1050527193583548) - rollout_corr/rollout_log_ppl:np.float64(0.09504363221367385) - rollout_corr/log_ppl_diff:np.float64(0.000588351848819002) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029859036067136913) - rollout_corr/log_ppl_diff_max:np.float64(0.025081709027290344) - rollout_corr/log_ppl_diff_min:np.float64(-0.022050082683563232) - rollout_corr/ppl_ratio:np.float64(1.0005988392513245) - rollout_corr/chi2_token:np.float64(0.0014067043084651232) - rollout_corr/chi2_seq:np.float64(0.052731548668816686) - actor/pg_loss:np.float64(-0.00016857311129570007) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0001265746056802186) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4780052602291107) - perf/mfu/actor:np.float64(0.06219252515776021) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.77994155883789) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:0 - critic/score/mean:0.33984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007633587811142206 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0007633587811142206 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:86.9921875 - response_length/max:1018.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.9921875 - response_length_non_aborted/max:1018.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:573.5625 - prompt_length/max:1024.0 - prompt_length/min:256.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.940005838871002e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3933531232178211) - timing_s/agent_loop/generate_sequences/max:np.float64(5.514592096209526) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9159768263270962) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.514592096209526) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:497 - timing_s/agent_loop/slowest/response_length:1018 - timing_s/gen:11.491664243862033 - timing_s/reward:0.07059245370328426 - timing_s/old_log_prob:2.4098829869180918 - timing_s/adv:0.11902144737541676 - timing_s/update_actor:9.850708981975913 - timing_s/step:24.02591587230563 - timing_s/stop_profile:0.00011333636939525604 - timing_per_token_ms/update_actor:0.05825306017655565 - timing_per_token_ms/adv:0.0007038441140578867 - timing_per_token_ms/gen:0.5160154577396512 - perf/total_num_tokens:169102 - perf/time_per_step:24.02591587230563 - perf/throughput:879.7895619190616 (TaskRunner pid=2186821) Training Progress: 97%|█████████▋| 97/100 [47:34<01:28, 29.54s/it] (TaskRunner pid=2186821) step:98 - global_seqlen/min:17149 - global_seqlen/max:36781 - global_seqlen/minmax_diff:19632 - global_seqlen/balanced_min:20694 - global_seqlen/balanced_max:27893 - global_seqlen/mean:22397.625 - actor/entropy:0.1040242463350296 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34714338183403015 - training/rollout_probs_diff_mean:0.0017863105749711394 - training/rollout_probs_diff_std:0.008350610733032227 - training/rollout_actor_probs_pearson_corr:0.998758852481842 - rollout_corr/rollout_is_mean:0.999933660030365 - rollout_corr/rollout_is_ratio_fraction_high:2.6740111934486777e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.022033580346033e-05 - rollout_corr/rollout_is_max:2.0221502780914307 - rollout_corr/rollout_is_min:0.2599737048149109 - rollout_corr/rollout_is_std:0.02273000404238701 - rollout_corr/rollout_is_eff_sample_size:0.9994834946454421 - rollout_corr/rollout_is_seq_mean:1.0001521110534668 - rollout_corr/rollout_is_seq_std:0.0030455931555479765 - rollout_corr/rollout_is_seq_max:1.0114482641220093 - rollout_corr/rollout_is_seq_min:0.9909152984619141 - rollout_corr/rollout_is_seq_max_deviation:0.011448264122009277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1604433851316571) - rollout_corr/training_log_ppl:np.float64(0.14228921985341003) - rollout_corr/kl:np.float64(-1.667551759254593e-05) - rollout_corr/k3_kl:np.float64(0.00044376492397635303) - rollout_corr/rollout_ppl:np.float64(1.1605023499578238) - rollout_corr/rollout_log_ppl:np.float64(0.1423058943655633) - rollout_corr/log_ppl_diff:np.float64(-1.6674512153258547e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.002513227216695668) - rollout_corr/log_ppl_diff_max:np.float64(0.008114248514175415) - rollout_corr/log_ppl_diff_min:np.float64(-0.012333065271377563) - rollout_corr/ppl_ratio:np.float64(0.9999894560314715) - rollout_corr/chi2_token:np.float64(0.0019017679151147604) - rollout_corr/chi2_seq:np.float64(0.056306053418666124) - actor/pg_loss:np.float64(8.791685104370117e-07) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.394094998540822e-05) - actor/ppo_kl:np.float64(-0.00015192375955308535) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.10722477734088898) - perf/mfu/actor:np.float64(0.061854402783942386) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.71384239196777) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:0 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006651603034697473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006651603034697473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:146.08203125 - response_length/max:8192.0 - response_length/min:22.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:146.08203125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:553.84375 - prompt_length/max:810.0 - prompt_length/min:318.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012129172682762146 - timing_s/agent_loop/generate_sequences/min:np.float64(0.35201247595250607) - timing_s/agent_loop/generate_sequences/max:np.float64(41.830545203760266) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1741379976738244) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(41.830545203760266) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:542 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:47.6993387080729 - timing_s/reward:0.07287946529686451 - timing_s/old_log_prob:2.554050227627158 - timing_s/adv:0.14231567829847336 - timing_s/update_actor:10.73162966966629 - timing_s/step:61.27866987325251 - timing_s/stop_profile:0.00014525838196277618 - timing_per_token_ms/update_actor:0.059892676509598064 - timing_per_token_ms/adv:0.0007942565243997597 - timing_per_token_ms/gen:1.2754856996035218 - perf/total_num_tokens:179181 - perf/time_per_step:61.27866987325251 - perf/throughput:365.5044250524166 (TaskRunner pid=2186821) Training Progress: 98%|█████████▊| 98/100 [48:35<01:18, 39.08s/it] (TaskRunner pid=2186821) step:99 - global_seqlen/min:18425 - global_seqlen/max:27041 - global_seqlen/minmax_diff:8616 - global_seqlen/balanced_min:21666 - global_seqlen/balanced_max:25545 - global_seqlen/mean:22203.75 - actor/entropy:0.12311642616987228 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.13173753023147583 - training/rollout_probs_diff_mean:0.0018586608348414302 - training/rollout_probs_diff_std:0.006972834002226591 - training/rollout_actor_probs_pearson_corr:0.9992296695709229 - rollout_corr/rollout_is_mean:1.0000172853469849 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.639548659324646 - rollout_corr/rollout_is_min:0.5505461692810059 - rollout_corr/rollout_is_std:0.022024136036634445 - rollout_corr/rollout_is_eff_sample_size:0.9995152916662099 - rollout_corr/rollout_is_seq_mean:1.0001254081726074 - rollout_corr/rollout_is_seq_std:0.0029176895041018724 - rollout_corr/rollout_is_seq_max:1.0127778053283691 - rollout_corr/rollout_is_seq_min:0.9914768934249878 - rollout_corr/rollout_is_seq_max_deviation:0.01277780532836914 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.122967625502497) - rollout_corr/training_log_ppl:np.float64(0.1121234093880048) - rollout_corr/kl:np.float64(6.719642799524195e-05) - rollout_corr/k3_kl:np.float64(0.00036489025642794104) - rollout_corr/rollout_ppl:np.float64(1.1229209606535733) - rollout_corr/rollout_log_ppl:np.float64(0.11205621204862837) - rollout_corr/log_ppl_diff:np.float64(6.719733937643468e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.002387066531809978) - rollout_corr/log_ppl_diff_max:np.float64(0.009327642619609833) - rollout_corr/log_ppl_diff_min:np.float64(-0.019913308322429657) - rollout_corr/ppl_ratio:np.float64(1.000073105096817) - rollout_corr/chi2_token:np.float64(0.0015559385064989328) - rollout_corr/chi2_seq:np.float64(0.08049377938732505) - actor/pg_loss:np.float64(-1.337006688117981e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002156986520276405) - actor/ppo_kl:np.float64(-3.9702664338392424e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12718498706817627) - perf/mfu/actor:np.float64(0.06278020355373397) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.79438400268555) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:0 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00021608588576782495 - critic/advantages/max:0.75 - critic/advantages/min:-0.25 - critic/returns/mean:-0.00021608588576782495 - critic/returns/max:0.75 - critic/returns/min:-0.25 - response_length/mean:85.8671875 - response_length/max:4627.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:85.8671875 - response_length_non_aborted/max:4627.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:608.0 - prompt_length/max:946.0 - prompt_length/min:305.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014101900160312653 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4242385867983103) - timing_s/agent_loop/generate_sequences/max:np.float64(22.121572881937027) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8758462116529699) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.121572881937027) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:580 - timing_s/agent_loop/slowest/response_length:4627 - timing_s/gen:27.840653158724308 - timing_s/reward:0.07200900465250015 - timing_s/old_log_prob:2.4137296471744776 - timing_s/adv:0.13017542101442814 - timing_s/update_actor:10.260624293237925 - timing_s/step:40.80330477654934 - timing_s/stop_profile:0.00011400878429412842 - timing_per_token_ms/update_actor:0.057764027997736445 - timing_per_token_ms/adv:0.000732845921378304 - timing_per_token_ms/gen:1.2665204785153448 - perf/total_num_tokens:177630 - perf/time_per_step:40.80330477654934 - perf/throughput:544.1654817322797 (TaskRunner pid=2186821) Training Progress: 99%|█████████▉| 99/100 [49:16<00:39, 39.61s/it] (TaskRunner pid=2186821) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2186821) validation generation end (TaskRunner pid=2186821) [prompt] user (TaskRunner pid=2186821) Your task is to answer the user's question using available tools. (TaskRunner pid=2186821) You have access to the following tools: (TaskRunner pid=2186821) Name: Axolotl (TaskRunner pid=2186821) Description: Collection of axolotl pictures and facts (TaskRunner pid=2186821) Documentation: (TaskRunner pid=2186821) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2186821) Parameters: {} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{url, source, description} (TaskRunner pid=2186821) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2186821) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2186821) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2186821) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2186821) Output: Successful response. (TaskRunner pid=2186821) - Format: application/json (TaskRunner pid=2186821) - Structure: Array[Object{fact, source}] (TaskRunner pid=2186821) (TaskRunner pid=2186821) Use the following format: (TaskRunner pid=2186821) Thought: you should always think about what to do (TaskRunner pid=2186821) Action: the action to take, should be one of the tool names. (TaskRunner pid=2186821) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2186821) (TaskRunner pid=2186821) Begin! (TaskRunner pid=2186821) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2186821) assistant (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) (TaskRunner pid=2186821) [response] Thought: I need to retrieve a random axolotl image. (TaskRunner pid=2186821) Action: getRandomAxolotlImage (TaskRunner pid=2186821) Action Input: {} (TaskRunner pid=2186821) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2186821) [score] 1.0 (TaskRunner pid=2186821) [acc] 1.0 (TaskRunner pid=2186821) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2186821) [incorrect_format] 0 (TaskRunner pid=2186821) [feedback] (TaskRunner pid=2186821) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2187230) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2186821) step:100 - global_seqlen/min:18523 - global_seqlen/max:25286 - global_seqlen/minmax_diff:6763 - global_seqlen/balanced_min:21035 - global_seqlen/balanced_max:21036 - global_seqlen/mean:21035.875 - actor/entropy:0.14357170462608337 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.12089437246322632 - training/rollout_probs_diff_mean:0.002164518227800727 - training/rollout_probs_diff_std:0.007261213846504688 - training/rollout_actor_probs_pearson_corr:0.9992616772651672 - rollout_corr/rollout_is_mean:0.9998654127120972 - rollout_corr/rollout_is_ratio_fraction_high:4.97240325785242e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2650070190429688 - rollout_corr/rollout_is_min:0.6138291954994202 - rollout_corr/rollout_is_std:0.02362494170665741 - rollout_corr/rollout_is_eff_sample_size:0.9994420543741424 - rollout_corr/rollout_is_seq_mean:0.9998652338981628 - rollout_corr/rollout_is_seq_std:0.003272402100265026 - rollout_corr/rollout_is_seq_max:1.0165464878082275 - rollout_corr/rollout_is_seq_min:0.9906290769577026 - rollout_corr/rollout_is_seq_max_deviation:0.01654648780822754 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1511951768770814) - rollout_corr/training_log_ppl:np.float64(0.1364561979426071) - rollout_corr/kl:np.float64(0.0005012150181968877) - rollout_corr/k3_kl:np.float64(0.00043782229743827283) - rollout_corr/rollout_ppl:np.float64(1.1506293234415352) - rollout_corr/rollout_log_ppl:np.float64(0.13595498194990796) - rollout_corr/log_ppl_diff:np.float64(0.0005012159926991444) - rollout_corr/log_ppl_abs_diff:np.float64(0.002873804867704166) - rollout_corr/log_ppl_diff_max:np.float64(0.017064914107322693) - rollout_corr/log_ppl_diff_min:np.float64(-0.018534213304519653) - rollout_corr/ppl_ratio:np.float64(1.0005107254255563) - rollout_corr/chi2_token:np.float64(0.0007952849846333265) - rollout_corr/chi2_seq:np.float64(0.05812919745221734) - actor/pg_loss:np.float64(-0.00013048574328422546) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002753048320300877) - actor/ppo_kl:np.float64(9.126732021513817e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23568445537239313) - perf/mfu/actor:np.float64(0.06166838756342609) - perf/max_memory_allocated_gb:np.float64(124.25863409042358) - perf/max_memory_reserved_gb:np.float64(130.046875) - perf/cpu_memory_used_gb:np.float64(105.7668685913086) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5689338235294118) - val-aux/tooluse/reward/std@16:np.float64(0.07216799159273436) - val-aux/tooluse/reward/best@2/mean:np.float64(0.599735294117647) - val-aux/tooluse/reward/best@2/std:np.float64(0.05681896001528811) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5376176470588235) - val-aux/tooluse/reward/worst@2/std:np.float64(0.065873083983826) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5681911764705883) - val-aux/tooluse/reward/maj@2/std:np.float64(0.07217664475658733) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6216029411764705) - val-aux/tooluse/reward/best@4/std:np.float64(0.03612363935746805) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5104117647058823) - val-aux/tooluse/reward/worst@4/std:np.float64(0.050276642601264465) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.571764705882353) - val-aux/tooluse/reward/maj@4/std:np.float64(0.06112092824371511) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6337205882352942) - val-aux/tooluse/reward/best@8/std:np.float64(0.021009683207312644) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.4913382352941176) - val-aux/tooluse/reward/worst@8/std:np.float64(0.03169698340721522) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5750294117647058) - val-aux/tooluse/reward/maj@8/std:np.float64(0.04699356851985658) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6409558823529412) - val-aux/tooluse/reward/best@16/std:np.float64(0.011314008550944105) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.47960294117647057) - val-aux/tooluse/reward/worst@16/std:np.float64(0.018056061970488427) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5770294117647059) - val-aux/tooluse/reward/maj@16/std:np.float64(0.03730316048999515) - val-aux/tooluse/score/mean@16:np.float64(0.5689338235294118) - val-aux/tooluse/score/std@16:np.float64(0.07216799159273436) - val-aux/tooluse/score/best@2/mean:np.float64(0.599735294117647) - val-aux/tooluse/score/best@2/std:np.float64(0.05681896001528811) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5376176470588235) - val-aux/tooluse/score/worst@2/std:np.float64(0.065873083983826) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5681911764705883) - val-aux/tooluse/score/maj@2/std:np.float64(0.07217664475658733) - val-aux/tooluse/score/best@4/mean:np.float64(0.6216029411764705) - val-aux/tooluse/score/best@4/std:np.float64(0.03612363935746805) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5104117647058823) - val-aux/tooluse/score/worst@4/std:np.float64(0.050276642601264465) - val-aux/tooluse/score/maj@4/mean:np.float64(0.571764705882353) - val-aux/tooluse/score/maj@4/std:np.float64(0.06112092824371511) - val-aux/tooluse/score/best@8/mean:np.float64(0.6337205882352942) - val-aux/tooluse/score/best@8/std:np.float64(0.021009683207312644) - val-aux/tooluse/score/worst@8/mean:np.float64(0.4913382352941176) - val-aux/tooluse/score/worst@8/std:np.float64(0.03169698340721522) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5750294117647058) - val-aux/tooluse/score/maj@8/std:np.float64(0.04699356851985658) - val-aux/tooluse/score/best@16/mean:np.float64(0.6409558823529412) - val-aux/tooluse/score/best@16/std:np.float64(0.011314008550944105) - val-aux/tooluse/score/worst@16/mean:np.float64(0.47960294117647057) - val-aux/tooluse/score/worst@16/std:np.float64(0.018056061970488427) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5770294117647059) - val-aux/tooluse/score/maj@16/std:np.float64(0.03730316048999515) - val-core/tooluse/acc/mean@16:np.float64(0.5689338235294118) - val-aux/tooluse/acc/std@16:np.float64(0.07216799159273436) - val-aux/tooluse/acc/best@2/mean:np.float64(0.599735294117647) - val-aux/tooluse/acc/best@2/std:np.float64(0.05681896001528811) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5376176470588235) - val-aux/tooluse/acc/worst@2/std:np.float64(0.065873083983826) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5681911764705883) - val-aux/tooluse/acc/maj@2/std:np.float64(0.07217664475658733) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6216029411764705) - val-aux/tooluse/acc/best@4/std:np.float64(0.03612363935746805) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5104117647058823) - val-aux/tooluse/acc/worst@4/std:np.float64(0.050276642601264465) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.571764705882353) - val-aux/tooluse/acc/maj@4/std:np.float64(0.06112092824371511) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6337205882352942) - val-aux/tooluse/acc/best@8/std:np.float64(0.021009683207312644) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.4913382352941176) - val-aux/tooluse/acc/worst@8/std:np.float64(0.03169698340721522) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5750294117647058) - val-aux/tooluse/acc/maj@8/std:np.float64(0.04699356851985658) - val-core/tooluse/acc/best@16/mean:np.float64(0.6409558823529412) - val-core/tooluse/acc/best@16/std:np.float64(0.011314008550944105) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.47960294117647057) - val-aux/tooluse/acc/worst@16/std:np.float64(0.018056061970488427) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5770294117647059) - val-core/tooluse/acc/maj@16/std:np.float64(0.03730316048999515) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:0 - critic/score/mean:0.19140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.19140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0050159115344285965 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0050159115344285965 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:78.55859375 - response_length/max:403.0 - response_length/min:22.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:78.55859375 - response_length_non_aborted/max:403.0 - response_length_non_aborted/min:22.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:578.8125 - prompt_length/max:1063.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012426264584064484 - timing_s/agent_loop/generate_sequences/min:np.float64(0.25598333589732647) - timing_s/agent_loop/generate_sequences/max:np.float64(2.4420138355344534) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7321565171878319) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.4420138355344534) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:533 - timing_s/agent_loop/slowest/response_length:403 - timing_s/gen:8.107063824310899 - timing_s/reward:0.07088929787278175 - timing_s/old_log_prob:2.385152492672205 - timing_s/adv:0.1153049636632204 - timing_s/update_actor:9.803228594362736 - timing_s/step:20.5681207254529 - timing_s/testing:19.794103000313044 - timing_s/save_checkpoint:6.70261013135314 - timing_s/stop_profile:7.58729875087738e-05 - timing_per_token_ms/update_actor:0.058253035554515414 - timing_per_token_ms/adv:0.0006851685731115321 - timing_per_token_ms/gen:0.40311589798174624 - perf/total_num_tokens:168287 - perf/time_per_step:20.5681207254529 - perf/throughput:1022.7417118360385 (TaskRunner pid=2186821) ("Final validation metrics: {'val-aux/tooluse/reward/mean@16': " (TaskRunner pid=2186821) "np.float64(0.5689338235294118), 'val-aux/tooluse/reward/std@16': " (TaskRunner pid=2186821) "np.float64(0.07216799159273436), 'val-aux/tooluse/reward/best@2/mean': " (TaskRunner pid=2186821) "np.float64(0.599735294117647), 'val-aux/tooluse/reward/best@2/std': " (TaskRunner pid=2186821) "np.float64(0.05681896001528811), 'val-aux/tooluse/reward/worst@2/mean': " (TaskRunner pid=2186821) "np.float64(0.5376176470588235), 'val-aux/tooluse/reward/worst@2/std': " (TaskRunner pid=2186821) "np.float64(0.065873083983826), 'val-aux/tooluse/reward/maj@2/mean': " (TaskRunner pid=2186821) "np.float64(0.5681911764705883), 'val-aux/tooluse/reward/maj@2/std': " (TaskRunner pid=2186821) "np.float64(0.07217664475658733), 'val-aux/tooluse/reward/best@4/mean': " (TaskRunner pid=2186821) "np.float64(0.6216029411764705), 'val-aux/tooluse/reward/best@4/std': " (TaskRunner pid=2186821) "np.float64(0.03612363935746805), 'val-aux/tooluse/reward/worst@4/mean': " (TaskRunner pid=2186821) "np.float64(0.5104117647058823), 'val-aux/tooluse/reward/worst@4/std': " (TaskRunner pid=2186821) "np.float64(0.050276642601264465), 'val-aux/tooluse/reward/maj@4/mean': " (TaskRunner pid=2186821) "np.float64(0.571764705882353), 'val-aux/tooluse/reward/maj@4/std': " (TaskRunner pid=2186821) "np.float64(0.06112092824371511), 'val-aux/tooluse/reward/best@8/mean': " (TaskRunner pid=2186821) "np.float64(0.6337205882352942), 'val-aux/tooluse/reward/best@8/std': " (TaskRunner pid=2186821) "np.float64(0.021009683207312644), 'val-aux/tooluse/reward/worst@8/mean': " (TaskRunner pid=2186821) "np.float64(0.4913382352941176), 'val-aux/tooluse/reward/worst@8/std': " (TaskRunner pid=2186821) "np.float64(0.03169698340721522), 'val-aux/tooluse/reward/maj@8/mean': " (TaskRunner pid=2186821) "np.float64(0.5750294117647058), 'val-aux/tooluse/reward/maj@8/std': " (TaskRunner pid=2186821) "np.float64(0.04699356851985658), 'val-aux/tooluse/reward/best@16/mean': " (TaskRunner pid=2186821) "np.float64(0.6409558823529412), 'val-aux/tooluse/reward/best@16/std': " (TaskRunner pid=2186821) "np.float64(0.011314008550944105), 'val-aux/tooluse/reward/worst@16/mean': " (TaskRunner pid=2186821) "np.float64(0.47960294117647057), 'val-aux/tooluse/reward/worst@16/std': " (TaskRunner pid=2186821) "np.float64(0.018056061970488427), 'val-aux/tooluse/reward/maj@16/mean': " (TaskRunner pid=2186821) "np.float64(0.5770294117647059), 'val-aux/tooluse/reward/maj@16/std': " (TaskRunner pid=2186821) "np.float64(0.03730316048999515), 'val-aux/tooluse/score/mean@16': " (TaskRunner pid=2186821) "np.float64(0.5689338235294118), 'val-aux/tooluse/score/std@16': " (TaskRunner pid=2186821) "np.float64(0.07216799159273436), 'val-aux/tooluse/score/best@2/mean': " (TaskRunner pid=2186821) "np.float64(0.599735294117647), 'val-aux/tooluse/score/best@2/std': " (TaskRunner pid=2186821) "np.float64(0.05681896001528811), 'val-aux/tooluse/score/worst@2/mean': " (TaskRunner pid=2186821) "np.float64(0.5376176470588235), 'val-aux/tooluse/score/worst@2/std': " (TaskRunner pid=2186821) "np.float64(0.065873083983826), 'val-aux/tooluse/score/maj@2/mean': " (TaskRunner pid=2186821) "np.float64(0.5681911764705883), 'val-aux/tooluse/score/maj@2/std': " (TaskRunner pid=2186821) "np.float64(0.07217664475658733), 'val-aux/tooluse/score/best@4/mean': " (TaskRunner pid=2186821) "np.float64(0.6216029411764705), 'val-aux/tooluse/score/best@4/std': " (TaskRunner pid=2186821) "np.float64(0.03612363935746805), 'val-aux/tooluse/score/worst@4/mean': " (TaskRunner pid=2186821) "np.float64(0.5104117647058823), 'val-aux/tooluse/score/worst@4/std': " (TaskRunner pid=2186821) "np.float64(0.050276642601264465), 'val-aux/tooluse/score/maj@4/mean': " (TaskRunner pid=2186821) "np.float64(0.571764705882353), 'val-aux/tooluse/score/maj@4/std': " (TaskRunner pid=2186821) "np.float64(0.06112092824371511), 'val-aux/tooluse/score/best@8/mean': " (TaskRunner pid=2186821) "np.float64(0.6337205882352942), 'val-aux/tooluse/score/best@8/std': " (TaskRunner pid=2186821) "np.float64(0.021009683207312644), 'val-aux/tooluse/score/worst@8/mean': " (TaskRunner pid=2186821) "np.float64(0.4913382352941176), 'val-aux/tooluse/score/worst@8/std': " (TaskRunner pid=2186821) "np.float64(0.03169698340721522), 'val-aux/tooluse/score/maj@8/mean': " (TaskRunner pid=2186821) "np.float64(0.5750294117647058), 'val-aux/tooluse/score/maj@8/std': " (TaskRunner pid=2186821) "np.float64(0.04699356851985658), 'val-aux/tooluse/score/best@16/mean': " (TaskRunner pid=2186821) "np.float64(0.6409558823529412), 'val-aux/tooluse/score/best@16/std': " (TaskRunner pid=2186821) "np.float64(0.011314008550944105), 'val-aux/tooluse/score/worst@16/mean': " (TaskRunner pid=2186821) "np.float64(0.47960294117647057), 'val-aux/tooluse/score/worst@16/std': " (TaskRunner pid=2186821) "np.float64(0.018056061970488427), 'val-aux/tooluse/score/maj@16/mean': " (TaskRunner pid=2186821) "np.float64(0.5770294117647059), 'val-aux/tooluse/score/maj@16/std': " (TaskRunner pid=2186821) "np.float64(0.03730316048999515), 'val-core/tooluse/acc/mean@16': " (TaskRunner pid=2186821) "np.float64(0.5689338235294118), 'val-aux/tooluse/acc/std@16': " (TaskRunner pid=2186821) "np.float64(0.07216799159273436), 'val-aux/tooluse/acc/best@2/mean': " (TaskRunner pid=2186821) "np.float64(0.599735294117647), 'val-aux/tooluse/acc/best@2/std': " (TaskRunner pid=2186821) "np.float64(0.05681896001528811), 'val-aux/tooluse/acc/worst@2/mean': " (TaskRunner pid=2186821) "np.float64(0.5376176470588235), 'val-aux/tooluse/acc/worst@2/std': " (TaskRunner pid=2186821) "np.float64(0.065873083983826), 'val-aux/tooluse/acc/maj@2/mean': " (TaskRunner pid=2186821) "np.float64(0.5681911764705883), 'val-aux/tooluse/acc/maj@2/std': " (TaskRunner pid=2186821) "np.float64(0.07217664475658733), 'val-aux/tooluse/acc/best@4/mean': " (TaskRunner pid=2186821) "np.float64(0.6216029411764705), 'val-aux/tooluse/acc/best@4/std': " (TaskRunner pid=2186821) "np.float64(0.03612363935746805), 'val-aux/tooluse/acc/worst@4/mean': " (TaskRunner pid=2186821) "np.float64(0.5104117647058823), 'val-aux/tooluse/acc/worst@4/std': " (TaskRunner pid=2186821) "np.float64(0.050276642601264465), 'val-aux/tooluse/acc/maj@4/mean': " (TaskRunner pid=2186821) "np.float64(0.571764705882353), 'val-aux/tooluse/acc/maj@4/std': " (TaskRunner pid=2186821) "np.float64(0.06112092824371511), 'val-aux/tooluse/acc/best@8/mean': " (TaskRunner pid=2186821) "np.float64(0.6337205882352942), 'val-aux/tooluse/acc/best@8/std': " (TaskRunner pid=2186821) "np.float64(0.021009683207312644), 'val-aux/tooluse/acc/worst@8/mean': " (TaskRunner pid=2186821) "np.float64(0.4913382352941176), 'val-aux/tooluse/acc/worst@8/std': " (TaskRunner pid=2186821) "np.float64(0.03169698340721522), 'val-aux/tooluse/acc/maj@8/mean': " (TaskRunner pid=2186821) "np.float64(0.5750294117647058), 'val-aux/tooluse/acc/maj@8/std': " (TaskRunner pid=2186821) "np.float64(0.04699356851985658), 'val-core/tooluse/acc/best@16/mean': " (TaskRunner pid=2186821) "np.float64(0.6409558823529412), 'val-core/tooluse/acc/best@16/std': " (TaskRunner pid=2186821) "np.float64(0.011314008550944105), 'val-aux/tooluse/acc/worst@16/mean': " (TaskRunner pid=2186821) "np.float64(0.47960294117647057), 'val-aux/tooluse/acc/worst@16/std': " (TaskRunner pid=2186821) "np.float64(0.018056061970488427), 'val-core/tooluse/acc/maj@16/mean': " (TaskRunner pid=2186821) "np.float64(0.5770294117647059), 'val-core/tooluse/acc/maj@16/std': " (TaskRunner pid=2186821) "np.float64(0.03730316048999515), 'val-aux/tooluse/incorrect_format/mean@16': " (TaskRunner pid=2186821) "np.float64(0.0), 'val-aux/tooluse/incorrect_format/std@16': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/best@2/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/best@2/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/worst@2/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/worst@2/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/maj@2/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/maj@2/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/best@4/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/worst@4/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/worst@4/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/maj@4/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/maj@4/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/best@8/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/worst@8/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/worst@8/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/maj@8/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/best@16/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/worst@16/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/worst@16/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/maj@16/mean': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/tooluse/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2186821) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2186821) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2186821) Training Progress: 100%|██████████| 100/100 [50:03<00:00, 41.87s/it] Training Progress: 100%|██████████| 100/100 [50:03<00:00, 30.04s/it] (TaskRunner pid=2186821) wandb: updating run metadata; uploading output.log; uploading wandb-summary.json (TaskRunner pid=2186821) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2186821) wandb: uploading history steps 98-99, summary, console lines 577-683 (TaskRunner pid=2186821) wandb: (TaskRunner pid=2186821) wandb: Run history: (TaskRunner pid=2186821) wandb: actor/entropy █▆▆▆▅▅▆█▃▂▂▁▄▁▂▂▂▃▄▄▅▃█▅▇▅▃▃▃▄▆▆▆▄▃▅▅▃▆▆ (TaskRunner pid=2186821) wandb: actor/grad_norm ▁▅▄▅▄▅▄▃▄▃▄▁▅▅▃▄▅▅▃▃▅▂▄▅▃▅▃▆▄▄▂▃▃▂▂▄█▄█▃ (TaskRunner pid=2186821) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2186821) wandb: actor/lr ▁▂▂▄▅███████████████████████████████████ (TaskRunner pid=2186821) wandb: actor/pg_clipfrac ▁▄▃▂▂▂▄▁▃▂▁▆▁▃▂▄▃▅█▂▂▂▁▄▆▃▃▇▅▄▇▆▅▂▁▁▄▃▃▃ (TaskRunner pid=2186821) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆▂▁▁▁▁ (TaskRunner pid=2186821) wandb: actor/pg_loss ▄▄▆▅▄▃▄▅▅▆▆▃▃▄▅▂▄█▃▄▄▅▆▃▅▆▄▃▃▅▄▅▃▅▄▄▄▁▄▄ (TaskRunner pid=2186821) wandb: actor/ppo_kl ▂▂▂▂▁▂▂▃▂▂█▂▄▃▂▁▁▃▂▂▃▂▂▂▂▂▂▂▃▃▃▂▂▂▂▁▁▁▃▂ (TaskRunner pid=2186821) wandb: critic/advantages/max █▆█▇███▂▁▅▇▅█▇███▄█▇█▇█▃███▇█▁▇▆▆▄▂█▄█▆█ (TaskRunner pid=2186821) wandb: critic/advantages/mean ▆▇▆▅▆█▁▇▆▆▅▅▆▆▆▆▁▆▆▇▄▅▆▃▆▆▆▅▅▅▆█▆▆▆▆▆▅▄█ (TaskRunner pid=2186821) wandb: +204 ... (TaskRunner pid=2186821) wandb: (TaskRunner pid=2186821) wandb: Run summary: (TaskRunner pid=2186821) wandb: actor/entropy 0.14357 (TaskRunner pid=2186821) wandb: actor/grad_norm 0.23568 (TaskRunner pid=2186821) wandb: actor/kl_loss 0 (TaskRunner pid=2186821) wandb: actor/lr 0.0 (TaskRunner pid=2186821) wandb: actor/pg_clipfrac 0.00028 (TaskRunner pid=2186821) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2186821) wandb: actor/pg_loss -0.00013 (TaskRunner pid=2186821) wandb: actor/ppo_kl 9e-05 (TaskRunner pid=2186821) wandb: critic/advantages/max 0.875 (TaskRunner pid=2186821) wandb: critic/advantages/mean 0.00502 (TaskRunner pid=2186821) wandb: +204 ... (TaskRunner pid=2186821) wandb: (TaskRunner pid=2186821) wandb: 🚀 View run qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/dx38nnn7 (TaskRunner pid=2186821) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2186821) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2186821) wandb: Find logs at: ./wandb/run-20260702_145459-dx38nnn7/logs main_ppo exit code: 0 [2026-07-02 15:45:10] Finished tooluse grpo [2026-07-02 15:45:10] Starting tooluse rlsd_tr Experiment: qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: tooluse Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/tooluse/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/tooluse/test.parquet Ray tmp: /tmp/ray_q3g_tooluse_rlsd_tr_Qwen3_4B 2026-07-02 15:45:11,788 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/tooluse Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/tooluse/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/tooluse/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_tooluse_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/tooluse +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_tooluse_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/tooluse', 'EXPERIMENT': 'qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 15:45:21,567 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2204243) TaskRunner hostname: mole-workspace-rw, PID: 2204243 (TaskRunner pid=2204243) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2204243) 'calculate_entropy': False, (TaskRunner pid=2204243) 'calculate_sum_pi_squared': False, (TaskRunner pid=2204243) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2204243) 'async_save': False, (TaskRunner pid=2204243) 'load_contents': ['model', (TaskRunner pid=2204243) 'optimizer', (TaskRunner pid=2204243) 'extra'], (TaskRunner pid=2204243) 'save_contents': ['model', (TaskRunner pid=2204243) 'optimizer', (TaskRunner pid=2204243) 'extra']}, (TaskRunner pid=2204243) 'clip_ratio': 0.2, (TaskRunner pid=2204243) 'clip_ratio_c': 3.0, (TaskRunner pid=2204243) 'clip_ratio_high': 0.28, (TaskRunner pid=2204243) 'clip_ratio_low': 0.2, (TaskRunner pid=2204243) 'data_loader_seed': 42, (TaskRunner pid=2204243) 'entropy_checkpointing': False, (TaskRunner pid=2204243) 'entropy_coeff': 0, (TaskRunner pid=2204243) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2204243) 'freeze_vision_tower': False, (TaskRunner pid=2204243) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2204243) 'dtype': 'bfloat16', (TaskRunner pid=2204243) 'entropy_checkpointing': False, (TaskRunner pid=2204243) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2204243) 'forward_only': False, (TaskRunner pid=2204243) 'forward_prefetch': False, (TaskRunner pid=2204243) 'fsdp_size': -1, (TaskRunner pid=2204243) 'full_determinism': False, (TaskRunner pid=2204243) 'model_dtype': 'fp32', (TaskRunner pid=2204243) 'offload_policy': False, (TaskRunner pid=2204243) 'optimizer_offload': False, (TaskRunner pid=2204243) 'param_offload': False, (TaskRunner pid=2204243) 'reshard_after_forward': True, (TaskRunner pid=2204243) 'seed': 42, (TaskRunner pid=2204243) 'strategy': 'fsdp', (TaskRunner pid=2204243) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2204243) 'use_orig_params': False, (TaskRunner pid=2204243) 'use_torch_compile': True, (TaskRunner pid=2204243) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2204243) 'grad_clip': 1.0, (TaskRunner pid=2204243) 'kl_loss_coef': 0.001, (TaskRunner pid=2204243) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2204243) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2204243) 'loss_scale_factor': None, (TaskRunner pid=2204243) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2204243) 'betas': [0.9, 0.999], (TaskRunner pid=2204243) 'clip_grad': 1.0, (TaskRunner pid=2204243) 'lr': 1e-06, (TaskRunner pid=2204243) 'lr_scheduler_type': 'constant', (TaskRunner pid=2204243) 'lr_warmup_steps': 10, (TaskRunner pid=2204243) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2204243) 'min_lr_ratio': 0.0, (TaskRunner pid=2204243) 'num_cycles': 0.5, (TaskRunner pid=2204243) 'optimizer': 'AdamW', (TaskRunner pid=2204243) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2204243) 'override_optimizer_config': None, (TaskRunner pid=2204243) 'total_training_steps': -1, (TaskRunner pid=2204243) 'warmup_style': None, (TaskRunner pid=2204243) 'weight_decay': 0.01}, (TaskRunner pid=2204243) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2204243) 'clip_cov_lb': 1.0, (TaskRunner pid=2204243) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2204243) 'clip_cov_ub': 5.0, (TaskRunner pid=2204243) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2204243) 'loss_mode': 'rlsd', (TaskRunner pid=2204243) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2204243) 'ppo_epochs': 1, (TaskRunner pid=2204243) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2204243) 'ppo_micro_batch_size': None, (TaskRunner pid=2204243) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2204243) 'ppo_mini_batch_size': 8, (TaskRunner pid=2204243) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2204243) 'all_ranks': False, (TaskRunner pid=2204243) 'enable': False, (TaskRunner pid=2204243) 'ranks': [], (TaskRunner pid=2204243) 'save_path': 'outputs/profile', (TaskRunner pid=2204243) 'tool': None, (TaskRunner pid=2204243) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2204243) 'analysis': True, (TaskRunner pid=2204243) 'contents': [], (TaskRunner pid=2204243) 'discrete': False, (TaskRunner pid=2204243) 'level': 'level0'}, (TaskRunner pid=2204243) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2204243) 'discrete': False}, (TaskRunner pid=2204243) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2204243) 'step_end': None, (TaskRunner pid=2204243) 'step_start': 0}, (TaskRunner pid=2204243) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2204243) 'stack_depth': 32, (TaskRunner pid=2204243) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2204243) 'rollout_n': 8, (TaskRunner pid=2204243) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2204243) 'mode': 'disabled', (TaskRunner pid=2204243) 'record_file': None, (TaskRunner pid=2204243) 'replay_file': None}, (TaskRunner pid=2204243) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2204243) 'alpha': 0.5, (TaskRunner pid=2204243) 'distillation_add_tail': True, (TaskRunner pid=2204243) 'distillation_topk': 100, (TaskRunner pid=2204243) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2204243) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2204243) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2204243) 'enable': False, (TaskRunner pid=2204243) 'loss_weight': 0.0, (TaskRunner pid=2204243) 'mask': 'all'}, (TaskRunner pid=2204243) 'feedback_template': '\n' (TaskRunner pid=2204243) 'The ' (TaskRunner pid=2204243) 'following ' (TaskRunner pid=2204243) 'is ' (TaskRunner pid=2204243) 'feedback ' (TaskRunner pid=2204243) 'from ' (TaskRunner pid=2204243) 'your ' (TaskRunner pid=2204243) 'unsuccessful ' (TaskRunner pid=2204243) 'earlier ' (TaskRunner pid=2204243) 'attempt:\n' (TaskRunner pid=2204243) '\n' (TaskRunner pid=2204243) '{feedback_raw}', (TaskRunner pid=2204243) 'full_logit_distillation': True, (TaskRunner pid=2204243) 'include_environment_feedback': True, (TaskRunner pid=2204243) 'is_clip': 2, (TaskRunner pid=2204243) 'max_reprompt_len': 10240, (TaskRunner pid=2204243) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2204243) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2204243) '\n' (TaskRunner pid=2204243) 'Correctly ' (TaskRunner pid=2204243) 'solve ' (TaskRunner pid=2204243) 'the ' (TaskRunner pid=2204243) 'original ' (TaskRunner pid=2204243) 'question.', (TaskRunner pid=2204243) 'reprompt_truncation': 'right', (TaskRunner pid=2204243) 'solution_template': '\n' (TaskRunner pid=2204243) 'Correct ' (TaskRunner pid=2204243) 'solution:\n' (TaskRunner pid=2204243) '\n' (TaskRunner pid=2204243) '{successful_previous_attempt}', (TaskRunner pid=2204243) 'srpo_dynamic_weighting': False, (TaskRunner pid=2204243) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2204243) 'success_reward_threshold': 0.5, (TaskRunner pid=2204243) 'teacher_regularization': 'trust-region', (TaskRunner pid=2204243) 'teacher_update_rate': 0.1, (TaskRunner pid=2204243) 'token_reweight_decay_steps': 0, (TaskRunner pid=2204243) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2204243) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2204243) 'shuffle': False, (TaskRunner pid=2204243) 'strategy': 'fsdp', (TaskRunner pid=2204243) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2204243) 'tau_neg': 1.05, (TaskRunner pid=2204243) 'tau_pos': 1.0, (TaskRunner pid=2204243) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2204243) 'use_dynamic_bsz': False, (TaskRunner pid=2204243) 'use_fused_kernels': False, (TaskRunner pid=2204243) 'use_kl_loss': False, (TaskRunner pid=2204243) 'use_prefix_grouper': False, (TaskRunner pid=2204243) 'use_remove_padding': True, (TaskRunner pid=2204243) 'use_torch_compile': True}, (TaskRunner pid=2204243) 'hybrid_engine': True, (TaskRunner pid=2204243) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2204243) 'custom_chat_template': None, (TaskRunner pid=2204243) 'enable_activation_offload': False, (TaskRunner pid=2204243) 'enable_gradient_checkpointing': True, (TaskRunner pid=2204243) 'exclude_modules': None, (TaskRunner pid=2204243) 'external_lib': None, (TaskRunner pid=2204243) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2204243) 'hf_config_path': None, (TaskRunner pid=2204243) 'lora_adapter_path': None, (TaskRunner pid=2204243) 'lora_alpha': 16, (TaskRunner pid=2204243) 'lora_rank': 0, (TaskRunner pid=2204243) 'override_config': {}, (TaskRunner pid=2204243) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2204243) 'target_modules': 'all-linear', (TaskRunner pid=2204243) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2204243) 'num_shards': 4}, (TaskRunner pid=2204243) 'tokenizer_path': None, (TaskRunner pid=2204243) 'trust_remote_code': True, (TaskRunner pid=2204243) 'use_fused_kernels': False, (TaskRunner pid=2204243) 'use_liger': False, (TaskRunner pid=2204243) 'use_remove_padding': True, (TaskRunner pid=2204243) 'use_shm': False}, (TaskRunner pid=2204243) 'nccl_timeout': 600, (TaskRunner pid=2204243) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2204243) 'entropy_checkpointing': False, (TaskRunner pid=2204243) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2204243) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2204243) 'dtype': 'bfloat16', (TaskRunner pid=2204243) 'entropy_checkpointing': False, (TaskRunner pid=2204243) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2204243) 'forward_only': True, (TaskRunner pid=2204243) 'forward_prefetch': False, (TaskRunner pid=2204243) 'fsdp_size': -1, (TaskRunner pid=2204243) 'full_determinism': False, (TaskRunner pid=2204243) 'model_dtype': 'fp32', (TaskRunner pid=2204243) 'offload_policy': False, (TaskRunner pid=2204243) 'optimizer_offload': False, (TaskRunner pid=2204243) 'param_offload': False, (TaskRunner pid=2204243) 'reshard_after_forward': True, (TaskRunner pid=2204243) 'seed': 42, (TaskRunner pid=2204243) 'strategy': 'fsdp', (TaskRunner pid=2204243) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2204243) 'use_orig_params': False, (TaskRunner pid=2204243) 'use_torch_compile': True, (TaskRunner pid=2204243) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2204243) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2204243) 'log_prob_micro_batch_size': None, (TaskRunner pid=2204243) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2204243) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2204243) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2204243) 'all_ranks': False, (TaskRunner pid=2204243) 'enable': False, (TaskRunner pid=2204243) 'ranks': [], (TaskRunner pid=2204243) 'save_path': 'outputs/profile', (TaskRunner pid=2204243) 'tool': None, (TaskRunner pid=2204243) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2204243) 'analysis': True, (TaskRunner pid=2204243) 'contents': [], (TaskRunner pid=2204243) 'discrete': False, (TaskRunner pid=2204243) 'level': 'level0'}, (TaskRunner pid=2204243) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2204243) 'discrete': False}, (TaskRunner pid=2204243) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2204243) 'step_end': None, (TaskRunner pid=2204243) 'step_start': 0}, (TaskRunner pid=2204243) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2204243) 'stack_depth': 32, (TaskRunner pid=2204243) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2204243) 'rollout_n': 8, (TaskRunner pid=2204243) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2204243) 'mode': 'disabled', (TaskRunner pid=2204243) 'record_file': None, (TaskRunner pid=2204243) 'replay_file': None}, (TaskRunner pid=2204243) 'strategy': 'fsdp', (TaskRunner pid=2204243) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2204243) 'use_torch_compile': True}, (TaskRunner pid=2204243) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2204243) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2204243) 'agent_loop_config_path': None, (TaskRunner pid=2204243) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2204243) 'name': None, (TaskRunner pid=2204243) 'path': None}, (TaskRunner pid=2204243) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2204243) 'num_workers': 8}, (TaskRunner pid=2204243) 'calculate_log_probs': True, (TaskRunner pid=2204243) 'cudagraph_capture_sizes': None, (TaskRunner pid=2204243) 'data_parallel_size': 1, (TaskRunner pid=2204243) 'disable_log_stats': True, (TaskRunner pid=2204243) 'do_sample': True, (TaskRunner pid=2204243) 'dtype': 'bfloat16', (TaskRunner pid=2204243) 'enable_chunked_prefill': True, (TaskRunner pid=2204243) 'enable_prefix_caching': True, (TaskRunner pid=2204243) 'enable_rollout_routing_replay': False, (TaskRunner pid=2204243) 'enforce_eager': False, (TaskRunner pid=2204243) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2204243) 'expert_parallel_size': 1, (TaskRunner pid=2204243) 'free_cache_engine': True, (TaskRunner pid=2204243) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2204243) 'ignore_eos': False, (TaskRunner pid=2204243) 'layered_summon': False, (TaskRunner pid=2204243) 'load_format': 'dummy', (TaskRunner pid=2204243) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2204243) 'log_prob_micro_batch_size': None, (TaskRunner pid=2204243) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2204243) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2204243) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2204243) 'max_model_len': 10240, (TaskRunner pid=2204243) 'max_num_batched_tokens': 10240, (TaskRunner pid=2204243) 'max_num_seqs': 1024, (TaskRunner pid=2204243) 'mode': 'async', (TaskRunner pid=2204243) 'multi_stage_wake_up': False, (TaskRunner pid=2204243) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2204243) 'enable': False, (TaskRunner pid=2204243) 'format': 'hermes', (TaskRunner pid=2204243) 'interaction_config_path': None, (TaskRunner pid=2204243) 'max_assistant_turns': None, (TaskRunner pid=2204243) 'max_parallel_calls': 1, (TaskRunner pid=2204243) 'max_tool_response_length': 256, (TaskRunner pid=2204243) 'max_user_turns': None, (TaskRunner pid=2204243) 'num_repeat_rollouts': None, (TaskRunner pid=2204243) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2204243) 'tool_config_path': None, (TaskRunner pid=2204243) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2204243) 'use_inference_chat_template': False}, (TaskRunner pid=2204243) 'n': 8, (TaskRunner pid=2204243) 'name': 'vllm', (TaskRunner pid=2204243) 'over_sample_rate': 0, (TaskRunner pid=2204243) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2204243) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2204243) 'all_ranks': False, (TaskRunner pid=2204243) 'enable': False, (TaskRunner pid=2204243) 'ranks': [], (TaskRunner pid=2204243) 'save_path': 'outputs/profile', (TaskRunner pid=2204243) 'tool': None, (TaskRunner pid=2204243) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2204243) 'analysis': True, (TaskRunner pid=2204243) 'contents': [], (TaskRunner pid=2204243) 'discrete': False, (TaskRunner pid=2204243) 'level': 'level0'}, (TaskRunner pid=2204243) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2204243) 'discrete': False}, (TaskRunner pid=2204243) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2204243) 'step_end': None, (TaskRunner pid=2204243) 'step_start': 0}, (TaskRunner pid=2204243) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2204243) 'stack_depth': 32, (TaskRunner pid=2204243) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2204243) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2204243) 'enable': False, (TaskRunner pid=2204243) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2204243) 'port': 9090, (TaskRunner pid=2204243) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2204243) 'prompt_length': 2048, (TaskRunner pid=2204243) 'quantization': None, (TaskRunner pid=2204243) 'quantization_config_file': None, (TaskRunner pid=2204243) 'response_length': 8192, (TaskRunner pid=2204243) 'scheduling_policy': 'fcfs', (TaskRunner pid=2204243) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2204243) 'skip_rollout': False, (TaskRunner pid=2204243) 'skip_tokenizer_init': True, (TaskRunner pid=2204243) 'temperature': 1.0, (TaskRunner pid=2204243) 'tensor_model_parallel_size': 2, (TaskRunner pid=2204243) 'top_k': -1, (TaskRunner pid=2204243) 'top_p': 1.0, (TaskRunner pid=2204243) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2204243) 'backend': None, (TaskRunner pid=2204243) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2204243) 'token2text': False}, (TaskRunner pid=2204243) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2204243) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2204243) 'do_sample': True, (TaskRunner pid=2204243) 'n': 16, (TaskRunner pid=2204243) 'temperature': 0.6, (TaskRunner pid=2204243) 'top_k': -1, (TaskRunner pid=2204243) 'top_p': 0.95}}}, (TaskRunner pid=2204243) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2204243) 'adv_estimator': 'grpo', (TaskRunner pid=2204243) 'gamma': 1.0, (TaskRunner pid=2204243) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2204243) 'horizon': 10000, (TaskRunner pid=2204243) 'kl_coef': 0.001, (TaskRunner pid=2204243) 'target_kl': 0.1, (TaskRunner pid=2204243) 'type': 'fixed'}, (TaskRunner pid=2204243) 'kl_penalty': 'kl', (TaskRunner pid=2204243) 'lam': 1.0, (TaskRunner pid=2204243) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2204243) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2204243) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2204243) 'loss_type': 'ppo_clip', (TaskRunner pid=2204243) 'rollout_is': 'token', (TaskRunner pid=2204243) 'rollout_is_batch_normalize': False, (TaskRunner pid=2204243) 'rollout_is_threshold': 2.0, (TaskRunner pid=2204243) 'rollout_rs': None, (TaskRunner pid=2204243) 'rollout_rs_threshold': None}, (TaskRunner pid=2204243) 'use_kl_in_reward': False, (TaskRunner pid=2204243) 'use_pf_ppo': False}, (TaskRunner pid=2204243) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2204243) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2204243) 'async_save': False, (TaskRunner pid=2204243) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2204243) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2204243) 'cliprange_value': 0.5, (TaskRunner pid=2204243) 'data_loader_seed': 42, (TaskRunner pid=2204243) 'enable': None, (TaskRunner pid=2204243) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2204243) 'forward_micro_batch_size': None, (TaskRunner pid=2204243) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2204243) 'grad_clip': 1.0, (TaskRunner pid=2204243) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2204243) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2204243) 'enable_activation_offload': False, (TaskRunner pid=2204243) 'enable_gradient_checkpointing': True, (TaskRunner pid=2204243) 'external_lib': None, (TaskRunner pid=2204243) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2204243) 'dtype': 'bfloat16', (TaskRunner pid=2204243) 'entropy_checkpointing': False, (TaskRunner pid=2204243) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2204243) 'forward_only': False, (TaskRunner pid=2204243) 'forward_prefetch': False, (TaskRunner pid=2204243) 'fsdp_size': -1, (TaskRunner pid=2204243) 'full_determinism': False, (TaskRunner pid=2204243) 'model_dtype': 'fp32', (TaskRunner pid=2204243) 'offload_policy': False, (TaskRunner pid=2204243) 'optimizer_offload': False, (TaskRunner pid=2204243) 'param_offload': False, (TaskRunner pid=2204243) 'reshard_after_forward': True, (TaskRunner pid=2204243) 'seed': 42, (TaskRunner pid=2204243) 'strategy': 'fsdp', (TaskRunner pid=2204243) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2204243) 'use_orig_params': False, (TaskRunner pid=2204243) 'use_torch_compile': True, (TaskRunner pid=2204243) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2204243) 'lora_alpha': 16, (TaskRunner pid=2204243) 'lora_rank': 0, (TaskRunner pid=2204243) 'override_config': {}, (TaskRunner pid=2204243) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2204243) 'target_modules': 'all-linear', (TaskRunner pid=2204243) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2204243) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2204243) 'trust_remote_code': True, (TaskRunner pid=2204243) 'use_remove_padding': False, (TaskRunner pid=2204243) 'use_shm': False}, (TaskRunner pid=2204243) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2204243) 'betas': [0.9, 0.999], (TaskRunner pid=2204243) 'clip_grad': 1.0, (TaskRunner pid=2204243) 'lr': 1e-05, (TaskRunner pid=2204243) 'lr_scheduler_type': 'constant', (TaskRunner pid=2204243) 'lr_warmup_steps': -1, (TaskRunner pid=2204243) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2204243) 'min_lr_ratio': 0.0, (TaskRunner pid=2204243) 'num_cycles': 0.5, (TaskRunner pid=2204243) 'optimizer': 'AdamW', (TaskRunner pid=2204243) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2204243) 'override_optimizer_config': None, (TaskRunner pid=2204243) 'total_training_steps': -1, (TaskRunner pid=2204243) 'warmup_style': None, (TaskRunner pid=2204243) 'weight_decay': 0.01}, (TaskRunner pid=2204243) 'ppo_epochs': 1, (TaskRunner pid=2204243) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2204243) 'ppo_micro_batch_size': None, (TaskRunner pid=2204243) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2204243) 'ppo_mini_batch_size': 8, (TaskRunner pid=2204243) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2204243) 'all_ranks': False, (TaskRunner pid=2204243) 'enable': False, (TaskRunner pid=2204243) 'ranks': [], (TaskRunner pid=2204243) 'save_path': 'outputs/profile', (TaskRunner pid=2204243) 'tool': None, (TaskRunner pid=2204243) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2204243) 'analysis': True, (TaskRunner pid=2204243) 'contents': [], (TaskRunner pid=2204243) 'discrete': False, (TaskRunner pid=2204243) 'level': 'level0'}, (TaskRunner pid=2204243) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2204243) 'discrete': False}, (TaskRunner pid=2204243) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2204243) 'step_end': None, (TaskRunner pid=2204243) 'step_start': 0}, (TaskRunner pid=2204243) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2204243) 'stack_depth': 32, (TaskRunner pid=2204243) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2204243) 'rollout_n': 8, (TaskRunner pid=2204243) 'shuffle': False, (TaskRunner pid=2204243) 'strategy': 'fsdp', (TaskRunner pid=2204243) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2204243) 'use_dynamic_bsz': False}, (TaskRunner pid=2204243) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2204243) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2204243) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2204243) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2204243) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2204243) 'dataloader_num_workers': 8, (TaskRunner pid=2204243) 'filter_overlong_prompts': True, (TaskRunner pid=2204243) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2204243) 'image_key': 'images', (TaskRunner pid=2204243) 'image_patch_size': 14, (TaskRunner pid=2204243) 'max_prompt_length': 2048, (TaskRunner pid=2204243) 'max_response_length': 8192, (TaskRunner pid=2204243) 'prompt_key': 'prompt', (TaskRunner pid=2204243) 'return_full_prompt': False, (TaskRunner pid=2204243) 'return_multi_modal_inputs': True, (TaskRunner pid=2204243) 'return_raw_chat': True, (TaskRunner pid=2204243) 'return_raw_input_ids': False, (TaskRunner pid=2204243) 'reward_fn_key': 'data_source', (TaskRunner pid=2204243) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2204243) 'seed': None, (TaskRunner pid=2204243) 'shuffle': True, (TaskRunner pid=2204243) 'tokenizer': None, (TaskRunner pid=2204243) 'tool_config_path': None, (TaskRunner pid=2204243) 'train_batch_size': 32, (TaskRunner pid=2204243) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/tooluse/train.parquet'], (TaskRunner pid=2204243) 'train_max_samples': 3200, (TaskRunner pid=2204243) 'truncation': 'error', (TaskRunner pid=2204243) 'trust_remote_code': True, (TaskRunner pid=2204243) 'use_shm': False, (TaskRunner pid=2204243) 'val_batch_size': None, (TaskRunner pid=2204243) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/tooluse/test.parquet'], (TaskRunner pid=2204243) 'val_max_samples': -1, (TaskRunner pid=2204243) 'validation_shuffle': False, (TaskRunner pid=2204243) 'video_key': 'videos'}, (TaskRunner pid=2204243) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2204243) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2204243) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2204243) 'cuda-memory-usage': 'true', (TaskRunner pid=2204243) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2204243) 'discrete': False, (TaskRunner pid=2204243) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2204243) 'capture-range-end': None, (TaskRunner pid=2204243) 'cuda-graph-trace': 'graph', (TaskRunner pid=2204243) 'cuda-memory-usage': 'true', (TaskRunner pid=2204243) 'kill': 'none', (TaskRunner pid=2204243) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2204243) 'torch_memory': {'context': 'all', (TaskRunner pid=2204243) 'kw_args': {}, (TaskRunner pid=2204243) 'stack_depth': 32, (TaskRunner pid=2204243) 'stacks': 'all', (TaskRunner pid=2204243) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2204243) 'profile_continuous_steps': False, (TaskRunner pid=2204243) 'save_path': 'outputs/profile', (TaskRunner pid=2204243) 'steps': None, (TaskRunner pid=2204243) 'tool': None}, (TaskRunner pid=2204243) 'max_model_len': 10240, (TaskRunner pid=2204243) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_tooluse_rlsd_tr_Qwen3_4B', (TaskRunner pid=2204243) 'include_dashboard': False, (TaskRunner pid=2204243) 'num_cpus': None, (TaskRunner pid=2204243) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2204243) 'TASK': 'datasets/tooluse', (TaskRunner pid=2204243) 'USER': 'root'}}}, (TaskRunner pid=2204243) 'timeline_json_file': None}, (TaskRunner pid=2204243) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2204243) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2204243) 'name': 'custom_reward_manager', (TaskRunner pid=2204243) 'path': None}, (TaskRunner pid=2204243) 'name': 'naive', (TaskRunner pid=2204243) 'source': 'register'}, (TaskRunner pid=2204243) 'reward_model': {'enable': False, (TaskRunner pid=2204243) 'enable_resource_pool': False, (TaskRunner pid=2204243) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2204243) 'launch_reward_fn_async': False, (TaskRunner pid=2204243) 'max_length': None, (TaskRunner pid=2204243) 'micro_batch_size': None, (TaskRunner pid=2204243) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2204243) 'model': {'external_lib': None, (TaskRunner pid=2204243) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2204243) 'forward_prefetch': False, (TaskRunner pid=2204243) 'fsdp_size': -1, (TaskRunner pid=2204243) 'param_offload': False, (TaskRunner pid=2204243) 'reshard_after_forward': True, (TaskRunner pid=2204243) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2204243) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2204243) 'override_config': {}, (TaskRunner pid=2204243) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2204243) 'trust_remote_code': False, (TaskRunner pid=2204243) 'use_fused_kernels': False, (TaskRunner pid=2204243) 'use_remove_padding': False, (TaskRunner pid=2204243) 'use_shm': False}, (TaskRunner pid=2204243) 'n_gpus_per_node': 8, (TaskRunner pid=2204243) 'nnodes': 0, (TaskRunner pid=2204243) 'num_workers': 1, (TaskRunner pid=2204243) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2204243) 'all_ranks': False, (TaskRunner pid=2204243) 'enable': False, (TaskRunner pid=2204243) 'ranks': [], (TaskRunner pid=2204243) 'save_path': 'outputs/profile', (TaskRunner pid=2204243) 'tool': None, (TaskRunner pid=2204243) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2204243) 'analysis': True, (TaskRunner pid=2204243) 'contents': [], (TaskRunner pid=2204243) 'discrete': False, (TaskRunner pid=2204243) 'level': 'level0'}, (TaskRunner pid=2204243) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2204243) 'discrete': False}, (TaskRunner pid=2204243) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2204243) 'step_end': None, (TaskRunner pid=2204243) 'step_start': 0}, (TaskRunner pid=2204243) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2204243) 'stack_depth': 32, (TaskRunner pid=2204243) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2204243) 'reward_loop_class_name': None, (TaskRunner pid=2204243) 'reward_loop_module_path': None, (TaskRunner pid=2204243) 'reward_loop_source': 'register', (TaskRunner pid=2204243) 'reward_manager': 'naive', (TaskRunner pid=2204243) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2204243) 'cudagraph_capture_sizes': None, (TaskRunner pid=2204243) 'data_parallel_size': 1, (TaskRunner pid=2204243) 'disable_log_stats': True, (TaskRunner pid=2204243) 'dtype': 'bfloat16', (TaskRunner pid=2204243) 'enable_chunked_prefill': True, (TaskRunner pid=2204243) 'enable_prefix_caching': True, (TaskRunner pid=2204243) 'enforce_eager': True, (TaskRunner pid=2204243) 'engine_kwargs': {}, (TaskRunner pid=2204243) 'expert_parallel_size': 1, (TaskRunner pid=2204243) 'free_cache_engine': True, (TaskRunner pid=2204243) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2204243) 'limit_images': None, (TaskRunner pid=2204243) 'load_format': 'auto', (TaskRunner pid=2204243) 'max_model_len': None, (TaskRunner pid=2204243) 'max_num_batched_tokens': 8192, (TaskRunner pid=2204243) 'max_num_seqs': 1024, (TaskRunner pid=2204243) 'name': '???', (TaskRunner pid=2204243) 'prompt_length': 2048, (TaskRunner pid=2204243) 'response_length': 2048, (TaskRunner pid=2204243) 'skip_tokenizer_init': False, (TaskRunner pid=2204243) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2204243) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2204243) 'memory_limit_mb': 1024, (TaskRunner pid=2204243) 'url': None}, (TaskRunner pid=2204243) 'strategy': 'fsdp', (TaskRunner pid=2204243) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2204243) 'use_dynamic_bsz': False, (TaskRunner pid=2204243) 'use_reward_loop': False}, (TaskRunner pid=2204243) 'trainer': {'balance_batch': True, (TaskRunner pid=2204243) 'critic_warmup': 0, (TaskRunner pid=2204243) 'default_hdfs_dir': None, (TaskRunner pid=2204243) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2204243) 'del_local_ckpt_after_load': False, (TaskRunner pid=2204243) 'device': 'cuda', (TaskRunner pid=2204243) 'esi_redundant_time': 0, (TaskRunner pid=2204243) 'experiment_name': 'qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2204243) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2204243) 'log_val_generations': 0, (TaskRunner pid=2204243) 'logger': ['console', 'wandb'], (TaskRunner pid=2204243) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2204243) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2204243) 'n_gpus_per_node': 8, (TaskRunner pid=2204243) 'nnodes': 1, (TaskRunner pid=2204243) 'project_name': 'SDPO-root', (TaskRunner pid=2204243) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2204243) 'resume_from_path': None, (TaskRunner pid=2204243) 'resume_mode': 'auto', (TaskRunner pid=2204243) 'rollout_data_dir': None, (TaskRunner pid=2204243) 'save_freq': 10, (TaskRunner pid=2204243) 'test_freq': 10, (TaskRunner pid=2204243) 'total_epochs': 30, (TaskRunner pid=2204243) 'total_training_steps': 100, (TaskRunner pid=2204243) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2204243) 'val_before_train': False, (TaskRunner pid=2204243) 'val_only': False, (TaskRunner pid=2204243) 'validation_data_dir': None}, (TaskRunner pid=2204243) 'transfer_queue': {'enable': False}, (TaskRunner pid=2204243) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/tooluse', (TaskRunner pid=2204243) 'dir': '/users/root/SDPO', (TaskRunner pid=2204243) 'log_dir': '/users/root/output', (TaskRunner pid=2204243) 'task': 'datasets/tooluse'}} (TaskRunner pid=2204243) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2204243) use_critic=need_critic(config), (TaskRunner pid=2204243) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2204243) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2204243) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2204243) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2204243) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2204243) Using dataset class: RLHFDataset (TaskRunner pid=2204243) dataset len: 4046 (TaskRunner pid=2204243) selected 3200 random samples out of 4046 (TaskRunner pid=2204243) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2204243) WARNING:2026-07-02 15:45:30,639:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2204243) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/3200 [00:00 (TaskRunner pid=2204243) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2204243) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2204243) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2204243) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2204243) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (WorkerDict pid=2204657) [W702 15:45:42.945920450 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:46681 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2204656) [Gloo] Rank 1 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2204658) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2204658) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2204661) [W702 15:45:42.249150054 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:46681 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2204655) Model config after override: Qwen3Config { (WorkerDict pid=2204655) "architectures": [ (WorkerDict pid=2204655) "Qwen3ForCausalLM" (WorkerDict pid=2204655) ], (WorkerDict pid=2204655) "attention_bias": false, (WorkerDict pid=2204655) "attention_dropout": 0.0, (WorkerDict pid=2204655) "dtype": "bfloat16", (WorkerDict pid=2204655) "eos_token_id": 151645, (WorkerDict pid=2204655) "head_dim": 128, (WorkerDict pid=2204655) "hidden_act": "silu", (WorkerDict pid=2204655) "hidden_size": 2560, (WorkerDict pid=2204655) "initializer_range": 0.02, (WorkerDict pid=2204655) "intermediate_size": 9728, (WorkerDict pid=2204655) "layer_types": [ (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention", (WorkerDict pid=2204655) "full_attention" (WorkerDict pid=2204655) ], (WorkerDict pid=2204655) "max_position_embeddings": 40960, (WorkerDict pid=2204655) "max_window_layers": 36, (WorkerDict pid=2204655) "model_type": "qwen3", (WorkerDict pid=2204655) "num_attention_heads": 32, (WorkerDict pid=2204655) "num_hidden_layers": 36, (WorkerDict pid=2204655) "num_key_value_heads": 8, (WorkerDict pid=2204655) "pad_token_id": 151643, (WorkerDict pid=2204655) "rms_norm_eps": 1e-06, (WorkerDict pid=2204655) "rope_scaling": null, (WorkerDict pid=2204655) "rope_theta": 1000000, (WorkerDict pid=2204655) "sliding_window": null, (WorkerDict pid=2204655) "tie_word_embeddings": true, (WorkerDict pid=2204655) "transformers_version": "4.57.1", (WorkerDict pid=2204655) "use_cache": true, (WorkerDict pid=2204655) "use_sliding_window": false, (WorkerDict pid=2204655) "vocab_size": 151936 (WorkerDict pid=2204655) } (WorkerDict pid=2204655) (WorkerDict pid=2204658) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2204658) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2204658) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2204658) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2204655) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2204655) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2204662) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2204662) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2204658) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2204658) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2204662) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.94s/it] [repeated 14x across cluster] (WorkerDict pid=2204662) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.98s/it] [repeated 6x across cluster] (WorkerDict pid=2204655) Actor use_remove_padding=True (WorkerDict pid=2204655) Actor use_fused_kernels=False (WorkerDict pid=2204655) Actor use_prefix_grouper=False (WorkerDict pid=2204661) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2204661) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2204655) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2204662) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2204662) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2204658) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2204658) warnings.warn( (WorkerDict pid=2204655) reference model: Qwen/Qwen3-4B (WorkerDict pid=2204658) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2204658) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2204657) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2204655) Ref use_remove_padding=True (WorkerDict pid=2204655) Ref use_fused_kernels=False (WorkerDict pid=2204655) Ref use_prefix_grouper=False (TaskRunner pid=2204243) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2204243) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2204662) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.96s/it] [repeated 12x across cluster] (WorkerDict pid=2204659) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.95s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.99s/it] [repeated 2x across cluster] (WorkerDict pid=2204662) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.00s/it] [repeated 4x across cluster] (vLLMHttpServer pid=2205546) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2205546) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2205546) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2205546) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2205547) ['serve', (vLLMHttpServer pid=2205547) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2205547) '--dtype', (vLLMHttpServer pid=2205547) 'bfloat16', (vLLMHttpServer pid=2205547) '--load_format', (vLLMHttpServer pid=2205547) 'dummy', (vLLMHttpServer pid=2205547) '--trust_remote_code', (vLLMHttpServer pid=2205547) '--max_model_len', (vLLMHttpServer pid=2205547) '40960', (vLLMHttpServer pid=2205547) '--max_num_seqs', (vLLMHttpServer pid=2205547) '1024', (vLLMHttpServer pid=2205547) '--enable_chunked_prefill', (vLLMHttpServer pid=2205547) '--max_num_batched_tokens', (vLLMHttpServer pid=2205547) '10240', (vLLMHttpServer pid=2205547) '--enable_prefix_caching', (vLLMHttpServer pid=2205547) '--enable_sleep_mode', (vLLMHttpServer pid=2205547) '--logprobs_mode', (vLLMHttpServer pid=2205547) 'processed_logprobs', (vLLMHttpServer pid=2205547) '--disable_custom_all_reduce', (vLLMHttpServer pid=2205547) '--gpu_memory_utilization', (vLLMHttpServer pid=2205547) '0.8', (vLLMHttpServer pid=2205547) '--disable_log_stats', (vLLMHttpServer pid=2205547) '--tensor_parallel_size', (vLLMHttpServer pid=2205547) '2', (vLLMHttpServer pid=2205547) '--seed', (vLLMHttpServer pid=2205547) '0', (vLLMHttpServer pid=2205547) '--override_generation_config', (vLLMHttpServer pid=2205547) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2205547) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2205547) '--hf_overrides', (vLLMHttpServer pid=2205547) '{}', (vLLMHttpServer pid=2205547) '--scheduling_policy', (vLLMHttpServer pid=2205547) 'fcfs'] (WorkerDict pid=2204659) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2204659) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (vLLMHttpServer pid=2205546) WARNING 07-02 15:46:39 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2204662) WARNING 07-02 15:46:47 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2205548) WARNING 07-02 15:46:40 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2204657) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2204657) 2026-07-02 15:47:02,770 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (vLLMHttpServer pid=2205548) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2205548) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2205545) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2205545) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2204659) 2026-07-02 15:47:02,696 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2204655) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: The user is asking for a random picture of an axolotl. The appropriate tool for this is getRandomAxolotlImage, which retrieves a random axolotl image with information on the image source. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2204243) step:10 - global_seqlen/min:16902 - global_seqlen/max:28919 - global_seqlen/minmax_diff:12017 - global_seqlen/balanced_min:21597 - global_seqlen/balanced_max:21600 - global_seqlen/mean:21598.5 - actor/entropy:0.13048547506332397 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8160732388496399 - training/rollout_probs_diff_mean:0.004491306841373444 - training/rollout_probs_diff_std:0.015517027117311954 - training/rollout_actor_probs_pearson_corr:0.996421217918396 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.4375 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.58984375 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000616312026978 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.4549422202399e-05 - rollout_corr/rollout_is_max:1.9516314268112183 - rollout_corr/rollout_is_min:0.0018361398251727223 - rollout_corr/rollout_is_std:0.041942477226257324 - rollout_corr/rollout_is_eff_sample_size:0.9982441553459014 - rollout_corr/rollout_is_seq_mean:1.0008928775787354 - rollout_corr/rollout_is_seq_std:0.011417483910918236 - rollout_corr/rollout_is_seq_max:1.095253348350525 - rollout_corr/rollout_is_seq_min:0.9723833203315735 - rollout_corr/rollout_is_seq_max_deviation:0.0952533483505249 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1318232482299209) - rollout_corr/training_log_ppl:np.float64(0.11975393812099355) - rollout_corr/kl:np.float64(0.00036851357227618564) - rollout_corr/k3_kl:np.float64(0.0018039138175512115) - rollout_corr/rollout_ppl:np.float64(1.1314550628885627) - rollout_corr/rollout_log_ppl:np.float64(0.11938542358984705) - rollout_corr/log_ppl_diff:np.float64(0.0003685145311465021) - rollout_corr/log_ppl_abs_diff:np.float64(0.005026785514928633) - rollout_corr/log_ppl_diff_max:np.float64(0.10496306419372559) - rollout_corr/log_ppl_diff_min:np.float64(-0.08793950080871582) - rollout_corr/ppl_ratio:np.float64(1.0004476632457227) - rollout_corr/chi2_token:np.float64(0.006944536929950118) - rollout_corr/chi2_seq:np.float64(0.3551388084888458) - actor/pg_loss:np.float64(0.0006074672564864159) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011681771866278723) - actor/ppo_kl:np.float64(-0.00020348432867933752) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(190316.37712003174) - self_distillation/token_reweight_w_std:np.float64(1368305.9383808027) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.99982038885355) - self_distillation/token_reweight_w_clip_frac:np.float64(0.00871973461471498) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.5214780941605568) - perf/mfu/actor:np.float64(0.03986966335328584) - perf/max_memory_allocated_gb:np.float64(125.39397525787354) - perf/max_memory_reserved_gb:np.float64(131.90234375) - perf/cpu_memory_used_gb:np.float64(141.3655776977539) - actor/lr:np.float64(9e-07) - val-aux/tooluse/reward/mean@16:np.float64(0.5762867647058824) - val-aux/tooluse/reward/std@16:np.float64(0.038186735903151886) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5934852941176472) - val-aux/tooluse/reward/best@2/std:np.float64(0.02982570434726312) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5583529411764705) - val-aux/tooluse/reward/worst@2/std:np.float64(0.035811116275167025) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.575514705882353) - val-aux/tooluse/reward/maj@2/std:np.float64(0.038442166565847244) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6039705882352943) - val-aux/tooluse/reward/best@4/std:np.float64(0.016952338343354353) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5423382352941176) - val-aux/tooluse/reward/worst@4/std:np.float64(0.025766963713467317) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.580029411764706) - val-aux/tooluse/reward/maj@4/std:np.float64(0.03351803005569556) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6088823529411764) - val-aux/tooluse/reward/best@8/std:np.float64(0.009079688218069783) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5328970588235294) - val-aux/tooluse/reward/worst@8/std:np.float64(0.012312187797583406) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5847058823529412) - val-aux/tooluse/reward/maj@8/std:np.float64(0.02739498127252076) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6126029411764705) - val-aux/tooluse/reward/best@16/std:np.float64(0.006981051343076578) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5299705882352941) - val-aux/tooluse/reward/worst@16/std:np.float64(0.0036691767481211277) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5873382352941177) - val-aux/tooluse/reward/maj@16/std:np.float64(0.02252479966713278) - val-aux/tooluse/score/mean@16:np.float64(0.5762867647058824) - val-aux/tooluse/score/std@16:np.float64(0.038186735903151886) - val-aux/tooluse/score/best@2/mean:np.float64(0.5934852941176472) - val-aux/tooluse/score/best@2/std:np.float64(0.02982570434726312) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5583529411764705) - val-aux/tooluse/score/worst@2/std:np.float64(0.035811116275167025) - val-aux/tooluse/score/maj@2/mean:np.float64(0.575514705882353) - val-aux/tooluse/score/maj@2/std:np.float64(0.038442166565847244) - val-aux/tooluse/score/best@4/mean:np.float64(0.6039705882352943) - val-aux/tooluse/score/best@4/std:np.float64(0.016952338343354353) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5423382352941176) - val-aux/tooluse/score/worst@4/std:np.float64(0.025766963713467317) - val-aux/tooluse/score/maj@4/mean:np.float64(0.580029411764706) - val-aux/tooluse/score/maj@4/std:np.float64(0.03351803005569556) - val-aux/tooluse/score/best@8/mean:np.float64(0.6088823529411764) - val-aux/tooluse/score/best@8/std:np.float64(0.009079688218069783) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5328970588235294) - val-aux/tooluse/score/worst@8/std:np.float64(0.012312187797583406) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5847058823529412) - val-aux/tooluse/score/maj@8/std:np.float64(0.02739498127252076) - val-aux/tooluse/score/best@16/mean:np.float64(0.6126029411764705) - val-aux/tooluse/score/best@16/std:np.float64(0.006981051343076578) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5299705882352941) - val-aux/tooluse/score/worst@16/std:np.float64(0.0036691767481211277) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5873382352941177) - val-aux/tooluse/score/maj@16/std:np.float64(0.02252479966713278) - val-core/tooluse/acc/mean@16:np.float64(0.5762867647058824) - val-aux/tooluse/acc/std@16:np.float64(0.038186735903151886) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5934852941176472) - val-aux/tooluse/acc/best@2/std:np.float64(0.02982570434726312) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5583529411764705) - val-aux/tooluse/acc/worst@2/std:np.float64(0.035811116275167025) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.575514705882353) - val-aux/tooluse/acc/maj@2/std:np.float64(0.038442166565847244) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6039705882352943) - val-aux/tooluse/acc/best@4/std:np.float64(0.016952338343354353) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5423382352941176) - val-aux/tooluse/acc/worst@4/std:np.float64(0.025766963713467317) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.580029411764706) - val-aux/tooluse/acc/maj@4/std:np.float64(0.03351803005569556) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6088823529411764) - val-aux/tooluse/acc/best@8/std:np.float64(0.009079688218069783) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5328970588235294) - val-aux/tooluse/acc/worst@8/std:np.float64(0.012312187797583406) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5847058823529412) - val-aux/tooluse/acc/maj@8/std:np.float64(0.02739498127252076) - val-core/tooluse/acc/best@16/mean:np.float64(0.6126029411764705) - val-core/tooluse/acc/best@16/std:np.float64(0.006981051343076578) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5299705882352941) - val-aux/tooluse/acc/worst@16/std:np.float64(0.0036691767481211277) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5873382352941177) - val-core/tooluse/acc/maj@16/std:np.float64(0.02252479966713278) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007432358688674867 - critic/advantages/max:0.5 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0007432358688674867 - critic/returns/max:0.5 - critic/returns/min:-0.625 - response_length/mean:71.609375 - response_length/max:232.0 - response_length/min:10.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:71.609375 - response_length_non_aborted/max:232.0 - response_length_non_aborted/min:10.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:603.34375 - prompt_length/max:976.0 - prompt_length/min:242.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.189756095409393e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.2576142232865095) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7269353736191988) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9316517743936856) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7269353736191988) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:835 - timing_s/agent_loop/slowest/response_length:208 - timing_s/gen:7.533783840015531 - timing_s/reward:0.07022674940526485 - timing_s/old_log_prob:2.3177663199603558 - timing_s/adv:0.5777656361460686 - timing_s/update_actor:15.49716048501432 - timing_s/step:26.084238247945905 - timing_s/testing:21.160797089338303 - timing_s/save_checkpoint:6.602799087762833 - timing_s/stop_profile:0.00012887455523014069 - timing_per_token_ms/update_actor:0.08968887008944093 - timing_per_token_ms/gen:0.41096355225919323 - timing_per_token_ms/adv:0.003343783342281111 - perf/total_num_tokens:172788 - perf/time_per_step:26.084238247945905 - perf/throughput:828.0287810091924 (TaskRunner pid=2204243) Training Progress: 10%|█ | 10/100 [06:26<57:01, 38.02s/it] (TaskRunner pid=2204243) step:11 - global_seqlen/min:17809 - global_seqlen/max:30963 - global_seqlen/minmax_diff:13154 - global_seqlen/balanced_min:22280 - global_seqlen/balanced_max:22293 - global_seqlen/mean:22289.375 - actor/entropy:0.13205944001674652 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3284083604812622 - training/rollout_probs_diff_mean:0.00413705175742507 - training/rollout_probs_diff_std:0.014121152460575104 - training/rollout_actor_probs_pearson_corr:0.9970347881317139 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.43359375 - self_distillation/correct_sample_fraction:0.3984375 - self_distillation/feedback_available_fraction:0.6015625 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9996955394744873 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013774736726190895 - rollout_corr/rollout_is_max:1.5178592205047607 - rollout_corr/rollout_is_min:0.35117778182029724 - rollout_corr/rollout_is_std:0.03697985038161278 - rollout_corr/rollout_is_eff_sample_size:0.9986334665343347 - rollout_corr/rollout_is_seq_mean:0.9997003078460693 - rollout_corr/rollout_is_seq_std:0.004266595002263784 - rollout_corr/rollout_is_seq_max:1.0124309062957764 - rollout_corr/rollout_is_seq_min:0.9797662496566772 - rollout_corr/rollout_is_seq_max_deviation:0.020233750343322754 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1272121807560325) - rollout_corr/training_log_ppl:np.float64(0.11546109981645714) - rollout_corr/kl:np.float64(0.001709336568144204) - rollout_corr/k3_kl:np.float64(0.001170722132520785) - rollout_corr/rollout_ppl:np.float64(1.125215726904571) - rollout_corr/rollout_log_ppl:np.float64(0.11375176303408807) - rollout_corr/log_ppl_diff:np.float64(0.0017093367823690642) - rollout_corr/log_ppl_abs_diff:np.float64(0.004259654986526584) - rollout_corr/log_ppl_diff_max:np.float64(0.05087852478027344) - rollout_corr/log_ppl_diff_min:np.float64(-0.014373783022165298) - rollout_corr/ppl_ratio:np.float64(1.00173244997859) - rollout_corr/chi2_token:np.float64(0.00102601433172822) - rollout_corr/chi2_seq:np.float64(0.16015938157215714) - actor/pg_loss:np.float64(-0.00017554033547639847) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010177231415582355) - actor/ppo_kl:np.float64(0.0007112429386282315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3984375) - self_distillation/token_reweight_w_mean:np.float64(337890.3608085343) - self_distillation/token_reweight_w_std:np.float64(1974194.657909121) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000302076106891) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01300497050397098) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.41401390824466944) - perf/mfu/actor:np.float64(0.04099695193319455) - perf/max_memory_allocated_gb:np.float64(125.39397525787354) - perf/max_memory_reserved_gb:np.float64(131.90234375) - perf/cpu_memory_used_gb:np.float64(143.51392364501953) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.3984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006141237099654973 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006141237099654973 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:85.07421875 - response_length/max:290.0 - response_length/min:41.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:85.07421875 - response_length_non_aborted/max:290.0 - response_length_non_aborted/min:41.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:611.46875 - prompt_length/max:1522.0 - prompt_length/min:267.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.838647484779358e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.593901002779603) - timing_s/agent_loop/generate_sequences/max:np.float64(2.0821895822882652) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9925621511138161) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.0821895822882652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1522 - timing_s/agent_loop/slowest/response_length:290 - timing_s/gen:7.757367093116045 - timing_s/reward:0.06819679960608482 - timing_s/old_log_prob:2.550508553162217 - timing_s/adv:0.6781942471861839 - timing_s/update_actor:15.644280266016722 - timing_s/step:26.80492282100022 - timing_s/stop_profile:5.130469799041748e-05 - timing_per_token_ms/update_actor:0.08773395544971944 - timing_per_token_ms/gen:0.35618564181624707 - timing_per_token_ms/adv:0.0038033493939723744 - perf/total_num_tokens:178315 - perf/time_per_step:26.80492282100022 - perf/throughput:831.5403535703326 (TaskRunner pid=2204243) Training Progress: 11%|█ | 11/100 [06:52<51:18, 34.59s/it] (TaskRunner pid=2204243) step:12 - global_seqlen/min:15743 - global_seqlen/max:23111 - global_seqlen/minmax_diff:7368 - global_seqlen/balanced_min:20155 - global_seqlen/balanced_max:20658 - global_seqlen/mean:20279.0 - actor/entropy:0.1364346444606781 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.28070271015167236 - training/rollout_probs_diff_mean:0.004087985027581453 - training/rollout_probs_diff_std:0.013860108330845833 - training/rollout_actor_probs_pearson_corr:0.9972864389419556 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.3125 - self_distillation/correct_sample_fraction:0.26953125 - self_distillation/feedback_available_fraction:0.73046875 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999499320983887 - rollout_corr/rollout_is_ratio_fraction_high:8.085381705313921e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020213454263284802 - rollout_corr/rollout_is_max:3.1891632080078125 - rollout_corr/rollout_is_min:0.32974931597709656 - rollout_corr/rollout_is_std:0.03923454508185387 - rollout_corr/rollout_is_eff_sample_size:0.9984626598698045 - rollout_corr/rollout_is_seq_mean:1.000026822090149 - rollout_corr/rollout_is_seq_std:0.005337757524102926 - rollout_corr/rollout_is_seq_max:1.0251529216766357 - rollout_corr/rollout_is_seq_min:0.9882858991622925 - rollout_corr/rollout_is_seq_max_deviation:0.025152921676635742 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1549143712036312) - rollout_corr/training_log_ppl:np.float64(0.13888394820241956) - rollout_corr/kl:np.float64(0.0008059948504524073) - rollout_corr/k3_kl:np.float64(0.0010741546440933246) - rollout_corr/rollout_ppl:np.float64(1.1539734262041748) - rollout_corr/rollout_log_ppl:np.float64(0.13807795221509878) - rollout_corr/log_ppl_diff:np.float64(0.0008059959873207845) - rollout_corr/log_ppl_abs_diff:np.float64(0.004131796020374168) - rollout_corr/log_ppl_diff_max:np.float64(0.013977006077766418) - rollout_corr/log_ppl_diff_min:np.float64(-0.017158910632133484) - rollout_corr/ppl_ratio:np.float64(1.0008202090393752) - rollout_corr/chi2_token:np.float64(0.0028365016914904118) - rollout_corr/chi2_seq:np.float64(0.3671533667948097) - actor/pg_loss:np.float64(0.0004832702688872814) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019586185226216912) - actor/ppo_kl:np.float64(-0.00010159479775495583) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.26953125) - self_distillation/token_reweight_w_mean:np.float64(92820.25307788816) - self_distillation/token_reweight_w_std:np.float64(781165.4137994419) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007921361830086) - self_distillation/token_reweight_w_clip_frac:np.float64(0.013472498365445063) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.5638359934091568) - perf/mfu/actor:np.float64(0.037294426483057024) - perf/max_memory_allocated_gb:np.float64(125.39397525787354) - perf/max_memory_reserved_gb:np.float64(131.90234375) - perf/cpu_memory_used_gb:np.float64(143.40973281860352) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.26953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.26953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028147234115749598 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0028147234115749598 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:96.625 - response_length/max:1115.0 - response_length/min:35.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:96.625 - response_length_non_aborted/max:1115.0 - response_length_non_aborted/min:35.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:537.09375 - prompt_length/max:853.0 - prompt_length/min:241.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.808790981769562e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4843508079648018) - timing_s/agent_loop/generate_sequences/max:np.float64(5.990847427397966) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0360853948513977) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.990847427397966) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:680 - timing_s/agent_loop/slowest/response_length:1115 - timing_s/gen:11.613326989114285 - timing_s/reward:0.058538541197776794 - timing_s/old_log_prob:2.376751884818077 - timing_s/adv:0.5395427662879229 - timing_s/update_actor:15.559926142916083 - timing_s/step:30.182284390553832 - timing_s/stop_profile:2.763606607913971e-05 - timing_per_token_ms/update_actor:0.0959115719643232 - timing_per_token_ms/gen:0.4694909035055904 - timing_per_token_ms/adv:0.003325748103259054 - perf/total_num_tokens:162232 - perf/time_per_step:30.182284390553832 - perf/throughput:671.8841999363948 (TaskRunner pid=2204243) Training Progress: 12%|█▏ | 12/100 [07:23<48:46, 33.26s/it] (TaskRunner pid=2204243) step:13 - global_seqlen/min:18352 - global_seqlen/max:29044 - global_seqlen/minmax_diff:10692 - global_seqlen/balanced_min:22282 - global_seqlen/balanced_max:22292 - global_seqlen/mean:22285.875 - actor/entropy:0.1585795134305954 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24444299936294556 - training/rollout_probs_diff_mean:0.004745909944176674 - training/rollout_probs_diff_std:0.014600160531699657 - training/rollout_actor_probs_pearson_corr:0.9972651600837708 - self_distillation/success_group_fraction:0.46875 - self_distillation/success_sample_fraction:0.4609375 - self_distillation/correct_sample_fraction:0.39453125 - self_distillation/feedback_available_fraction:0.60546875 - self_distillation/feedback_used_fraction:0.53125 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:1.0003151893615723 - rollout_corr/rollout_is_ratio_fraction_high:4.1351362597197294e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0528807640075684 - rollout_corr/rollout_is_min:0.5117019414901733 - rollout_corr/rollout_is_std:0.04184858128428459 - rollout_corr/rollout_is_eff_sample_size:0.9982525895803692 - rollout_corr/rollout_is_seq_mean:1.0004205703735352 - rollout_corr/rollout_is_seq_std:0.004512621555477381 - rollout_corr/rollout_is_seq_max:1.0182092189788818 - rollout_corr/rollout_is_seq_min:0.9872854948043823 - rollout_corr/rollout_is_seq_max_deviation:0.018209218978881836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.167494063731283) - rollout_corr/training_log_ppl:np.float64(0.15012345152717899) - rollout_corr/kl:np.float64(0.0015586957418562974) - rollout_corr/k3_kl:np.float64(0.001272493793234769) - rollout_corr/rollout_ppl:np.float64(1.1657104846090078) - rollout_corr/rollout_log_ppl:np.float64(0.1485647542285733) - rollout_corr/log_ppl_diff:np.float64(0.0015586972986056935) - rollout_corr/log_ppl_abs_diff:np.float64(0.0042413812516315375) - rollout_corr/log_ppl_diff_max:np.float64(0.01899927854537964) - rollout_corr/log_ppl_diff_min:np.float64(-0.01597270369529724) - rollout_corr/ppl_ratio:np.float64(1.0015750946477056) - rollout_corr/chi2_token:np.float64(0.0018511868547648191) - rollout_corr/chi2_seq:np.float64(0.24974323832429945) - actor/pg_loss:np.float64(0.0012441084254533052) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009887394207908073) - actor/ppo_kl:np.float64(0.0011345302436538418) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.39453125) - self_distillation/token_reweight_w_mean:np.float64(64533.085790291196) - self_distillation/token_reweight_w_std:np.float64(643186.4801101075) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001893350854516) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02673893648898229) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.440546490252018) - perf/mfu/actor:np.float64(0.04115852207807518) - perf/max_memory_allocated_gb:np.float64(125.39397525787354) - perf/max_memory_reserved_gb:np.float64(131.90234375) - perf/cpu_memory_used_gb:np.float64(143.49291229248047) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.39453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.39453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00831679254770279 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00831679254770279 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:94.46484375 - response_length/max:682.0 - response_length/min:39.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:94.46484375 - response_length_non_aborted/max:682.0 - response_length_non_aborted/min:39.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:601.96875 - prompt_length/max:1127.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6628375053405762e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5508524533361197) - timing_s/agent_loop/generate_sequences/max:np.float64(3.94191025570035) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.040025423768384) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.94191025570035) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:621 - timing_s/agent_loop/slowest/response_length:682 - timing_s/gen:9.777624344453216 - timing_s/reward:0.07144170999526978 - timing_s/old_log_prob:2.3670394476503134 - timing_s/adv:0.5728748068213463 - timing_s/update_actor:15.623477401211858 - timing_s/step:28.49779985100031 - timing_s/stop_profile:2.7643516659736633e-05 - timing_per_token_ms/update_actor:0.0876310521867094 - timing_per_token_ms/gen:0.40431808892417054 - timing_per_token_ms/adv:0.0032132169301258436 - perf/total_num_tokens:178287 - perf/time_per_step:28.49779985100031 - perf/throughput:782.0208969296182 (TaskRunner pid=2204243) Training Progress: 13%|█▎ | 13/100 [07:51<46:08, 31.82s/it] (TaskRunner pid=2204243) step:14 - global_seqlen/min:20337 - global_seqlen/max:28734 - global_seqlen/minmax_diff:8397 - global_seqlen/balanced_min:23072 - global_seqlen/balanced_max:30839 - global_seqlen/mean:24067.0 - actor/entropy:0.10385475307703018 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730053663253784 - training/rollout_probs_diff_mean:0.0035572447814047337 - training/rollout_probs_diff_std:0.013373459689319134 - training/rollout_actor_probs_pearson_corr:0.9967379570007324 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.3125 - self_distillation/correct_sample_fraction:0.30859375 - self_distillation/feedback_available_fraction:0.69140625 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9997586607933044 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.959911115700379e-05 - rollout_corr/rollout_is_max:1.6115058660507202 - rollout_corr/rollout_is_min:0.4281511604785919 - rollout_corr/rollout_is_std:0.03538691997528076 - rollout_corr/rollout_is_eff_sample_size:0.9987487970461657 - rollout_corr/rollout_is_seq_mean:0.999555766582489 - rollout_corr/rollout_is_seq_std:0.00536120543256402 - rollout_corr/rollout_is_seq_max:1.016081690788269 - rollout_corr/rollout_is_seq_min:0.9759222269058228 - rollout_corr/rollout_is_seq_max_deviation:0.024077773094177246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1504421811550856) - rollout_corr/training_log_ppl:np.float64(0.13622803962789476) - rollout_corr/kl:np.float64(0.001957618100323799) - rollout_corr/k3_kl:np.float64(0.0011290082794435108) - rollout_corr/rollout_ppl:np.float64(1.148147135041654) - rollout_corr/rollout_log_ppl:np.float64(0.13427042113471543) - rollout_corr/log_ppl_diff:np.float64(0.001957618493179325) - rollout_corr/log_ppl_abs_diff:np.float64(0.004795097549504135) - rollout_corr/log_ppl_diff_max:np.float64(0.02307165414094925) - rollout_corr/log_ppl_diff_min:np.float64(-0.01278572529554367) - rollout_corr/ppl_ratio:np.float64(1.0019770292565227) - rollout_corr/chi2_token:np.float64(0.0005782651714980602) - rollout_corr/chi2_seq:np.float64(0.08962924219667912) - actor/pg_loss:np.float64(9.298184886574745e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.278716344037093e-05) - actor/ppo_kl:np.float64(0.0006035562432131769) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.30859375) - self_distillation/token_reweight_w_mean:np.float64(3.0437494276557118) - self_distillation/token_reweight_w_std:np.float64(16.477517940686084) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996021708939224) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006471035303547978) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.11434643715620041) - perf/mfu/actor:np.float64(0.04126608994272067) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(143.5423240661621) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.30859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0001957474887603894 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:0.0001957474887603894 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:112.25 - response_length/max:8192.0 - response_length/min:31.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:112.25 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:31.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:639.84375 - prompt_length/max:1106.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.6574358344078064e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.26495047472417355) - timing_s/agent_loop/generate_sequences/max:np.float64(43.91585084423423) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9833037493517622) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(43.91585084423423) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1106 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:50.52212469652295 - timing_s/reward:0.07585105299949646 - timing_s/old_log_prob:2.6375533547252417 - timing_s/adv:0.6892005149275064 - timing_s/update_actor:16.909741088747978 - timing_s/step:70.92736879736185 - timing_s/stop_profile:2.9671937227249146e-05 - timing_per_token_ms/update_actor:0.08782638617582156 - timing_per_token_ms/gen:1.7581474351518287 - timing_per_token_ms/adv:0.0035795929848314416 - perf/total_num_tokens:192536 - perf/time_per_step:70.92736879736185 - perf/throughput:339.3189456774995 (TaskRunner pid=2204243) Training Progress: 14%|█▍ | 14/100 [09:02<1:02:32, 43.64s/it] (TaskRunner pid=2204243) step:15 - global_seqlen/min:17684 - global_seqlen/max:52915 - global_seqlen/minmax_diff:35231 - global_seqlen/balanced_min:20225 - global_seqlen/balanced_max:27831 - global_seqlen/mean:24066.0 - actor/entropy:0.06711412966251373 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2256726622581482 - training/rollout_probs_diff_mean:0.001889920444227755 - training/rollout_probs_diff_std:0.009426300413906574 - training/rollout_actor_probs_pearson_corr:0.9976748824119568 - self_distillation/success_group_fraction:0.40625 - self_distillation/success_sample_fraction:0.3984375 - self_distillation/correct_sample_fraction:0.30078125 - self_distillation/feedback_available_fraction:0.69921875 - self_distillation/feedback_used_fraction:0.59375 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:0.9998626708984375 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.635305984062143e-05 - rollout_corr/rollout_is_max:1.9132332801818848 - rollout_corr/rollout_is_min:0.3824036121368408 - rollout_corr/rollout_is_std:0.02555675432085991 - rollout_corr/rollout_is_eff_sample_size:0.999346921464983 - rollout_corr/rollout_is_seq_mean:0.9996876120567322 - rollout_corr/rollout_is_seq_std:0.0047794366255402565 - rollout_corr/rollout_is_seq_max:1.0161978006362915 - rollout_corr/rollout_is_seq_min:0.9816624522209167 - rollout_corr/rollout_is_seq_max_deviation:0.018337547779083252 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1593449721112847) - rollout_corr/training_log_ppl:np.float64(0.14341101238460396) - rollout_corr/kl:np.float64(0.0024125542440813774) - rollout_corr/k3_kl:np.float64(0.0016599729483983694) - rollout_corr/rollout_ppl:np.float64(1.1565193845890462) - rollout_corr/rollout_log_ppl:np.float64(0.140998456714442) - rollout_corr/log_ppl_diff:np.float64(0.0024125556701619644) - rollout_corr/log_ppl_abs_diff:np.float64(0.005106302985950606) - rollout_corr/log_ppl_diff_max:np.float64(0.0857316181063652) - rollout_corr/log_ppl_diff_min:np.float64(-0.01823902130126953) - rollout_corr/ppl_ratio:np.float64(1.0024651768617332) - rollout_corr/chi2_token:np.float64(0.0008277972228825092) - rollout_corr/chi2_seq:np.float64(0.2294014417566359) - actor/pg_loss:np.float64(0.0010964818065986037) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008852892278810032) - actor/ppo_kl:np.float64(0.0013436021003863097) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.30078125) - self_distillation/token_reweight_w_mean:np.float64(378198.9193202753) - self_distillation/token_reweight_w_std:np.float64(3054533.765288242) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000970728462562) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03137301281094551) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.7317163683474064) - perf/mfu/actor:np.float64(0.043099961220870786) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(143.48568725585938) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.30078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0028150901198387146 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0028150901198387146 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:214.90625 - response_length/max:8192.0 - response_length/min:36.0 - response_length/clip_ratio:0.015625 - response_length_non_aborted/mean:214.90625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:36.0 - response_length_non_aborted/clip_ratio:0.015625 - response/aborted_ratio:0.0 - prompt_length/mean:537.15625 - prompt_length/max:1015.0 - prompt_length/min:256.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.862699329853058e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.41149362549185753) - timing_s/agent_loop/generate_sequences/max:np.float64(44.91431352868676) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.6067351909296121) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(44.91431352868676) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:943 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:50.565141160041094 - timing_s/reward:0.07363336533308029 - timing_s/old_log_prob:2.607298841699958 - timing_s/adv:0.5679159741848707 - timing_s/update_actor:16.89909751340747 - timing_s/step:70.80118860863149 - timing_s/stop_profile:0.00011917948722839355 - timing_per_token_ms/update_actor:0.08777475231346853 - timing_per_token_ms/gen:0.9190988287051238 - timing_per_token_ms/adv:0.0029497837934475543 - perf/total_num_tokens:192528 - perf/time_per_step:70.80118860863149 - perf/throughput:339.90954774827145 (TaskRunner pid=2204243) Training Progress: 15%|█▌ | 15/100 [10:13<1:13:26, 51.84s/it] (TaskRunner pid=2204243) step:16 - global_seqlen/min:18134 - global_seqlen/max:25137 - global_seqlen/minmax_diff:7003 - global_seqlen/balanced_min:21522 - global_seqlen/balanced_max:21524 - global_seqlen/mean:21523.0 - actor/entropy:0.15608933568000793 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33947527408599854 - training/rollout_probs_diff_mean:0.004753048066049814 - training/rollout_probs_diff_std:0.01468186266720295 - training/rollout_actor_probs_pearson_corr:0.9971400499343872 - self_distillation/success_group_fraction:0.28125 - self_distillation/success_sample_fraction:0.27734375 - self_distillation/correct_sample_fraction:0.21875 - self_distillation/feedback_available_fraction:0.78125 - self_distillation/feedback_used_fraction:0.71875 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9998989105224609 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.852691098582e-05 - rollout_corr/rollout_is_max:1.681875228881836 - rollout_corr/rollout_is_min:0.22342675924301147 - rollout_corr/rollout_is_std:0.04125615209341049 - rollout_corr/rollout_is_eff_sample_size:0.9983004654708637 - rollout_corr/rollout_is_seq_mean:0.999842643737793 - rollout_corr/rollout_is_seq_std:0.004647100809961557 - rollout_corr/rollout_is_seq_max:1.0163575410842896 - rollout_corr/rollout_is_seq_min:0.9866822957992554 - rollout_corr/rollout_is_seq_max_deviation:0.01635754108428955 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1600838671438396) - rollout_corr/training_log_ppl:np.float64(0.14459570915641962) - rollout_corr/kl:np.float64(0.0014603760846880753) - rollout_corr/k3_kl:np.float64(0.0016853843018935777) - rollout_corr/rollout_ppl:np.float64(1.1582884965464473) - rollout_corr/rollout_log_ppl:np.float64(0.14313533152017044) - rollout_corr/log_ppl_diff:np.float64(0.0014603776362491772) - rollout_corr/log_ppl_abs_diff:np.float64(0.005193322474951856) - rollout_corr/log_ppl_diff_max:np.float64(0.018202543258666992) - rollout_corr/log_ppl_diff_min:np.float64(-0.021419793367385864) - rollout_corr/ppl_ratio:np.float64(1.0014839046634734) - rollout_corr/chi2_token:np.float64(0.004006753675639629) - rollout_corr/chi2_seq:np.float64(0.3391839792020619) - actor/pg_loss:np.float64(0.0001044153468683362) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000996570412098663) - actor/ppo_kl:np.float64(0.0004483901382954514) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.21875) - self_distillation/token_reweight_w_mean:np.float64(290404.4731923456) - self_distillation/token_reweight_w_std:np.float64(2073000.0896679654) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013609533198178) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02073262211342808) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.43698752857744694) - perf/mfu/actor:np.float64(0.039998291860713873) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(143.62589263916016) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.21875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.21875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00624114740639925 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00624114740639925 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:88.25 - response_length/max:362.0 - response_length/min:36.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:88.25 - response_length_non_aborted/max:362.0 - response_length_non_aborted/min:36.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:584.34375 - prompt_length/max:1121.0 - prompt_length/min:247.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001529194414615631 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5448823664337397) - timing_s/agent_loop/generate_sequences/max:np.float64(2.5207018349319696) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.093533922903589) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.5207018349319696) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:653 - timing_s/agent_loop/slowest/response_length:362 - timing_s/gen:8.412553956732154 - timing_s/reward:0.07087826356291771 - timing_s/old_log_prob:2.5032861474901438 - timing_s/adv:0.6022136602550745 - timing_s/update_actor:15.354411419481039 - timing_s/step:27.029569501057267 - timing_s/stop_profile:2.7680769562721252e-05 - timing_per_token_ms/update_actor:0.08917443792385495 - timing_per_token_ms/gen:0.3723687126740507 - timing_per_token_ms/adv:0.0034975006984102734 - perf/total_num_tokens:172184 - perf/time_per_step:27.029569501057267 - perf/throughput:796.2760930823601 (TaskRunner pid=2204243) Training Progress: 16%|█▌ | 16/100 [10:40<1:02:08, 44.38s/it] (TaskRunner pid=2204243) step:17 - global_seqlen/min:19637 - global_seqlen/max:26737 - global_seqlen/minmax_diff:7100 - global_seqlen/balanced_min:22022 - global_seqlen/balanced_max:29242 - global_seqlen/mean:22925.25 - actor/entropy:0.13723574578762054 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2582600712776184 - training/rollout_probs_diff_mean:0.0035684085451066494 - training/rollout_probs_diff_std:0.012286622077226639 - training/rollout_actor_probs_pearson_corr:0.9978755712509155 - self_distillation/success_group_fraction:0.40625 - self_distillation/success_sample_fraction:0.40625 - self_distillation/correct_sample_fraction:0.40234375 - self_distillation/feedback_available_fraction:0.59765625 - self_distillation/feedback_used_fraction:0.59375 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000933408737183 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.1444564228877425e-05 - rollout_corr/rollout_is_max:1.8658337593078613 - rollout_corr/rollout_is_min:0.4920615255832672 - rollout_corr/rollout_is_std:0.03550882637500763 - rollout_corr/rollout_is_eff_sample_size:0.998740830061587 - rollout_corr/rollout_is_seq_mean:1.0004183053970337 - rollout_corr/rollout_is_seq_std:0.004439284559339285 - rollout_corr/rollout_is_seq_max:1.0172277688980103 - rollout_corr/rollout_is_seq_min:0.985140323638916 - rollout_corr/rollout_is_seq_max_deviation:0.017227768898010254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1754278410226107) - rollout_corr/training_log_ppl:np.float64(0.15572502662962506) - rollout_corr/kl:np.float64(0.0004782027402878697) - rollout_corr/k3_kl:np.float64(0.0010337071946722176) - rollout_corr/rollout_ppl:np.float64(1.1747629027813673) - rollout_corr/rollout_log_ppl:np.float64(0.15524682396699063) - rollout_corr/log_ppl_diff:np.float64(0.00047820266263443045) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036260130545997526) - rollout_corr/log_ppl_diff_max:np.float64(0.017128705978393555) - rollout_corr/log_ppl_diff_min:np.float64(-0.020957063883543015) - rollout_corr/ppl_ratio:np.float64(1.000490999314934) - rollout_corr/chi2_token:np.float64(0.0033014367800205946) - rollout_corr/chi2_seq:np.float64(0.2546185730025172) - actor/pg_loss:np.float64(0.000275188940577209) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010247878162772395) - actor/ppo_kl:np.float64(6.998297881466442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40234375) - self_distillation/token_reweight_w_mean:np.float64(213.85709377052262) - self_distillation/token_reweight_w_std:np.float64(2096.1374244292965) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997129596304148) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0057285287184640765) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.09788979031145573) - perf/mfu/actor:np.float64(0.039936719995079986) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(143.6447868347168) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000742877833545208 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000742877833545208 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:124.2265625 - response_length/max:8192.0 - response_length/min:41.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:124.2265625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:41.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:592.1875 - prompt_length/max:1091.0 - prompt_length/min:353.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.3899667263031e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6088914647698402) - timing_s/agent_loop/generate_sequences/max:np.float64(46.61487030424178) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.3251419956068275) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(46.61487030424178) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:595 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.81660504639149 - timing_s/reward:0.07286571525037289 - timing_s/old_log_prob:2.5335957184433937 - timing_s/adv:0.5873170793056488 - timing_s/update_actor:16.558109875768423 - timing_s/step:72.65578968264163 - timing_s/stop_profile:3.7262216210365295e-05 - timing_per_token_ms/update_actor:0.09028314781610028 - timing_per_token_ms/gen:1.6607950772401574 - timing_per_token_ms/adv:0.003202348280311277 - perf/total_num_tokens:183402 - perf/time_per_step:72.65578968264163 - perf/throughput:315.5323216516787 (TaskRunner pid=2204243) Training Progress: 17%|█▋ | 17/100 [11:53<1:13:10, 52.89s/it] (TaskRunner pid=2204243) step:18 - global_seqlen/min:17484 - global_seqlen/max:34291 - global_seqlen/minmax_diff:16807 - global_seqlen/balanced_min:22352 - global_seqlen/balanced_max:30020 - global_seqlen/mean:23340.625 - actor/entropy:0.1662994772195816 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2773323357105255 - training/rollout_probs_diff_mean:0.0033141474705189466 - training/rollout_probs_diff_std:0.012316255830228329 - training/rollout_actor_probs_pearson_corr:0.9980096220970154 - self_distillation/success_group_fraction:0.40625 - self_distillation/success_sample_fraction:0.40625 - self_distillation/correct_sample_fraction:0.38671875 - self_distillation/feedback_available_fraction:0.61328125 - self_distillation/feedback_used_fraction:0.59375 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.000059723854065 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.2445412216475233e-05 - rollout_corr/rollout_is_max:1.992296814918518 - rollout_corr/rollout_is_min:0.389501690864563 - rollout_corr/rollout_is_std:0.034339748322963715 - rollout_corr/rollout_is_eff_sample_size:0.9988222895543597 - rollout_corr/rollout_is_seq_mean:1.0000029802322388 - rollout_corr/rollout_is_seq_std:0.004191135056316853 - rollout_corr/rollout_is_seq_max:1.0129445791244507 - rollout_corr/rollout_is_seq_min:0.9896302223205566 - rollout_corr/rollout_is_seq_max_deviation:0.012944579124450684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1759867602959275) - rollout_corr/training_log_ppl:np.float64(0.1413091654649179) - rollout_corr/kl:np.float64(0.0009373990615131333) - rollout_corr/k3_kl:np.float64(0.0008979423765822503) - rollout_corr/rollout_ppl:np.float64(1.1748770074918866) - rollout_corr/rollout_log_ppl:np.float64(0.14037176547572017) - rollout_corr/log_ppl_diff:np.float64(0.0009373999891977292) - rollout_corr/log_ppl_abs_diff:np.float64(0.003861199253151426) - rollout_corr/log_ppl_diff_max:np.float64(0.02352091670036316) - rollout_corr/log_ppl_diff_min:np.float64(-0.013352692127227783) - rollout_corr/ppl_ratio:np.float64(1.0009507380891591) - rollout_corr/chi2_token:np.float64(0.0017263228073716164) - rollout_corr/chi2_seq:np.float64(0.22461562207899988) - actor/pg_loss:np.float64(0.0013536256738007069) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.3879310144111514e-05) - actor/ppo_kl:np.float64(0.00018159921697247228) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.38671875) - self_distillation/token_reweight_w_mean:np.float64(48179.300504226005) - self_distillation/token_reweight_w_std:np.float64(489912.16841455095) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997732501942664) - self_distillation/token_reweight_w_clip_frac:np.float64(0.015619186277035624) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.3859127089381218) - perf/mfu/actor:np.float64(0.040177895624024934) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(143.70674896240234) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004692417569458485 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004692417569458485 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:120.39453125 - response_length/max:8192.0 - response_length/min:29.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:120.39453125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:609.0 - prompt_length/max:1133.0 - prompt_length/min:320.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.3408945202827454e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4475587550550699) - timing_s/agent_loop/generate_sequences/max:np.float64(43.882089495658875) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1723115561835584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(43.882089495658875) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1133 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:49.70155741274357 - timing_s/reward:0.06336938217282295 - timing_s/old_log_prob:2.543882865458727 - timing_s/adv:0.6199127919971943 - timing_s/update_actor:16.801265999674797 - timing_s/step:69.76396719552577 - timing_s/stop_profile:2.829916775226593e-05 - timing_per_token_ms/update_actor:0.08997866380867478 - timing_per_token_ms/gen:1.6125874375504872 - timing_per_token_ms/adv:0.0033199239094775434 - perf/total_num_tokens:186725 - perf/time_per_step:69.76396719552577 - perf/throughput:334.5656208825367 (TaskRunner pid=2204243) Training Progress: 18%|█▊ | 18/100 [13:02<1:19:13, 57.97s/it] (TaskRunner pid=2204243) step:19 - global_seqlen/min:19063 - global_seqlen/max:24676 - global_seqlen/minmax_diff:5613 - global_seqlen/balanced_min:20995 - global_seqlen/balanced_max:20999 - global_seqlen/mean:20996.75 - actor/entropy:0.13203923404216766 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24134299159049988 - training/rollout_probs_diff_mean:0.003839091630652547 - training/rollout_probs_diff_std:0.013071263208985329 - training/rollout_actor_probs_pearson_corr:0.9974849224090576 - self_distillation/success_group_fraction:0.5 - self_distillation/success_sample_fraction:0.5 - self_distillation/correct_sample_fraction:0.44921875 - self_distillation/feedback_available_fraction:0.55078125 - self_distillation/feedback_used_fraction:0.5 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999523758888245 - rollout_corr/rollout_is_ratio_fraction_high:4.064049426233396e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020320246403571218 - rollout_corr/rollout_is_max:2.737149477005005 - rollout_corr/rollout_is_min:0.33375799655914307 - rollout_corr/rollout_is_std:0.037705305963754654 - rollout_corr/rollout_is_eff_sample_size:0.9985799717327595 - rollout_corr/rollout_is_seq_mean:0.9999047517776489 - rollout_corr/rollout_is_seq_std:0.0044456059113144875 - rollout_corr/rollout_is_seq_max:1.0194222927093506 - rollout_corr/rollout_is_seq_min:0.9853204488754272 - rollout_corr/rollout_is_seq_max_deviation:0.019422292709350586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1342645599506795) - rollout_corr/training_log_ppl:np.float64(0.12157863749598619) - rollout_corr/kl:np.float64(0.0013247929520616708) - rollout_corr/k3_kl:np.float64(0.0010882196712538317) - rollout_corr/rollout_ppl:np.float64(1.1328144851140678) - rollout_corr/rollout_log_ppl:np.float64(0.1202538444485981) - rollout_corr/log_ppl_diff:np.float64(0.0013247930473880842) - rollout_corr/log_ppl_abs_diff:np.float64(0.003940825452446006) - rollout_corr/log_ppl_diff_max:np.float64(0.02686990797519684) - rollout_corr/log_ppl_diff_min:np.float64(-0.015836119651794434) - rollout_corr/ppl_ratio:np.float64(1.0013401887845248) - rollout_corr/chi2_token:np.float64(0.0016203562263399363) - rollout_corr/chi2_seq:np.float64(0.10664522647857666) - actor/pg_loss:np.float64(0.0009464026661589742) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006123458115325775) - actor/ppo_kl:np.float64(0.0005135015452637504) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44921875) - self_distillation/token_reweight_w_mean:np.float64(253684.17952676094) - self_distillation/token_reweight_w_std:np.float64(2037986.8133215215) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999500410631299) - self_distillation/token_reweight_w_clip_frac:np.float64(0.021009555392083712) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.7424543350934982) - perf/mfu/actor:np.float64(0.03831287601090984) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(143.63737630844116) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038811671547591686 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038811671547591686 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:96.1171875 - response_length/max:532.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:96.1171875 - response_length_non_aborted/max:532.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:560.03125 - prompt_length/max:1163.0 - prompt_length/min:344.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.755783498287201e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4376736767590046) - timing_s/agent_loop/generate_sequences/max:np.float64(3.308537792414427) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9640539156389423) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.308537792414427) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:576 - timing_s/agent_loop/slowest/response_length:529 - timing_s/gen:9.182144444435835 - timing_s/reward:0.059934504330158234 - timing_s/old_log_prob:2.4501505363732576 - timing_s/adv:0.5914808381348848 - timing_s/update_actor:15.765520054847002 - timing_s/step:28.082664351910353 - timing_s/stop_profile:2.9847025871276855e-05 - timing_per_token_ms/update_actor:0.09385690675251528 - timing_per_token_ms/gen:0.3731668879312296 - timing_per_token_ms/adv:0.0035212642321721505 - perf/total_num_tokens:167974 - perf/time_per_step:28.082664351910353 - perf/throughput:747.6765643346685 (TaskRunner pid=2204243) Training Progress: 19%|█▉ | 19/100 [13:31<1:06:08, 49.00s/it] (TaskRunner pid=2204243) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2204243) validation generation end (TaskRunner pid=2204243) [prompt] user (TaskRunner pid=2204243) Your task is to answer the user's question using available tools. (TaskRunner pid=2204243) You have access to the following tools: (TaskRunner pid=2204243) Name: Axolotl (TaskRunner pid=2204243) Description: Collection of axolotl pictures and facts (TaskRunner pid=2204243) Documentation: (TaskRunner pid=2204243) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2204243) Parameters: {} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{url, source, description} (TaskRunner pid=2204243) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2204243) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2204243) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2204243) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Array[Object{fact, source}] (TaskRunner pid=2204243) (TaskRunner pid=2204243) Use the following format: (TaskRunner pid=2204243) Thought: you should always think about what to do (TaskRunner pid=2204243) Action: the action to take, should be one of the tool names. (TaskRunner pid=2204243) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2204243) (TaskRunner pid=2204243) Begin! (TaskRunner pid=2204243) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2204243) assistant (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: The user is asking for a random picture of an axolotl. The appropriate tool for this is getRandomAxolotlImage, which retrieves a random axolotl image with information on the image source. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (TaskRunner pid=2204243) step:20 - global_seqlen/min:19328 - global_seqlen/max:32043 - global_seqlen/minmax_diff:12715 - global_seqlen/balanced_min:24133 - global_seqlen/balanced_max:24139 - global_seqlen/mean:24136.375 - actor/entropy:0.13509546220302582 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2613658905029297 - training/rollout_probs_diff_mean:0.004037723410874605 - training/rollout_probs_diff_std:0.01323476992547512 - training/rollout_actor_probs_pearson_corr:0.9974843859672546 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.375 - self_distillation/correct_sample_fraction:0.3125 - self_distillation/feedback_available_fraction:0.6875 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.999825656414032 - rollout_corr/rollout_is_ratio_fraction_high:9.511579992249608e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.03082275390625 - rollout_corr/rollout_is_min:0.5086526274681091 - rollout_corr/rollout_is_std:0.038519326597452164 - rollout_corr/rollout_is_eff_sample_size:0.9985180437987692 - rollout_corr/rollout_is_seq_mean:0.9998537302017212 - rollout_corr/rollout_is_seq_std:0.004693410824984312 - rollout_corr/rollout_is_seq_max:1.0225037336349487 - rollout_corr/rollout_is_seq_min:0.9843899011611938 - rollout_corr/rollout_is_seq_max_deviation:0.02250373363494873 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1422954429872334) - rollout_corr/training_log_ppl:np.float64(0.12942408723029075) - rollout_corr/kl:np.float64(0.00153353812269863) - rollout_corr/k3_kl:np.float64(0.001262979240971518) - rollout_corr/rollout_ppl:np.float64(1.1404378479346633) - rollout_corr/rollout_log_ppl:np.float64(0.1278905481703987) - rollout_corr/log_ppl_diff:np.float64(0.001533539059892064) - rollout_corr/log_ppl_abs_diff:np.float64(0.004827938540984178) - rollout_corr/log_ppl_diff_max:np.float64(0.03444957733154297) - rollout_corr/log_ppl_diff_min:np.float64(-0.025876760482788086) - rollout_corr/ppl_ratio:np.float64(1.0015575678553432) - rollout_corr/chi2_token:np.float64(0.0021749071311205626) - rollout_corr/chi2_seq:np.float64(0.14012844231911004) - actor/pg_loss:np.float64(0.0012385804438963532) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029507137878681533) - actor/ppo_kl:np.float64(0.0006677311765678207) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3125) - self_distillation/token_reweight_w_mean:np.float64(46022.63570580399) - self_distillation/token_reweight_w_std:np.float64(399565.44032346434) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004668897017837) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02515652126749046) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.6425938457250595) - perf/mfu/actor:np.float64(0.043702184071574085) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(143.72253799438477) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5863970588235294) - val-aux/tooluse/reward/std@16:np.float64(0.04582179821616536) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6030441176470589) - val-aux/tooluse/reward/best@2/std:np.float64(0.029364274034540096) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.568764705882353) - val-aux/tooluse/reward/worst@2/std:np.float64(0.0477393225087001) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5853235294117646) - val-aux/tooluse/reward/maj@2/std:np.float64(0.04688011435889171) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6117499999999999) - val-aux/tooluse/reward/best@4/std:np.float64(0.017567233786550646) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5494852941176472) - val-aux/tooluse/reward/worst@4/std:np.float64(0.04589940870036069) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.592970588235294) - val-aux/tooluse/reward/maj@4/std:np.float64(0.03220154710732844) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6188382352941177) - val-aux/tooluse/reward/best@8/std:np.float64(0.014158354973193033) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5292205882352942) - val-aux/tooluse/reward/worst@8/std:np.float64(0.038624477400439734) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5968088235294119) - val-aux/tooluse/reward/maj@8/std:np.float64(0.017867101316453672) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6255588235294117) - val-aux/tooluse/reward/best@16/std:np.float64(0.011742647096355837) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.513485294117647) - val-aux/tooluse/reward/worst@16/std:np.float64(0.02665920585732603) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.6001323529411764) - val-aux/tooluse/reward/maj@16/std:np.float64(0.009429342504448173) - val-aux/tooluse/score/mean@16:np.float64(0.5863970588235294) - val-aux/tooluse/score/std@16:np.float64(0.04582179821616536) - val-aux/tooluse/score/best@2/mean:np.float64(0.6030441176470589) - val-aux/tooluse/score/best@2/std:np.float64(0.029364274034540096) - val-aux/tooluse/score/worst@2/mean:np.float64(0.568764705882353) - val-aux/tooluse/score/worst@2/std:np.float64(0.0477393225087001) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5853235294117646) - val-aux/tooluse/score/maj@2/std:np.float64(0.04688011435889171) - val-aux/tooluse/score/best@4/mean:np.float64(0.6117499999999999) - val-aux/tooluse/score/best@4/std:np.float64(0.017567233786550646) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5494852941176472) - val-aux/tooluse/score/worst@4/std:np.float64(0.04589940870036069) - val-aux/tooluse/score/maj@4/mean:np.float64(0.592970588235294) - val-aux/tooluse/score/maj@4/std:np.float64(0.03220154710732844) - val-aux/tooluse/score/best@8/mean:np.float64(0.6188382352941177) - val-aux/tooluse/score/best@8/std:np.float64(0.014158354973193033) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5292205882352942) - val-aux/tooluse/score/worst@8/std:np.float64(0.038624477400439734) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5968088235294119) - val-aux/tooluse/score/maj@8/std:np.float64(0.017867101316453672) - val-aux/tooluse/score/best@16/mean:np.float64(0.6255588235294117) - val-aux/tooluse/score/best@16/std:np.float64(0.011742647096355837) - val-aux/tooluse/score/worst@16/mean:np.float64(0.513485294117647) - val-aux/tooluse/score/worst@16/std:np.float64(0.02665920585732603) - val-aux/tooluse/score/maj@16/mean:np.float64(0.6001323529411764) - val-aux/tooluse/score/maj@16/std:np.float64(0.009429342504448173) - val-core/tooluse/acc/mean@16:np.float64(0.5863970588235294) - val-aux/tooluse/acc/std@16:np.float64(0.04582179821616536) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6030441176470589) - val-aux/tooluse/acc/best@2/std:np.float64(0.029364274034540096) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.568764705882353) - val-aux/tooluse/acc/worst@2/std:np.float64(0.0477393225087001) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5853235294117646) - val-aux/tooluse/acc/maj@2/std:np.float64(0.04688011435889171) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6117499999999999) - val-aux/tooluse/acc/best@4/std:np.float64(0.017567233786550646) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5494852941176472) - val-aux/tooluse/acc/worst@4/std:np.float64(0.04589940870036069) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.592970588235294) - val-aux/tooluse/acc/maj@4/std:np.float64(0.03220154710732844) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6188382352941177) - val-aux/tooluse/acc/best@8/std:np.float64(0.014158354973193033) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5292205882352942) - val-aux/tooluse/acc/worst@8/std:np.float64(0.038624477400439734) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5968088235294119) - val-aux/tooluse/acc/maj@8/std:np.float64(0.017867101316453672) - val-core/tooluse/acc/best@16/mean:np.float64(0.6255588235294117) - val-core/tooluse/acc/best@16/std:np.float64(0.011742647096355837) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.513485294117647) - val-aux/tooluse/acc/worst@16/std:np.float64(0.02665920585732603) - val-core/tooluse/acc/maj@16/mean:np.float64(0.6001323529411764) - val-core/tooluse/acc/maj@16/std:np.float64(0.009429342504448173) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.012008369900286198 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.012008369900286198 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:82.13671875 - response_length/max:231.0 - response_length/min:26.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:82.13671875 - response_length_non_aborted/max:231.0 - response_length_non_aborted/min:26.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:672.125 - prompt_length/max:1518.0 - prompt_length/min:256.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7406960725784302e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.41428385861217976) - timing_s/agent_loop/generate_sequences/max:np.float64(1.779176840558648) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.986818541532557) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.779176840558648) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:256 - timing_s/agent_loop/slowest/response_length:231 - timing_s/gen:7.6312979981303215 - timing_s/reward:0.06997117027640343 - timing_s/old_log_prob:2.343086402863264 - timing_s/adv:0.6590765044093132 - timing_s/update_actor:15.942656245082617 - timing_s/step:26.68303732573986 - timing_s/testing:20.753679210320115 - timing_s/save_checkpoint:6.566767690703273 - timing_s/stop_profile:0.0001407284289598465 - timing_per_token_ms/update_actor:0.0825655066527317 - timing_per_token_ms/gen:0.3629285203847587 - timing_per_token_ms/adv:0.003413294790587408 - perf/total_num_tokens:193091 - perf/time_per_step:26.68303732573986 - perf/throughput:904.5587541384123 (TaskRunner pid=2204243) Training Progress: 20%|██ | 20/100 [14:25<1:07:21, 50.52s/it] (WorkerDict pid=2204655) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2204243) step:21 - global_seqlen/min:18281 - global_seqlen/max:26209 - global_seqlen/minmax_diff:7928 - global_seqlen/balanced_min:22463 - global_seqlen/balanced_max:22473 - global_seqlen/mean:22470.25 - actor/entropy:0.16318483650684357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135541915893555 - training/rollout_probs_diff_mean:0.0045090289786458015 - training/rollout_probs_diff_std:0.01366505492478609 - training/rollout_actor_probs_pearson_corr:0.9976838827133179 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.33984375 - self_distillation/correct_sample_fraction:0.3125 - self_distillation/feedback_available_fraction:0.6875 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.0002528429031372 - rollout_corr/rollout_is_ratio_fraction_high:4.6533270506188273e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.010491132736206 - rollout_corr/rollout_is_min:0.5044595003128052 - rollout_corr/rollout_is_std:0.039867497980594635 - rollout_corr/rollout_is_eff_sample_size:0.9984138177693591 - rollout_corr/rollout_is_seq_mean:1.0002862215042114 - rollout_corr/rollout_is_seq_std:0.00472763879224658 - rollout_corr/rollout_is_seq_max:1.0192320346832275 - rollout_corr/rollout_is_seq_min:0.9857943654060364 - rollout_corr/rollout_is_seq_max_deviation:0.01923203468322754 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.164235977921635) - rollout_corr/training_log_ppl:np.float64(0.14601151449460303) - rollout_corr/kl:np.float64(0.0003984103697263208) - rollout_corr/k3_kl:np.float64(0.0009126389778899013) - rollout_corr/rollout_ppl:np.float64(1.1638221787288785) - rollout_corr/rollout_log_ppl:np.float64(0.14561310308636166) - rollout_corr/log_ppl_diff:np.float64(0.00039841140824137256) - rollout_corr/log_ppl_abs_diff:np.float64(0.003489054382953327) - rollout_corr/log_ppl_diff_max:np.float64(0.017350539565086365) - rollout_corr/log_ppl_diff_min:np.float64(-0.012856259942054749) - rollout_corr/ppl_ratio:np.float64(1.0004089111462235) - rollout_corr/chi2_token:np.float64(0.0030136662535369396) - rollout_corr/chi2_seq:np.float64(0.20880368584766984) - actor/pg_loss:np.float64(0.00035233062226325274) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016748869165894575) - actor/ppo_kl:np.float64(-7.641052897877643e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3125) - self_distillation/token_reweight_w_mean:np.float64(56018.450126351556) - self_distillation/token_reweight_w_std:np.float64(456781.9859616625) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006133876740932) - self_distillation/token_reweight_w_clip_frac:np.float64(0.010979209560900927) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.1436886489391327) - perf/mfu/actor:np.float64(0.041583387370465706) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.42555618286133) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015414146473631263 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015414146473631263 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:83.9453125 - response_length/max:407.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:83.9453125 - response_length_non_aborted/max:407.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:618.25 - prompt_length/max:948.0 - prompt_length/min:253.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.368546307086945e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.43967240676283836) - timing_s/agent_loop/generate_sequences/max:np.float64(2.6777198780328035) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9358330626928364) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.6777198780328035) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:869 - timing_s/agent_loop/slowest/response_length:407 - timing_s/gen:8.13176535628736 - timing_s/reward:0.06986739858984947 - timing_s/old_log_prob:2.472238589078188 - timing_s/adv:0.603289719671011 - timing_s/update_actor:15.426496025174856 - timing_s/step:26.789035188034177 - timing_s/stop_profile:0.00011346489191055298 - timing_per_token_ms/update_actor:0.08581622381356936 - timing_per_token_ms/gen:0.37839764338238063 - timing_per_token_ms/adv:0.003356046993641654 - perf/total_num_tokens:179762 - perf/time_per_step:26.789035188034177 - perf/throughput:838.7853404305041 (TaskRunner pid=2204243) Training Progress: 21%|██ | 21/100 [14:51<57:09, 43.41s/it] (TaskRunner pid=2204243) step:22 - global_seqlen/min:16798 - global_seqlen/max:30493 - global_seqlen/minmax_diff:13695 - global_seqlen/balanced_min:22871 - global_seqlen/balanced_max:22878 - global_seqlen/mean:22876.0 - actor/entropy:0.1423337608575821 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2910901606082916 - training/rollout_probs_diff_mean:0.003937973640859127 - training/rollout_probs_diff_std:0.013108985498547554 - training/rollout_actor_probs_pearson_corr:0.9975479245185852 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.375 - self_distillation/correct_sample_fraction:0.359375 - self_distillation/feedback_available_fraction:0.640625 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.999701201915741 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00019993602472823113 - rollout_corr/rollout_is_max:1.81659996509552 - rollout_corr/rollout_is_min:0.42187532782554626 - rollout_corr/rollout_is_std:0.037015292793512344 - rollout_corr/rollout_is_eff_sample_size:0.9986309699827669 - rollout_corr/rollout_is_seq_mean:0.9994570016860962 - rollout_corr/rollout_is_seq_std:0.004646854940801859 - rollout_corr/rollout_is_seq_max:1.0205856561660767 - rollout_corr/rollout_is_seq_min:0.9852480888366699 - rollout_corr/rollout_is_seq_max_deviation:0.02058565616607666 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1465901811607182) - rollout_corr/training_log_ppl:np.float64(0.13207034853985533) - rollout_corr/kl:np.float64(0.0012624080195973164) - rollout_corr/k3_kl:np.float64(0.0009608019858000638) - rollout_corr/rollout_ppl:np.float64(1.1451159128919244) - rollout_corr/rollout_log_ppl:np.float64(0.1308079400259885) - rollout_corr/log_ppl_diff:np.float64(0.0012624085138668306) - rollout_corr/log_ppl_abs_diff:np.float64(0.003915511661034543) - rollout_corr/log_ppl_diff_max:np.float64(0.03924417495727539) - rollout_corr/log_ppl_diff_min:np.float64(-0.01869313418865204) - rollout_corr/ppl_ratio:np.float64(1.0012805289588869) - rollout_corr/chi2_token:np.float64(0.0012336813379079103) - rollout_corr/chi2_seq:np.float64(0.057158841053023934) - actor/pg_loss:np.float64(0.00018166203517466784) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040142949001165107) - actor/ppo_kl:np.float64(3.125276354598583e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.359375) - self_distillation/token_reweight_w_mean:np.float64(74608.66255049617) - self_distillation/token_reweight_w_std:np.float64(522300.32484421605) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998402339406312) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006556528634973802) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.24255643039941788) - perf/mfu/actor:np.float64(0.04235944030344961) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.34895706176758) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0035388674587011337 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0035388674587011337 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:97.6875 - response_length/max:586.0 - response_length/min:30.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:97.6875 - response_length_non_aborted/max:586.0 - response_length_non_aborted/min:30.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:617.1875 - prompt_length/max:1282.0 - prompt_length/min:231.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001120585948228836 - timing_s/agent_loop/generate_sequences/min:np.float64(0.430797154083848) - timing_s/agent_loop/generate_sequences/max:np.float64(3.5702572353184223) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0388906223670347) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.5702572353184223) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:596 - timing_s/agent_loop/slowest/response_length:586 - timing_s/gen:10.059444611892104 - timing_s/reward:0.07228243909776211 - timing_s/old_log_prob:2.585768071934581 - timing_s/adv:0.6817157976329327 - timing_s/update_actor:15.463645920157433 - timing_s/step:28.96110754273832 - timing_s/stop_profile:0.00012063980102539062 - timing_per_token_ms/update_actor:0.0844971035154607 - timing_per_token_ms/gen:0.4022490647749562 - timing_per_token_ms/adv:0.003725060093727775 - perf/total_num_tokens:183008 - perf/time_per_step:28.96110754273832 - perf/throughput:789.886918732012 (TaskRunner pid=2204243) Training Progress: 22%|██▏ | 22/100 [15:21<50:48, 39.09s/it] (TaskRunner pid=2204243) step:23 - global_seqlen/min:16826 - global_seqlen/max:23478 - global_seqlen/minmax_diff:6652 - global_seqlen/balanced_min:19655 - global_seqlen/balanced_max:20435 - global_seqlen/mean:19764.875 - actor/entropy:0.13859252631664276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18456989526748657 - training/rollout_probs_diff_mean:0.003764737630262971 - training/rollout_probs_diff_std:0.012588140554726124 - training/rollout_actor_probs_pearson_corr:0.9977132678031921 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.49609375 - self_distillation/feedback_available_fraction:0.50390625 - self_distillation/feedback_used_fraction:0.40625 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:0.9998646378517151 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.611057374859229e-05 - rollout_corr/rollout_is_max:1.5884497165679932 - rollout_corr/rollout_is_min:0.37117940187454224 - rollout_corr/rollout_is_std:0.03559098020195961 - rollout_corr/rollout_is_eff_sample_size:0.9987344090023405 - rollout_corr/rollout_is_seq_mean:0.9998131394386292 - rollout_corr/rollout_is_seq_std:0.004723796620965004 - rollout_corr/rollout_is_seq_max:1.0255852937698364 - rollout_corr/rollout_is_seq_min:0.9818201065063477 - rollout_corr/rollout_is_seq_max_deviation:0.025585293769836426 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1443326668813825) - rollout_corr/training_log_ppl:np.float64(0.13088138688908657) - rollout_corr/kl:np.float64(0.0009826503217595928) - rollout_corr/k3_kl:np.float64(0.001333915184940082) - rollout_corr/rollout_ppl:np.float64(1.1431675525382161) - rollout_corr/rollout_log_ppl:np.float64(0.12989873687547515) - rollout_corr/log_ppl_diff:np.float64(0.0009826500136114191) - rollout_corr/log_ppl_abs_diff:np.float64(0.00505472802251461) - rollout_corr/log_ppl_diff_max:np.float64(0.02458462119102478) - rollout_corr/log_ppl_diff_min:np.float64(-0.031808577477931976) - rollout_corr/ppl_ratio:np.float64(1.0010055068414658) - rollout_corr/chi2_token:np.float64(0.003803333267569542) - rollout_corr/chi2_seq:np.float64(0.2107151655945927) - actor/pg_loss:np.float64(0.001238261815160513) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009233404216502095) - actor/ppo_kl:np.float64(7.400157875281366e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.49609375) - self_distillation/token_reweight_w_mean:np.float64(252351.39504893008) - self_distillation/token_reweight_w_std:np.float64(1703078.905524379) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008135486859828) - self_distillation/token_reweight_w_clip_frac:np.float64(0.037577129929559305) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.5660313069820404) - perf/mfu/actor:np.float64(0.036456099855030705) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.37195587158203) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.49609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.49609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0079194912686944 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0079194912686944 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:84.71484375 - response_length/max:1354.0 - response_length/min:33.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:84.71484375 - response_length_non_aborted/max:1354.0 - response_length_non_aborted/min:33.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:532.9375 - prompt_length/max:845.0 - prompt_length/min:244.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010920129716396332 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4573214165866375) - timing_s/agent_loop/generate_sequences/max:np.float64(6.88495240919292) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8981881432337104) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.88495240919292) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:804 - timing_s/agent_loop/slowest/response_length:1354 - timing_s/gen:12.730165127664804 - timing_s/reward:0.06851024739444256 - timing_s/old_log_prob:2.3535508401691914 - timing_s/adv:0.5833722278475761 - timing_s/update_actor:15.605271374806762 - timing_s/step:31.42830360494554 - timing_s/stop_profile:0.00012807361781597137 - timing_per_token_ms/update_actor:0.098693208120509 - timing_per_token_ms/gen:0.5869952103870892 - timing_per_token_ms/adv:0.0036894505268030794 - perf/total_num_tokens:158119 - perf/time_per_step:31.42830360494554 - perf/throughput:628.8877455316999 (TaskRunner pid=2204243) Training Progress: 23%|██▎ | 23/100 [15:52<47:14, 36.81s/it] (TaskRunner pid=2204243) step:24 - global_seqlen/min:16430 - global_seqlen/max:24627 - global_seqlen/minmax_diff:8197 - global_seqlen/balanced_min:20776 - global_seqlen/balanced_max:20780 - global_seqlen/mean:20779.0 - actor/entropy:0.13504554331302643 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23082181811332703 - training/rollout_probs_diff_mean:0.0034335304517298937 - training/rollout_probs_diff_std:0.01163981482386589 - training/rollout_actor_probs_pearson_corr:0.9979947805404663 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.3125 - self_distillation/correct_sample_fraction:0.26953125 - self_distillation/feedback_available_fraction:0.73046875 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999052286148071 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.032206172472797e-05 - rollout_corr/rollout_is_max:1.8593839406967163 - rollout_corr/rollout_is_min:0.4920622408390045 - rollout_corr/rollout_is_std:0.03346240893006325 - rollout_corr/rollout_is_eff_sample_size:0.9988811627143783 - rollout_corr/rollout_is_seq_mean:0.9999405741691589 - rollout_corr/rollout_is_seq_std:0.004011598881334066 - rollout_corr/rollout_is_seq_max:1.0140236616134644 - rollout_corr/rollout_is_seq_min:0.9853073954582214 - rollout_corr/rollout_is_seq_max_deviation:0.014692604541778564 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1473897430114448) - rollout_corr/training_log_ppl:np.float64(0.1309895477243117) - rollout_corr/kl:np.float64(0.000908215488436781) - rollout_corr/k3_kl:np.float64(0.0006320715643724079) - rollout_corr/rollout_ppl:np.float64(1.1463820012286305) - rollout_corr/rollout_log_ppl:np.float64(0.13008133043695125) - rollout_corr/log_ppl_diff:np.float64(0.0009082172873604577) - rollout_corr/log_ppl_abs_diff:np.float64(0.003190347688359907) - rollout_corr/log_ppl_diff_max:np.float64(0.01913510262966156) - rollout_corr/log_ppl_diff_min:np.float64(-0.014032378792762756) - rollout_corr/ppl_ratio:np.float64(1.00091787497513) - rollout_corr/chi2_token:np.float64(0.0007186408620327711) - rollout_corr/chi2_seq:np.float64(0.03952546953223646) - actor/pg_loss:np.float64(0.00043765874579548836) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003319840907352045) - actor/ppo_kl:np.float64(0.00026008376003261446) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.26953125) - self_distillation/token_reweight_w_mean:np.float64(329286.216579993) - self_distillation/token_reweight_w_std:np.float64(2135011.63086932) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000414610374719) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014716415636939928) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.5758811086416245) - perf/mfu/actor:np.float64(0.03771007143924178) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.15919876098633) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.26953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.26953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0041452799923717976 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0041452799923717976 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:77.625 - response_length/max:336.0 - response_length/min:23.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:77.625 - response_length_non_aborted/max:336.0 - response_length_non_aborted/min:23.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:571.71875 - prompt_length/max:887.0 - prompt_length/min:300.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011283904314041138 - timing_s/agent_loop/generate_sequences/min:np.float64(0.43510134145617485) - timing_s/agent_loop/generate_sequences/max:np.float64(2.287069333717227) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.933089457830647) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.287069333717227) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:760 - timing_s/agent_loop/slowest/response_length:336 - timing_s/gen:8.52804215438664 - timing_s/reward:0.07083887979388237 - timing_s/old_log_prob:2.3156927060335875 - timing_s/adv:0.5853530578315258 - timing_s/update_actor:15.747144574299455 - timing_s/step:27.33303796313703 - timing_s/stop_profile:0.00012150593101978302 - timing_per_token_ms/update_actor:0.09472992308520294 - timing_per_token_ms/gen:0.42914865913781397 - timing_per_token_ms/adv:0.003521301902350485 - perf/total_num_tokens:166232 - perf/time_per_step:27.33303796313703 - perf/throughput:760.2155321345473 (TaskRunner pid=2204243) Training Progress: 24%|██▍ | 24/100 [16:19<43:02, 33.98s/it] (TaskRunner pid=2204243) step:25 - global_seqlen/min:15990 - global_seqlen/max:24252 - global_seqlen/minmax_diff:8262 - global_seqlen/balanced_min:19847 - global_seqlen/balanced_max:19850 - global_seqlen/mean:19849.125 - actor/entropy:0.18370242416858673 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4400070011615753 - training/rollout_probs_diff_mean:0.004255072679370642 - training/rollout_probs_diff_std:0.013054358772933483 - training/rollout_actor_probs_pearson_corr:0.9980756044387817 - self_distillation/success_group_fraction:0.40625 - self_distillation/success_sample_fraction:0.40234375 - self_distillation/correct_sample_fraction:0.34375 - self_distillation/feedback_available_fraction:0.65625 - self_distillation/feedback_used_fraction:0.59375 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.0001846551895142 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.51602388441097e-05 - rollout_corr/rollout_is_max:1.5528991222381592 - rollout_corr/rollout_is_min:0.4614916741847992 - rollout_corr/rollout_is_std:0.037242449820041656 - rollout_corr/rollout_is_eff_sample_size:0.9986153965381223 - rollout_corr/rollout_is_seq_mean:1.0001070499420166 - rollout_corr/rollout_is_seq_std:0.00453194510191679 - rollout_corr/rollout_is_seq_max:1.017516851425171 - rollout_corr/rollout_is_seq_min:0.9857626557350159 - rollout_corr/rollout_is_seq_max_deviation:0.0175168514251709 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1987385815009475) - rollout_corr/training_log_ppl:np.float64(0.17369019936450059) - rollout_corr/kl:np.float64(0.0009225753080563948) - rollout_corr/k3_kl:np.float64(0.0009482696568774429) - rollout_corr/rollout_ppl:np.float64(1.19747136766091) - rollout_corr/rollout_log_ppl:np.float64(0.1727676240625442) - rollout_corr/log_ppl_diff:np.float64(0.0009225753019563854) - rollout_corr/log_ppl_abs_diff:np.float64(0.004314064281061292) - rollout_corr/log_ppl_diff_max:np.float64(0.025165602564811707) - rollout_corr/log_ppl_diff_min:np.float64(-0.015242800116539001) - rollout_corr/ppl_ratio:np.float64(1.0009386378806084) - rollout_corr/chi2_token:np.float64(0.00184961361810565) - rollout_corr/chi2_seq:np.float64(0.13827242306433618) - actor/pg_loss:np.float64(0.000449606915935874) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019199000598746352) - actor/ppo_kl:np.float64(0.0003383922945516815) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.34375) - self_distillation/token_reweight_w_mean:np.float64(132104.07613411802) - self_distillation/token_reweight_w_std:np.float64(1005546.3060252169) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009718094952404) - self_distillation/token_reweight_w_clip_frac:np.float64(0.017530345736304298) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.3739614896476269) - perf/mfu/actor:np.float64(0.03623387641499423) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.2127571105957) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0027235369198024273 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.0027235369198024273 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:70.81640625 - response_length/max:422.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:70.81640625 - response_length_non_aborted/max:422.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:549.46875 - prompt_length/max:938.0 - prompt_length/min:222.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016481243073940277 - timing_s/agent_loop/generate_sequences/min:np.float64(0.37646069936454296) - timing_s/agent_loop/generate_sequences/max:np.float64(2.625546058639884) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7982408093885169) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.625546058639884) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:593 - timing_s/agent_loop/slowest/response_length:422 - timing_s/gen:8.280904261395335 - timing_s/reward:0.06931519508361816 - timing_s/old_log_prob:2.2948220018297434 - timing_s/adv:0.5678768809884787 - timing_s/update_actor:15.682980556041002 - timing_s/step:26.98428513109684 - timing_s/stop_profile:0.00011578761041164398 - timing_per_token_ms/update_actor:0.09876367696334852 - timing_per_token_ms/gen:0.45677667060485055 - timing_per_token_ms/adv:0.003576208529270677 - perf/total_num_tokens:158793 - perf/time_per_step:26.98428513109684 - perf/throughput:735.5809095392991 (TaskRunner pid=2204243) Training Progress: 25%|██▌ | 25/100 [16:46<39:52, 31.90s/it] (TaskRunner pid=2204243) step:26 - global_seqlen/min:14521 - global_seqlen/max:28012 - global_seqlen/minmax_diff:13491 - global_seqlen/balanced_min:21510 - global_seqlen/balanced_max:21515 - global_seqlen/mean:21512.75 - actor/entropy:0.1577775925397873 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846141517162323 - training/rollout_probs_diff_mean:0.004070638213306665 - training/rollout_probs_diff_std:0.012744957581162453 - training/rollout_actor_probs_pearson_corr:0.997860848903656 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.4375 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.5859375 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000407695770264 - rollout_corr/rollout_is_ratio_fraction_high:5.2859712013741955e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.5906641483306885 - rollout_corr/rollout_is_min:0.5352824330329895 - rollout_corr/rollout_is_std:0.0377400666475296 - rollout_corr/rollout_is_eff_sample_size:0.9985775943190434 - rollout_corr/rollout_is_seq_mean:1.0000988245010376 - rollout_corr/rollout_is_seq_std:0.004820846952497959 - rollout_corr/rollout_is_seq_max:1.0355533361434937 - rollout_corr/rollout_is_seq_min:0.9851596355438232 - rollout_corr/rollout_is_seq_max_deviation:0.03555333614349365 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1572111090645194) - rollout_corr/training_log_ppl:np.float64(0.14145961670510587) - rollout_corr/kl:np.float64(0.0005020288480821478) - rollout_corr/k3_kl:np.float64(0.0012065820215196865) - rollout_corr/rollout_ppl:np.float64(1.1565070189535618) - rollout_corr/rollout_log_ppl:np.float64(0.1409575857760501) - rollout_corr/log_ppl_diff:np.float64(0.0005020309290557634) - rollout_corr/log_ppl_abs_diff:np.float64(0.00385038403692306) - rollout_corr/log_ppl_diff_max:np.float64(0.029768317937850952) - rollout_corr/log_ppl_diff_min:np.float64(-0.027933597564697266) - rollout_corr/ppl_ratio:np.float64(1.0005186614580452) - rollout_corr/chi2_token:np.float64(0.0038898237980902195) - rollout_corr/chi2_seq:np.float64(0.20818186062388122) - actor/pg_loss:np.float64(0.001697965431958437) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008745168306631967) - actor/ppo_kl:np.float64(-8.103693695460379e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(246694.13111479068) - self_distillation/token_reweight_w_std:np.float64(1435240.613249147) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001536488998681) - self_distillation/token_reweight_w_clip_frac:np.float64(0.023137171316193417) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.6526615135371685) - perf/mfu/actor:np.float64(0.03937853319621008) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.36329460144043) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001704725669696927 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001704725669696927 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:73.8984375 - response_length/max:341.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:73.8984375 - response_length_non_aborted/max:341.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:598.375 - prompt_length/max:1062.0 - prompt_length/min:237.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013505294919013977 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4072894621640444) - timing_s/agent_loop/generate_sequences/max:np.float64(2.1728067006915808) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8106240943889134) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.1728067006915808) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:938 - timing_s/agent_loop/slowest/response_length:341 - timing_s/gen:7.936154516413808 - timing_s/reward:0.07093050330877304 - timing_s/old_log_prob:2.3134337700903416 - timing_s/adv:0.6077794898301363 - timing_s/update_actor:15.503288498148322 - timing_s/step:26.51817707531154 - timing_s/stop_profile:0.0001125335693359375 - timing_per_token_ms/update_actor:0.09008197753743898 - timing_per_token_ms/gen:0.4195028288621317 - timing_per_token_ms/adv:0.003531507419031367 - perf/total_num_tokens:172102 - perf/time_per_step:26.51817707531154 - perf/throughput:811.2454313471042 (TaskRunner pid=2204243) Training Progress: 26%|██▌ | 26/100 [17:13<37:22, 30.30s/it] (TaskRunner pid=2204243) step:27 - global_seqlen/min:19762 - global_seqlen/max:23308 - global_seqlen/minmax_diff:3546 - global_seqlen/balanced_min:20937 - global_seqlen/balanced_max:20938 - global_seqlen/mean:20937.25 - actor/entropy:0.13456927239894867 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35835766792297363 - training/rollout_probs_diff_mean:0.0033772585447877645 - training/rollout_probs_diff_std:0.012435518205165863 - training/rollout_actor_probs_pearson_corr:0.9977062344551086 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.30859375 - self_distillation/correct_sample_fraction:0.2421875 - self_distillation/feedback_available_fraction:0.7578125 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9998239278793335 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.095281812828034e-05 - rollout_corr/rollout_is_max:1.8572102785110474 - rollout_corr/rollout_is_min:0.4723653197288513 - rollout_corr/rollout_is_std:0.03426937758922577 - rollout_corr/rollout_is_eff_sample_size:0.9988265710047184 - rollout_corr/rollout_is_seq_mean:1.0000782012939453 - rollout_corr/rollout_is_seq_std:0.0044146510772407055 - rollout_corr/rollout_is_seq_max:1.0189458131790161 - rollout_corr/rollout_is_seq_min:0.9898042678833008 - rollout_corr/rollout_is_seq_max_deviation:0.018945813179016113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1363611575216055) - rollout_corr/training_log_ppl:np.float64(0.12343428970780224) - rollout_corr/kl:np.float64(0.00037306189533126144) - rollout_corr/k3_kl:np.float64(0.0007827056714450054) - rollout_corr/rollout_ppl:np.float64(1.1358372080139816) - rollout_corr/rollout_log_ppl:np.float64(0.1230612265644595) - rollout_corr/log_ppl_diff:np.float64(0.0003730631433427334) - rollout_corr/log_ppl_abs_diff:np.float64(0.003695476960274391) - rollout_corr/log_ppl_diff_max:np.float64(0.019456952810287476) - rollout_corr/log_ppl_diff_min:np.float64(-0.02849803864955902) - rollout_corr/ppl_ratio:np.float64(1.0003857228439301) - rollout_corr/chi2_token:np.float64(0.00230483990162611) - rollout_corr/chi2_seq:np.float64(0.13159463531337678) - actor/pg_loss:np.float64(0.0002091224305331707) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001895007590064779) - actor/ppo_kl:np.float64(-0.00013972085250202326) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.2421875) - self_distillation/token_reweight_w_mean:np.float64(266760.6500831749) - self_distillation/token_reweight_w_std:np.float64(1994094.001658647) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013147199060768) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011595111660426483) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.3792137950658798) - perf/mfu/actor:np.float64(0.038875435312673214) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.36589813232422) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.2421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019680524710565805 - critic/advantages/max:0.875 - critic/advantages/min:-0.375 - critic/returns/mean:0.0019680524710565805 - critic/returns/max:0.875 - critic/returns/min:-0.375 - response_length/mean:76.6640625 - response_length/max:277.0 - response_length/min:33.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:76.6640625 - response_length_non_aborted/max:277.0 - response_length_non_aborted/min:33.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:577.625 - prompt_length/max:981.0 - prompt_length/min:346.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011168234050273895 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5033054240047932) - timing_s/agent_loop/generate_sequences/max:np.float64(1.9895233232527971) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.91878823244042) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.9895233232527971) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:506 - timing_s/agent_loop/slowest/response_length:277 - timing_s/gen:8.16243251413107 - timing_s/reward:0.07052589952945709 - timing_s/old_log_prob:2.2430416010320187 - timing_s/adv:0.5714792534708977 - timing_s/update_actor:15.483441106975079 - timing_s/step:26.618662336841226 - timing_s/stop_profile:0.00014204159379005432 - timing_per_token_ms/update_actor:0.09243955812591839 - timing_per_token_ms/gen:0.4158989358061281 - timing_per_token_ms/adv:0.0034118571772253858 - perf/total_num_tokens:167498 - perf/time_per_step:26.618662336841226 - perf/throughput:786.5628157813949 (TaskRunner pid=2204243) Training Progress: 27%|██▋ | 27/100 [17:40<35:32, 29.21s/it] (TaskRunner pid=2204243) step:28 - global_seqlen/min:14908 - global_seqlen/max:25722 - global_seqlen/minmax_diff:10814 - global_seqlen/balanced_min:19353 - global_seqlen/balanced_max:19359 - global_seqlen/mean:19356.0 - actor/entropy:0.11507420986890793 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37560582160949707 - training/rollout_probs_diff_mean:0.0032681042794138193 - training/rollout_probs_diff_std:0.012753847986459732 - training/rollout_actor_probs_pearson_corr:0.9971713423728943 - self_distillation/success_group_fraction:0.53125 - self_distillation/success_sample_fraction:0.52734375 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.5 - self_distillation/feedback_used_fraction:0.46875 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9999052882194519 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0003223033854737878 - rollout_corr/rollout_is_max:1.6383334398269653 - rollout_corr/rollout_is_min:0.23995225131511688 - rollout_corr/rollout_is_std:0.03399433568120003 - rollout_corr/rollout_is_eff_sample_size:0.9988454811828674 - rollout_corr/rollout_is_seq_mean:0.999811589717865 - rollout_corr/rollout_is_seq_std:0.004302878864109516 - rollout_corr/rollout_is_seq_max:1.01652193069458 - rollout_corr/rollout_is_seq_min:0.9769849181175232 - rollout_corr/rollout_is_seq_max_deviation:0.023015081882476807 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1134974383749068) - rollout_corr/training_log_ppl:np.float64(0.10278381962052663) - rollout_corr/kl:np.float64(0.0030113065460000144) - rollout_corr/k3_kl:np.float64(0.002361738311546513) - rollout_corr/rollout_ppl:np.float64(1.1100820521824062) - rollout_corr/rollout_log_ppl:np.float64(0.09977251257441822) - rollout_corr/log_ppl_diff:np.float64(0.0030113070461084135) - rollout_corr/log_ppl_abs_diff:np.float64(0.005796657198516186) - rollout_corr/log_ppl_diff_max:np.float64(0.09232157468795776) - rollout_corr/log_ppl_diff_min:np.float64(-0.015916086733341217) - rollout_corr/ppl_ratio:np.float64(1.003102439455688) - rollout_corr/chi2_token:np.float64(0.0007812364492565393) - rollout_corr/chi2_seq:np.float64(0.21633366984315217) - actor/pg_loss:np.float64(-7.964414544403553e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004279052445781417) - actor/ppo_kl:np.float64(0.0021996558646844022) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(630279.3649263827) - self_distillation/token_reweight_w_std:np.float64(2927280.8185445787) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002432307228446) - self_distillation/token_reweight_w_clip_frac:np.float64(0.004905972004053183) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.58604696393013) - perf/mfu/actor:np.float64(0.03548455617888595) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.37328720092773) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00239041680470109 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00239041680470109 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:72.71875 - response_length/max:282.0 - response_length/min:30.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:72.71875 - response_length_non_aborted/max:282.0 - response_length_non_aborted/min:30.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:532.15625 - prompt_length/max:1162.0 - prompt_length/min:218.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010867230594158173 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3899351265281439) - timing_s/agent_loop/generate_sequences/max:np.float64(1.95234321244061) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.7840183967637131) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.95234321244061) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:250 - timing_s/agent_loop/slowest/response_length:282 - timing_s/gen:7.7110571432858706 - timing_s/reward:0.06747333705425262 - timing_s/old_log_prob:2.301404556259513 - timing_s/adv:0.5954117886722088 - timing_s/update_actor:15.663819970563054 - timing_s/step:26.426436468958855 - timing_s/stop_profile:0.00012237206101417542 - timing_per_token_ms/update_actor:0.1011561012771431 - timing_per_token_ms/gen:0.41421664929554525 - timing_per_token_ms/adv:0.0038451370936157313 - perf/total_num_tokens:154848 - perf/time_per_step:26.426436468958855 - perf/throughput:732.4483580196685 (TaskRunner pid=2204243) Training Progress: 28%|██▊ | 28/100 [18:06<34:04, 28.39s/it] (TaskRunner pid=2204243) step:29 - global_seqlen/min:14850 - global_seqlen/max:29753 - global_seqlen/minmax_diff:14903 - global_seqlen/balanced_min:21515 - global_seqlen/balanced_max:28914 - global_seqlen/mean:22441.625 - actor/entropy:0.09258575737476349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.15870711207389832 - training/rollout_probs_diff_mean:0.0025046183727681637 - training/rollout_probs_diff_std:0.00994956772774458 - training/rollout_actor_probs_pearson_corr:0.9979792833328247 - self_distillation/success_group_fraction:0.1875 - self_distillation/success_sample_fraction:0.1875 - self_distillation/correct_sample_fraction:0.1796875 - self_distillation/feedback_available_fraction:0.8203125 - self_distillation/feedback_used_fraction:0.8125 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000776052474976 - rollout_corr/rollout_is_ratio_fraction_high:3.331889456603676e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.331889456603676e-05 - rollout_corr/rollout_is_max:2.7329421043395996 - rollout_corr/rollout_is_min:0.4850614368915558 - rollout_corr/rollout_is_std:0.028899485245347023 - rollout_corr/rollout_is_eff_sample_size:0.9991657546823269 - rollout_corr/rollout_is_seq_mean:1.0001424551010132 - rollout_corr/rollout_is_seq_std:0.0046637170016765594 - rollout_corr/rollout_is_seq_max:1.0382357835769653 - rollout_corr/rollout_is_seq_min:0.9889055490493774 - rollout_corr/rollout_is_seq_max_deviation:0.03823578357696533 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.128700166940689) - rollout_corr/training_log_ppl:np.float64(0.11776682642266678) - rollout_corr/kl:np.float64(0.0006739236730339826) - rollout_corr/k3_kl:np.float64(0.0006705749172066788) - rollout_corr/rollout_ppl:np.float64(1.1279347543604672) - rollout_corr/rollout_log_ppl:np.float64(0.11709290267390315) - rollout_corr/log_ppl_diff:np.float64(0.0006739237487636274) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032938644635578385) - rollout_corr/log_ppl_diff_max:np.float64(0.015930399298667908) - rollout_corr/log_ppl_diff_min:np.float64(-0.01815931499004364) - rollout_corr/ppl_ratio:np.float64(1.000684769358486) - rollout_corr/chi2_token:np.float64(0.0014096698723733425) - rollout_corr/chi2_seq:np.float64(0.14886167040094733) - actor/pg_loss:np.float64(0.00022194022312760353) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011814871322712861) - actor/ppo_kl:np.float64(0.00026197478377909533) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.1796875) - self_distillation/token_reweight_w_mean:np.float64(27145.350898608798) - self_distillation/token_reweight_w_std:np.float64(319492.4450351661) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998513124883175) - self_distillation/token_reweight_w_clip_frac:np.float64(0.00563345942646265) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.13272320106625557) - perf/mfu/actor:np.float64(0.03906606512264499) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.35293579101562) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.1796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.1796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036317596677690744 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0036317596677690744 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:117.23828125 - response_length/max:8192.0 - response_length/min:29.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:117.23828125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:584.0625 - prompt_length/max:970.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.163733243942261e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.45840337313711643) - timing_s/agent_loop/generate_sequences/max:np.float64(46.40092860348523) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2065847312187543) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(46.40092860348523) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:621 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.51207978092134 - timing_s/reward:0.07397806830704212 - timing_s/old_log_prob:2.4925561882555485 - timing_s/adv:0.5751299653202295 - timing_s/update_actor:16.691483560949564 - timing_s/step:72.43281129747629 - timing_s/stop_profile:0.00016228482127189636 - timing_per_token_ms/update_actor:0.09297167407078122 - timing_per_token_ms/gen:1.7496444800893394 - timing_per_token_ms/adv:0.003203477718972164 - perf/total_num_tokens:179533 - perf/time_per_step:72.43281129747629 - perf/throughput:309.8267842709277 (TaskRunner pid=2204243) Training Progress: 29%|██▉ | 29/100 [19:19<49:14, 41.62s/it] (TaskRunner pid=2204243) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2204243) validation generation end (TaskRunner pid=2204243) [prompt] user (TaskRunner pid=2204243) Your task is to answer the user's question using available tools. (TaskRunner pid=2204243) You have access to the following tools: (TaskRunner pid=2204243) Name: Axolotl (TaskRunner pid=2204243) Description: Collection of axolotl pictures and facts (TaskRunner pid=2204243) Documentation: (TaskRunner pid=2204243) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2204243) Parameters: {} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{url, source, description} (TaskRunner pid=2204243) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2204243) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2204243) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2204243) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Array[Object{fact, source}] (TaskRunner pid=2204243) (TaskRunner pid=2204243) Use the following format: (TaskRunner pid=2204243) Thought: you should always think about what to do (TaskRunner pid=2204243) Action: the action to take, should be one of the tool names. (TaskRunner pid=2204243) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2204243) (TaskRunner pid=2204243) Begin! (TaskRunner pid=2204243) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2204243) assistant (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: The user is asking for a random picture of an axolotl. The appropriate tool for this is getRandomAxolotlImage, which retrieves a random axolotl image with information on the image source. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2204655) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2204243) step:30 - global_seqlen/min:18102 - global_seqlen/max:24074 - global_seqlen/minmax_diff:5972 - global_seqlen/balanced_min:21190 - global_seqlen/balanced_max:21192 - global_seqlen/mean:21190.375 - actor/entropy:0.13596133887767792 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18398946523666382 - training/rollout_probs_diff_mean:0.0038822696078568697 - training/rollout_probs_diff_std:0.012573973275721073 - training/rollout_actor_probs_pearson_corr:0.9975918531417847 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.37109375 - self_distillation/correct_sample_fraction:0.3046875 - self_distillation/feedback_available_fraction:0.6953125 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.0003259181976318 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.000162381591508165 - rollout_corr/rollout_is_max:1.6273715496063232 - rollout_corr/rollout_is_min:0.42815133929252625 - rollout_corr/rollout_is_std:0.0347229428589344 - rollout_corr/rollout_is_eff_sample_size:0.9987964827004508 - rollout_corr/rollout_is_seq_mean:1.0003494024276733 - rollout_corr/rollout_is_seq_std:0.0047649419866502285 - rollout_corr/rollout_is_seq_max:1.0166699886322021 - rollout_corr/rollout_is_seq_min:0.983977735042572 - rollout_corr/rollout_is_seq_max_deviation:0.01666998863220215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1335394196212292) - rollout_corr/training_log_ppl:np.float64(0.12130965807591565) - rollout_corr/kl:np.float64(0.0005046955394831798) - rollout_corr/k3_kl:np.float64(0.0008778169284237025) - rollout_corr/rollout_ppl:np.float64(1.13291754713282) - rollout_corr/rollout_log_ppl:np.float64(0.12080496110138483) - rollout_corr/log_ppl_diff:np.float64(0.0005046969745308161) - rollout_corr/log_ppl_abs_diff:np.float64(0.0042099613638129085) - rollout_corr/log_ppl_diff_max:np.float64(0.017709270119667053) - rollout_corr/log_ppl_diff_min:np.float64(-0.02398010343313217) - rollout_corr/ppl_ratio:np.float64(1.0005210421513766) - rollout_corr/chi2_token:np.float64(0.002539924345910549) - rollout_corr/chi2_seq:np.float64(0.15570673579350114) - actor/pg_loss:np.float64(-0.0002751435386016965) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007355975576501805) - actor/ppo_kl:np.float64(0.00023927147807434324) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3046875) - self_distillation/token_reweight_w_mean:np.float64(500302.85335762776) - self_distillation/token_reweight_w_std:np.float64(3251043.397108367) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000393622322008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.016639866109471768) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.5213334951549768) - perf/mfu/actor:np.float64(0.03892875805450664) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.32586669921875) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5753676470588235) - val-aux/tooluse/reward/std@16:np.float64(0.06617441853401575) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6009705882352941) - val-aux/tooluse/reward/best@2/std:np.float64(0.047994553457011764) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5497647058823529) - val-aux/tooluse/reward/worst@2/std:np.float64(0.062201579427345374) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5749117647058825) - val-aux/tooluse/reward/maj@2/std:np.float64(0.06528218774533484) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6175735294117648) - val-aux/tooluse/reward/best@4/std:np.float64(0.032958009367317485) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.523514705882353) - val-aux/tooluse/reward/worst@4/std:np.float64(0.05551768659622602) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5811176470588236) - val-aux/tooluse/reward/maj@4/std:np.float64(0.04810031437724897) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6307352941176471) - val-aux/tooluse/reward/best@8/std:np.float64(0.023936359834158642) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.4982205882352941) - val-aux/tooluse/reward/worst@8/std:np.float64(0.043495799992748406) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.584529411764706) - val-aux/tooluse/reward/maj@8/std:np.float64(0.02767171843471393) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6404117647058823) - val-aux/tooluse/reward/best@16/std:np.float64(0.013796084512722686) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.48122058823529423) - val-aux/tooluse/reward/worst@16/std:np.float64(0.024897366249794876) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5869264705882354) - val-aux/tooluse/reward/maj@16/std:np.float64(0.011038943889091662) - val-aux/tooluse/score/mean@16:np.float64(0.5753676470588235) - val-aux/tooluse/score/std@16:np.float64(0.06617441853401575) - val-aux/tooluse/score/best@2/mean:np.float64(0.6009705882352941) - val-aux/tooluse/score/best@2/std:np.float64(0.047994553457011764) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5497647058823529) - val-aux/tooluse/score/worst@2/std:np.float64(0.062201579427345374) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5749117647058825) - val-aux/tooluse/score/maj@2/std:np.float64(0.06528218774533484) - val-aux/tooluse/score/best@4/mean:np.float64(0.6175735294117648) - val-aux/tooluse/score/best@4/std:np.float64(0.032958009367317485) - val-aux/tooluse/score/worst@4/mean:np.float64(0.523514705882353) - val-aux/tooluse/score/worst@4/std:np.float64(0.05551768659622602) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5811176470588236) - val-aux/tooluse/score/maj@4/std:np.float64(0.04810031437724897) - val-aux/tooluse/score/best@8/mean:np.float64(0.6307352941176471) - val-aux/tooluse/score/best@8/std:np.float64(0.023936359834158642) - val-aux/tooluse/score/worst@8/mean:np.float64(0.4982205882352941) - val-aux/tooluse/score/worst@8/std:np.float64(0.043495799992748406) - val-aux/tooluse/score/maj@8/mean:np.float64(0.584529411764706) - val-aux/tooluse/score/maj@8/std:np.float64(0.02767171843471393) - val-aux/tooluse/score/best@16/mean:np.float64(0.6404117647058823) - val-aux/tooluse/score/best@16/std:np.float64(0.013796084512722686) - val-aux/tooluse/score/worst@16/mean:np.float64(0.48122058823529423) - val-aux/tooluse/score/worst@16/std:np.float64(0.024897366249794876) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5869264705882354) - val-aux/tooluse/score/maj@16/std:np.float64(0.011038943889091662) - val-core/tooluse/acc/mean@16:np.float64(0.5753676470588235) - val-aux/tooluse/acc/std@16:np.float64(0.06617441853401575) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6009705882352941) - val-aux/tooluse/acc/best@2/std:np.float64(0.047994553457011764) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5497647058823529) - val-aux/tooluse/acc/worst@2/std:np.float64(0.062201579427345374) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5749117647058825) - val-aux/tooluse/acc/maj@2/std:np.float64(0.06528218774533484) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6175735294117648) - val-aux/tooluse/acc/best@4/std:np.float64(0.032958009367317485) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.523514705882353) - val-aux/tooluse/acc/worst@4/std:np.float64(0.05551768659622602) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5811176470588236) - val-aux/tooluse/acc/maj@4/std:np.float64(0.04810031437724897) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6307352941176471) - val-aux/tooluse/acc/best@8/std:np.float64(0.023936359834158642) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.4982205882352941) - val-aux/tooluse/acc/worst@8/std:np.float64(0.043495799992748406) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.584529411764706) - val-aux/tooluse/acc/maj@8/std:np.float64(0.02767171843471393) - val-core/tooluse/acc/best@16/mean:np.float64(0.6404117647058823) - val-core/tooluse/acc/best@16/std:np.float64(0.013796084512722686) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.48122058823529423) - val-aux/tooluse/acc/worst@16/std:np.float64(0.024897366249794876) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5869264705882354) - val-core/tooluse/acc/maj@16/std:np.float64(0.011038943889091662) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.3046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007449255790561438 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007449255790561438 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:72.16796875 - response_length/max:210.0 - response_length/min:33.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:72.16796875 - response_length_non_aborted/max:210.0 - response_length_non_aborted/min:33.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:590.03125 - prompt_length/max:976.0 - prompt_length/min:244.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014488585293293 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4991429038345814) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7083760406821966) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.906144071239396) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7083760406821966) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:467 - timing_s/agent_loop/slowest/response_length:210 - timing_s/gen:7.429821198806167 - timing_s/reward:0.07062751613557339 - timing_s/old_log_prob:2.284553851932287 - timing_s/adv:0.5756561234593391 - timing_s/update_actor:15.53678573668003 - timing_s/step:25.985548539087176 - timing_s/testing:20.787529535591602 - timing_s/save_checkpoint:6.567745845764875 - timing_s/stop_profile:0.00015440024435520172 - timing_per_token_ms/update_actor:0.09165001643835957 - timing_per_token_ms/gen:0.40215540994891297 - timing_per_token_ms/adv:0.003395740539391936 - perf/total_num_tokens:169523 - perf/time_per_step:25.985548539087176 - perf/throughput:815.4676807428433 (TaskRunner pid=2204243) Training Progress: 30%|███ | 30/100 [20:12<52:41, 45.16s/it] (TaskRunner pid=2204243) step:31 - global_seqlen/min:16048 - global_seqlen/max:27089 - global_seqlen/minmax_diff:11041 - global_seqlen/balanced_min:21114 - global_seqlen/balanced_max:21332 - global_seqlen/mean:21144.75 - actor/entropy:0.11729671061038971 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2448694407939911 - training/rollout_probs_diff_mean:0.0035037219058722258 - training/rollout_probs_diff_std:0.012594467960298061 - training/rollout_actor_probs_pearson_corr:0.9974067211151123 - self_distillation/success_group_fraction:0.46875 - self_distillation/success_sample_fraction:0.46484375 - self_distillation/correct_sample_fraction:0.3828125 - self_distillation/feedback_available_fraction:0.6171875 - self_distillation/feedback_used_fraction:0.53125 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9998746514320374 - rollout_corr/rollout_is_ratio_fraction_high:4.7569214075338095e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2923789024353027 - rollout_corr/rollout_is_min:0.5706907510757446 - rollout_corr/rollout_is_std:0.03605685010552406 - rollout_corr/rollout_is_eff_sample_size:0.9987011159449863 - rollout_corr/rollout_is_seq_mean:0.9998646378517151 - rollout_corr/rollout_is_seq_std:0.004800043068826199 - rollout_corr/rollout_is_seq_max:1.0151492357254028 - rollout_corr/rollout_is_seq_min:0.9809422492980957 - rollout_corr/rollout_is_seq_max_deviation:0.019057750701904297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1260879202745855) - rollout_corr/training_log_ppl:np.float64(0.11524713151084143) - rollout_corr/kl:np.float64(0.001034425109960324) - rollout_corr/k3_kl:np.float64(0.0011317956499659054) - rollout_corr/rollout_ppl:np.float64(1.1248079356737435) - rollout_corr/rollout_log_ppl:np.float64(0.11421270408209239) - rollout_corr/log_ppl_diff:np.float64(0.0010344274287490407) - rollout_corr/log_ppl_abs_diff:np.float64(0.004364273036117083) - rollout_corr/log_ppl_diff_max:np.float64(0.04338109493255615) - rollout_corr/log_ppl_diff_min:np.float64(-0.031131070107221603) - rollout_corr/ppl_ratio:np.float64(1.0010556941851974) - rollout_corr/chi2_token:np.float64(0.002515688305720687) - rollout_corr/chi2_seq:np.float64(0.23659252282232046) - actor/pg_loss:np.float64(0.0018887566402554512) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003935188469768036) - actor/ppo_kl:np.float64(0.00023846273535355067) - actor/pg_clipfrac_lower:np.float64(5.461568616738077e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3828125) - self_distillation/token_reweight_w_mean:np.float64(89861.75544591364) - self_distillation/token_reweight_w_std:np.float64(778093.9731729474) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001081722555682) - self_distillation/token_reweight_w_clip_frac:np.float64(0.023077207668393385) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.5443007163703442) - perf/mfu/actor:np.float64(0.03887702844371984) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.5116958618164) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01867091655731201 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.01867091655731201 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:82.1171875 - response_length/max:1046.0 - response_length/min:29.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:82.1171875 - response_length_non_aborted/max:1046.0 - response_length_non_aborted/min:29.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:578.65625 - prompt_length/max:939.0 - prompt_length/min:260.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.680725634098053e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3577273339033127) - timing_s/agent_loop/generate_sequences/max:np.float64(5.4892747811973095) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8575148119198275) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.4892747811973095) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:655 - timing_s/agent_loop/slowest/response_length:1046 - timing_s/gen:11.400745110586286 - timing_s/reward:0.06854117289185524 - timing_s/old_log_prob:2.4845431726425886 - timing_s/adv:0.6156300362199545 - timing_s/update_actor:15.535736754536629 - timing_s/step:30.220368292182684 - timing_s/stop_profile:0.00011535920202732086 - timing_per_token_ms/update_actor:0.09184157269852226 - timing_per_token_ms/gen:0.5423244748637753 - timing_per_token_ms/adv:0.0036393787832674452 - perf/total_num_tokens:169158 - perf/time_per_step:30.220368292182684 - perf/throughput:699.6853842270897 (TaskRunner pid=2204243) Training Progress: 31%|███ | 31/100 [20:42<46:47, 40.69s/it] (TaskRunner pid=2204243) step:32 - global_seqlen/min:17629 - global_seqlen/max:25631 - global_seqlen/minmax_diff:8002 - global_seqlen/balanced_min:21287 - global_seqlen/balanced_max:21288 - global_seqlen/mean:21287.625 - actor/entropy:0.10964254289865494 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22960394620895386 - training/rollout_probs_diff_mean:0.0033697413746267557 - training/rollout_probs_diff_std:0.01223733276128769 - training/rollout_actor_probs_pearson_corr:0.9972966313362122 - self_distillation/success_group_fraction:0.53125 - self_distillation/success_sample_fraction:0.5234375 - self_distillation/correct_sample_fraction:0.44140625 - self_distillation/feedback_available_fraction:0.55859375 - self_distillation/feedback_used_fraction:0.46875 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:1.000203013420105 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8993748426437378 - rollout_corr/rollout_is_min:0.6195820569992065 - rollout_corr/rollout_is_std:0.03302673250436783 - rollout_corr/rollout_is_eff_sample_size:0.9989108992714004 - rollout_corr/rollout_is_seq_mean:1.0000993013381958 - rollout_corr/rollout_is_seq_std:0.004254916217178106 - rollout_corr/rollout_is_seq_max:1.0166430473327637 - rollout_corr/rollout_is_seq_min:0.9899933934211731 - rollout_corr/rollout_is_seq_max_deviation:0.016643047332763672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1091889929957688) - rollout_corr/training_log_ppl:np.float64(0.10019223563722335) - rollout_corr/kl:np.float64(0.0004733865565498263) - rollout_corr/k3_kl:np.float64(0.0006818838010698158) - rollout_corr/rollout_ppl:np.float64(1.1086445110850036) - rollout_corr/rollout_log_ppl:np.float64(0.09971884813785437) - rollout_corr/log_ppl_diff:np.float64(0.0004733874993689824) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032905937514442485) - rollout_corr/log_ppl_diff_max:np.float64(0.014335595071315765) - rollout_corr/log_ppl_diff_min:np.float64(-0.013067804276943207) - rollout_corr/ppl_ratio:np.float64(1.0004825873766094) - rollout_corr/chi2_token:np.float64(0.0017888417933136225) - rollout_corr/chi2_seq:np.float64(0.11832868098281324) - actor/pg_loss:np.float64(0.00019584479741752148) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005374089487304445) - actor/ppo_kl:np.float64(2.4458121842485525e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44140625) - self_distillation/token_reweight_w_mean:np.float64(534364.9745502141) - self_distillation/token_reweight_w_std:np.float64(4212561.683608852) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005811227019876) - self_distillation/token_reweight_w_clip_frac:np.float64(0.018688832176849246) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.44686488807201385) - perf/mfu/actor:np.float64(0.039263918705378986) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.64282608032227) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012824458070099354 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.012824458070099354 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:72.98828125 - response_length/max:207.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:72.98828125 - response_length_non_aborted/max:207.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:592.25 - prompt_length/max:885.0 - prompt_length/min:312.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011301785707473755 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5207717828452587) - timing_s/agent_loop/generate_sequences/max:np.float64(1.5998018439859152) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8878061983778025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.5998018439859152) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:568 - timing_s/agent_loop/slowest/response_length:207 - timing_s/gen:7.638254335150123 - timing_s/reward:0.07113899104297161 - timing_s/old_log_prob:2.317713053897023 - timing_s/adv:0.5902881920337677 - timing_s/update_actor:15.55817256681621 - timing_s/step:26.26331882365048 - timing_s/stop_profile:0.00011850334703922272 - timing_per_token_ms/update_actor:0.09135690669353798 - timing_per_token_ms/gen:0.40879070565427467 - timing_per_token_ms/adv:0.0034661463645766478 - perf/total_num_tokens:170301 - perf/time_per_step:26.26331882365048 - perf/throughput:810.5458850398679 (TaskRunner pid=2204243) Training Progress: 32%|███▏ | 32/100 [21:09<41:13, 36.38s/it] (TaskRunner pid=2204243) step:33 - global_seqlen/min:13657 - global_seqlen/max:27271 - global_seqlen/minmax_diff:13614 - global_seqlen/balanced_min:20785 - global_seqlen/balanced_max:25737 - global_seqlen/mean:21412.125 - actor/entropy:0.10153282433748245 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4760798215866089 - training/rollout_probs_diff_mean:0.002752305706962943 - training/rollout_probs_diff_std:0.011361275799572468 - training/rollout_actor_probs_pearson_corr:0.9976395964622498 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.33203125 - self_distillation/correct_sample_fraction:0.234375 - self_distillation/feedback_available_fraction:0.765625 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:0.9998109936714172 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.869819192914292e-05 - rollout_corr/rollout_is_max:1.6279867887496948 - rollout_corr/rollout_is_min:0.4765133559703827 - rollout_corr/rollout_is_std:0.03002135641872883 - rollout_corr/rollout_is_eff_sample_size:0.9990992322571801 - rollout_corr/rollout_is_seq_mean:0.9997429847717285 - rollout_corr/rollout_is_seq_std:0.004465841688215733 - rollout_corr/rollout_is_seq_max:1.0201562643051147 - rollout_corr/rollout_is_seq_min:0.9796060919761658 - rollout_corr/rollout_is_seq_max_deviation:0.02039390802383423 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1246954025700688) - rollout_corr/training_log_ppl:np.float64(0.11365259427839192) - rollout_corr/kl:np.float64(0.0013767038137110221) - rollout_corr/k3_kl:np.float64(0.000837868124147434) - rollout_corr/rollout_ppl:np.float64(1.1232251254841685) - rollout_corr/rollout_log_ppl:np.float64(0.11227588922338327) - rollout_corr/log_ppl_diff:np.float64(0.0013767050550086424) - rollout_corr/log_ppl_abs_diff:np.float64(0.003997023246483877) - rollout_corr/log_ppl_diff_max:np.float64(0.023544907569885254) - rollout_corr/log_ppl_diff_min:np.float64(-0.01086503267288208) - rollout_corr/ppl_ratio:np.float64(1.0013924250379205) - rollout_corr/chi2_token:np.float64(0.0005366860423237085) - rollout_corr/chi2_seq:np.float64(0.05336118210107088) - actor/pg_loss:np.float64(0.0005938555113971233) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005464539940476243) - actor/ppo_kl:np.float64(0.000532932751351467) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.234375) - self_distillation/token_reweight_w_mean:np.float64(622652.7698243186) - self_distillation/token_reweight_w_std:np.float64(4482182.83231299) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000739460112527) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01850280285725603) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.3471021130681038) - perf/mfu/actor:np.float64(0.03783976664517982) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.53648376464844) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.028704384341835976 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.028704384341835976 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:100.94140625 - response_length/max:5989.0 - response_length/min:30.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:100.94140625 - response_length_non_aborted/max:5989.0 - response_length_non_aborted/min:30.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:568.1875 - prompt_length/max:1108.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015487149357795715 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3436222467571497) - timing_s/agent_loop/generate_sequences/max:np.float64(28.1847985945642) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9430454507382819) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.1847985945642) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:451 - timing_s/agent_loop/slowest/response_length:5989 - timing_s/gen:33.98538126796484 - timing_s/reward:0.08009502664208412 - timing_s/old_log_prob:2.463214948773384 - timing_s/adv:0.605207247659564 - timing_s/update_actor:16.315522491931915 - timing_s/step:53.569678300991654 - timing_s/stop_profile:0.00013749673962593079 - timing_per_token_ms/update_actor:0.0952469832625902 - timing_per_token_ms/gen:1.315172836498775 - timing_per_token_ms/adv:0.003533087255816296 - perf/total_num_tokens:171297 - perf/time_per_step:53.569678300991654 - perf/throughput:399.70605908237513 (TaskRunner pid=2204243) Training Progress: 33%|███▎ | 33/100 [22:02<46:24, 41.56s/it] (TaskRunner pid=2204243) step:34 - global_seqlen/min:17969 - global_seqlen/max:25848 - global_seqlen/minmax_diff:7879 - global_seqlen/balanced_min:21161 - global_seqlen/balanced_max:21165 - global_seqlen/mean:21162.75 - actor/entropy:0.12450677156448364 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6356968879699707 - training/rollout_probs_diff_mean:0.0035766465589404106 - training/rollout_probs_diff_std:0.013188994489610195 - training/rollout_actor_probs_pearson_corr:0.9972649216651917 - self_distillation/success_group_fraction:0.28125 - self_distillation/success_sample_fraction:0.28125 - self_distillation/correct_sample_fraction:0.27734375 - self_distillation/feedback_available_fraction:0.72265625 - self_distillation/feedback_used_fraction:0.71875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9998759627342224 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00014009526057634503 - rollout_corr/rollout_is_max:1.9508037567138672 - rollout_corr/rollout_is_min:0.297272264957428 - rollout_corr/rollout_is_std:0.0351545549929142 - rollout_corr/rollout_is_eff_sample_size:0.998765444886919 - rollout_corr/rollout_is_seq_mean:0.9997957944869995 - rollout_corr/rollout_is_seq_std:0.004606260452419519 - rollout_corr/rollout_is_seq_max:1.0159170627593994 - rollout_corr/rollout_is_seq_min:0.9775380492210388 - rollout_corr/rollout_is_seq_max_deviation:0.02246195077896118 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1225557792931795) - rollout_corr/training_log_ppl:np.float64(0.11219164655813074) - rollout_corr/kl:np.float64(0.0014850281173206525) - rollout_corr/k3_kl:np.float64(0.0008669247226353871) - rollout_corr/rollout_ppl:np.float64(1.1209153877571225) - rollout_corr/rollout_log_ppl:np.float64(0.11070661674057192) - rollout_corr/log_ppl_diff:np.float64(0.001485029817558825) - rollout_corr/log_ppl_abs_diff:np.float64(0.004000365859610611) - rollout_corr/log_ppl_diff_max:np.float64(0.025356903672218323) - rollout_corr/log_ppl_diff_min:np.float64(-0.013209141790866852) - rollout_corr/ppl_ratio:np.float64(1.0015008652117103) - rollout_corr/chi2_token:np.float64(0.0004605215508490801) - rollout_corr/chi2_seq:np.float64(0.10048562847077847) - actor/pg_loss:np.float64(0.00023554940707981586) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019221231195842847) - actor/ppo_kl:np.float64(0.0006377677601712151) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.27734375) - self_distillation/token_reweight_w_mean:np.float64(44079.96737528453) - self_distillation/token_reweight_w_std:np.float64(285630.5778069004) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996303219813854) - self_distillation/token_reweight_w_clip_frac:np.float64(0.007314099930226803) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.14895102009177208) - perf/mfu/actor:np.float64(0.0392362460500893) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.54319381713867) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.27734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.27734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009339684038423002 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009339684038423002 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:83.6484375 - response_length/max:374.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:83.6484375 - response_length_non_aborted/max:374.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:577.6875 - prompt_length/max:966.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013347528874874115 - timing_s/agent_loop/generate_sequences/min:np.float64(0.37418762408196926) - timing_s/agent_loop/generate_sequences/max:np.float64(2.553917370736599) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9199840680885245) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.553917370736599) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:666 - timing_s/agent_loop/slowest/response_length:374 - timing_s/gen:8.278261573985219 - timing_s/reward:0.06969422847032547 - timing_s/old_log_prob:2.3550470918416977 - timing_s/adv:0.5683567803353071 - timing_s/update_actor:15.443117618560791 - timing_s/step:26.803275361657143 - timing_s/stop_profile:0.00012686476111412048 - timing_per_token_ms/update_actor:0.09121639211917633 - timing_per_token_ms/gen:0.3865817490419921 - timing_per_token_ms/adv:0.0033570588672036192 - perf/total_num_tokens:169302 - perf/time_per_step:26.803275361657143 - perf/throughput:789.5583548820277 (TaskRunner pid=2204243) Training Progress: 34%|███▍ | 34/100 [22:29<40:51, 37.15s/it] (TaskRunner pid=2204243) step:35 - global_seqlen/min:17803 - global_seqlen/max:25799 - global_seqlen/minmax_diff:7996 - global_seqlen/balanced_min:22280 - global_seqlen/balanced_max:22286 - global_seqlen/mean:22284.0 - actor/entropy:0.12716615200042725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3015265166759491 - training/rollout_probs_diff_mean:0.003504763590171933 - training/rollout_probs_diff_std:0.011952897533774376 - training/rollout_actor_probs_pearson_corr:0.9977961778640747 - self_distillation/success_group_fraction:0.40625 - self_distillation/success_sample_fraction:0.40234375 - self_distillation/correct_sample_fraction:0.33203125 - self_distillation/feedback_available_fraction:0.66796875 - self_distillation/feedback_used_fraction:0.59375 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.0002509355545044 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9493341445922852 - rollout_corr/rollout_is_min:0.6130911707878113 - rollout_corr/rollout_is_std:0.034454114735126495 - rollout_corr/rollout_is_eff_sample_size:0.9988149160314487 - rollout_corr/rollout_is_seq_mean:1.0002171993255615 - rollout_corr/rollout_is_seq_std:0.0038589341565966606 - rollout_corr/rollout_is_seq_max:1.0198655128479004 - rollout_corr/rollout_is_seq_min:0.9882329106330872 - rollout_corr/rollout_is_seq_max_deviation:0.01986551284790039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1345930509269238) - rollout_corr/training_log_ppl:np.float64(0.12193724305325304) - rollout_corr/kl:np.float64(0.0018675275469171027) - rollout_corr/k3_kl:np.float64(0.0015652623508017882) - rollout_corr/rollout_ppl:np.float64(1.132419385947287) - rollout_corr/rollout_log_ppl:np.float64(0.12006971505252295) - rollout_corr/log_ppl_diff:np.float64(0.0018675280007300898) - rollout_corr/log_ppl_abs_diff:np.float64(0.004344263506936841) - rollout_corr/log_ppl_diff_max:np.float64(0.06407910585403442) - rollout_corr/log_ppl_diff_min:np.float64(-0.02168000489473343) - rollout_corr/ppl_ratio:np.float64(1.0018969611264765) - rollout_corr/chi2_token:np.float64(0.0035244969185441732) - rollout_corr/chi2_seq:np.float64(0.2288229907862842) - actor/pg_loss:np.float64(0.0008445837302133441) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001427909817721229) - actor/ppo_kl:np.float64(0.0015198218841874223) - actor/pg_clipfrac_lower:np.float64(7.659313996555284e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.33203125) - self_distillation/token_reweight_w_mean:np.float64(480181.2635695706) - self_distillation/token_reweight_w_std:np.float64(3277926.578682977) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001161174615845) - self_distillation/token_reweight_w_clip_frac:np.float64(0.019464378798147663) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.391861982177943) - perf/mfu/actor:np.float64(0.04141082554176055) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.53392028808594) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.33203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009696440771222115 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009696440771222115 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:91.75 - response_length/max:663.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:91.75 - response_length_non_aborted/max:663.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:604.625 - prompt_length/max:1261.0 - prompt_length/min:253.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011283904314041138 - timing_s/agent_loop/generate_sequences/min:np.float64(0.49773729778826237) - timing_s/agent_loop/generate_sequences/max:np.float64(3.9378009010106325) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0267899722093716) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.9378009010106325) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:776 - timing_s/agent_loop/slowest/response_length:663 - timing_s/gen:9.797752356156707 - timing_s/reward:0.07320161908864975 - timing_s/old_log_prob:2.285966172814369 - timing_s/adv:0.6214185878634453 - timing_s/update_actor:15.468213368207216 - timing_s/step:28.334883397445083 - timing_s/stop_profile:0.00011717528104782104 - timing_per_token_ms/update_actor:0.08676748658346356 - timing_per_token_ms/gen:0.41713863914154914 - timing_per_token_ms/adv:0.0034857890631363607 - perf/total_num_tokens:178272 - perf/time_per_step:28.334883397445083 - perf/throughput:786.4510923665674 (TaskRunner pid=2204243) Training Progress: 35%|███▌ | 35/100 [22:58<37:23, 34.52s/it] (TaskRunner pid=2204243) step:36 - global_seqlen/min:17081 - global_seqlen/max:27027 - global_seqlen/minmax_diff:9946 - global_seqlen/balanced_min:22326 - global_seqlen/balanced_max:22331 - global_seqlen/mean:22328.5 - actor/entropy:0.11969231814146042 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4525737166404724 - training/rollout_probs_diff_mean:0.003613929497078061 - training/rollout_probs_diff_std:0.013309267349541187 - training/rollout_actor_probs_pearson_corr:0.9971823692321777 - self_distillation/success_group_fraction:0.46875 - self_distillation/success_sample_fraction:0.46484375 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.56640625 - self_distillation/feedback_used_fraction:0.53125 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.000740647315979 - rollout_corr/rollout_is_ratio_fraction_high:0.00017777777975425124 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.9340710639953613 - rollout_corr/rollout_is_min:0.5417481064796448 - rollout_corr/rollout_is_std:0.040347516536712646 - rollout_corr/rollout_is_eff_sample_size:0.9983769813900928 - rollout_corr/rollout_is_seq_mean:1.0007212162017822 - rollout_corr/rollout_is_seq_std:0.005331083200871944 - rollout_corr/rollout_is_seq_max:1.0240957736968994 - rollout_corr/rollout_is_seq_min:0.9861719608306885 - rollout_corr/rollout_is_seq_max_deviation:0.024095773696899414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1262263394892216) - rollout_corr/training_log_ppl:np.float64(0.11535167337751773) - rollout_corr/kl:np.float64(0.00044884114891097227) - rollout_corr/k3_kl:np.float64(0.0009380327576042902) - rollout_corr/rollout_ppl:np.float64(1.1256957650184631) - rollout_corr/rollout_log_ppl:np.float64(0.11490283015518798) - rollout_corr/log_ppl_diff:np.float64(0.0004488432223297423) - rollout_corr/log_ppl_abs_diff:np.float64(0.003919671062249108) - rollout_corr/log_ppl_diff_max:np.float64(0.025960735976696014) - rollout_corr/log_ppl_diff_min:np.float64(-0.01797609031200409) - rollout_corr/ppl_ratio:np.float64(1.0004648831672966) - rollout_corr/chi2_token:np.float64(0.0028864331543445587) - rollout_corr/chi2_seq:np.float64(0.3150283091235906) - actor/pg_loss:np.float64(0.0003566292580217123) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012181849524495192) - actor/ppo_kl:np.float64(0.00039274360072738546) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(60777.41255616862) - self_distillation/token_reweight_w_std:np.float64(677296.5336597499) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004060354549438) - self_distillation/token_reweight_w_clip_frac:np.float64(0.007099016453139484) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.17126407474279404) - perf/mfu/actor:np.float64(0.04127101962308499) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.58261489868164) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019277777755632997 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0019277777755632997 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:87.890625 - response_length/max:306.0 - response_length/min:30.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:87.890625 - response_length_non_aborted/max:306.0 - response_length_non_aborted/min:30.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:609.875 - prompt_length/max:1211.0 - prompt_length/min:293.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.68780368566513e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.37383944541215897) - timing_s/agent_loop/generate_sequences/max:np.float64(2.465050797909498) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9839773598869215) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.465050797909498) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1093 - timing_s/agent_loop/slowest/response_length:296 - timing_s/gen:8.322742203250527 - timing_s/reward:0.07294704765081406 - timing_s/old_log_prob:2.280948854982853 - timing_s/adv:0.6602698098868132 - timing_s/update_actor:15.514248428866267 - timing_s/step:26.9355057682842 - timing_s/stop_profile:0.00013580918312072754 - timing_per_token_ms/update_actor:0.08685227640048743 - timing_per_token_ms/gen:0.36989965347780124 - timing_per_token_ms/adv:0.0036963399348747855 - perf/total_num_tokens:178628 - perf/time_per_step:26.9355057682842 - perf/throughput:828.9616015412333 (TaskRunner pid=2204243) Training Progress: 36%|███▌ | 36/100 [23:24<34:24, 32.26s/it] (TaskRunner pid=2204243) step:37 - global_seqlen/min:18245 - global_seqlen/max:27732 - global_seqlen/minmax_diff:9487 - global_seqlen/balanced_min:22455 - global_seqlen/balanced_max:23382 - global_seqlen/mean:22688.375 - actor/entropy:0.12105684727430344 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135535955429077 - training/rollout_probs_diff_mean:0.003135975915938616 - training/rollout_probs_diff_std:0.011484574526548386 - training/rollout_actor_probs_pearson_corr:0.9979568123817444 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.375 - self_distillation/correct_sample_fraction:0.3046875 - self_distillation/feedback_available_fraction:0.6953125 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999040961265564 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7523047924041748 - rollout_corr/rollout_is_min:0.5378838181495667 - rollout_corr/rollout_is_std:0.033280227333307266 - rollout_corr/rollout_is_eff_sample_size:0.9988934139614173 - rollout_corr/rollout_is_seq_mean:0.9999574422836304 - rollout_corr/rollout_is_seq_std:0.004152799025177956 - rollout_corr/rollout_is_seq_max:1.0152549743652344 - rollout_corr/rollout_is_seq_min:0.9801881313323975 - rollout_corr/rollout_is_seq_max_deviation:0.01981186866760254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1384392962791026) - rollout_corr/training_log_ppl:np.float64(0.12397896202310221) - rollout_corr/kl:np.float64(0.00406089033602175) - rollout_corr/k3_kl:np.float64(0.003981667897365267) - rollout_corr/rollout_ppl:np.float64(1.1333657912909985) - rollout_corr/rollout_log_ppl:np.float64(0.11991806980586261) - rollout_corr/log_ppl_diff:np.float64(0.0040608922608953435) - rollout_corr/log_ppl_abs_diff:np.float64(0.006611916895053582) - rollout_corr/log_ppl_diff_max:np.float64(0.30686527490615845) - rollout_corr/log_ppl_diff_min:np.float64(-0.013046830892562866) - rollout_corr/ppl_ratio:np.float64(1.0046431948430836) - rollout_corr/chi2_token:np.float64(0.0017769683618098497) - rollout_corr/chi2_seq:np.float64(0.09899262548424304) - actor/pg_loss:np.float64(0.0014702435582876205) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009017036909426679) - actor/ppo_kl:np.float64(0.003443702057751352) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3046875) - self_distillation/token_reweight_w_mean:np.float64(34929.7093434399) - self_distillation/token_reweight_w_std:np.float64(326945.73618760816) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005393342580646) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020876311289612204) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.5830485932528973) - perf/mfu/actor:np.float64(0.04147857788853197) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.63002395629883) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.3046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004516994580626488 - critic/advantages/max:0.75 - critic/advantages/min:-0.75 - critic/returns/mean:-0.004516994580626488 - critic/returns/max:0.75 - critic/returns/min:-0.75 - response_length/mean:98.26171875 - response_length/max:1788.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:98.26171875 - response_length_non_aborted/max:1788.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:610.75 - prompt_length/max:1101.0 - prompt_length/min:300.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010062940418720245 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5667593404650688) - timing_s/agent_loop/generate_sequences/max:np.float64(9.098234385251999) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0783164864114951) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.098234385251999) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:676 - timing_s/agent_loop/slowest/response_length:1788 - timing_s/gen:14.9797503054142 - timing_s/reward:0.09207996912300587 - timing_s/old_log_prob:2.314271619543433 - timing_s/adv:0.6443353034555912 - timing_s/update_actor:15.659851836040616 - timing_s/step:33.776621002703905 - timing_s/stop_profile:0.000115908682346344 - timing_per_token_ms/update_actor:0.08627684792344437 - timing_per_token_ms/gen:0.5954979250810654 - timing_per_token_ms/adv:0.0035499198568407346 - perf/total_num_tokens:181507 - perf/time_per_step:33.776621002703905 - perf/throughput:671.7183165889725 (TaskRunner pid=2204243) Training Progress: 37%|███▋ | 37/100 [23:58<34:21, 32.73s/it] (TaskRunner pid=2204243) step:38 - global_seqlen/min:20093 - global_seqlen/max:25772 - global_seqlen/minmax_diff:5679 - global_seqlen/balanced_min:22341 - global_seqlen/balanced_max:22455 - global_seqlen/mean:22369.75 - actor/entropy:0.10248328000307083 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174861967563629 - training/rollout_probs_diff_mean:0.0032521637622267008 - training/rollout_probs_diff_std:0.012380418367683887 - training/rollout_actor_probs_pearson_corr:0.9971799254417419 - self_distillation/success_group_fraction:0.21875 - self_distillation/success_sample_fraction:0.21484375 - self_distillation/correct_sample_fraction:0.19140625 - self_distillation/feedback_available_fraction:0.80859375 - self_distillation/feedback_used_fraction:0.78125 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9999314546585083 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.9065551391104236e-05 - rollout_corr/rollout_is_max:1.805476188659668 - rollout_corr/rollout_is_min:0.3536285161972046 - rollout_corr/rollout_is_std:0.03392940014600754 - rollout_corr/rollout_is_eff_sample_size:0.9988500007025259 - rollout_corr/rollout_is_seq_mean:0.9998934268951416 - rollout_corr/rollout_is_seq_std:0.003775340737774968 - rollout_corr/rollout_is_seq_max:1.015328288078308 - rollout_corr/rollout_is_seq_min:0.9870959520339966 - rollout_corr/rollout_is_seq_max_deviation:0.015328288078308105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1171090742573142) - rollout_corr/training_log_ppl:np.float64(0.10824341754323541) - rollout_corr/kl:np.float64(0.0011568538624597835) - rollout_corr/k3_kl:np.float64(0.0010453060502113587) - rollout_corr/rollout_ppl:np.float64(1.115784048102796) - rollout_corr/rollout_log_ppl:np.float64(0.1070865631736524) - rollout_corr/log_ppl_diff:np.float64(0.0011568543668545317) - rollout_corr/log_ppl_abs_diff:np.float64(0.003914799021004001) - rollout_corr/log_ppl_diff_max:np.float64(0.03673376142978668) - rollout_corr/log_ppl_diff_min:np.float64(-0.023971155285835266) - rollout_corr/ppl_ratio:np.float64(1.0011736175511032) - rollout_corr/chi2_token:np.float64(0.0017816685140132904) - rollout_corr/chi2_seq:np.float64(0.19510890496894717) - actor/pg_loss:np.float64(8.41943547129631e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00028639717857004143) - actor/ppo_kl:np.float64(0.0004469240813236297) - actor/pg_clipfrac_lower:np.float64(9.990409125748556e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.19140625) - self_distillation/token_reweight_w_mean:np.float64(117857.35394334048) - self_distillation/token_reweight_w_std:np.float64(1110285.8078746796) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003664195537567) - self_distillation/token_reweight_w_clip_frac:np.float64(0.004489213606575504) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.14375882176682353) - perf/mfu/actor:np.float64(0.04145198214016006) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.5973129272461) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.19140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.19140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002939682686701417 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:-0.002939682686701417 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:99.9921875 - response_length/max:933.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:99.9921875 - response_length_non_aborted/max:933.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:599.0625 - prompt_length/max:960.0 - prompt_length/min:338.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011452846229076385 - timing_s/agent_loop/generate_sequences/min:np.float64(0.536233339458704) - timing_s/agent_loop/generate_sequences/max:np.float64(5.112826492637396) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.087942613477935) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.112826492637396) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:602 - timing_s/agent_loop/slowest/response_length:933 - timing_s/gen:10.963427999988198 - timing_s/reward:0.07507439143955708 - timing_s/old_log_prob:2.3259494230151176 - timing_s/adv:0.5862207878381014 - timing_s/update_actor:15.284737164154649 - timing_s/step:29.3207789324224 - timing_s/stop_profile:0.00011716410517692566 - timing_per_token_ms/update_actor:0.08540963334500078 - timing_per_token_ms/gen:0.4282923665906789 - timing_per_token_ms/adv:0.0032757450789464644 - perf/total_num_tokens:178958 - perf/time_per_step:29.3207789324224 - perf/throughput:762.9316414668617 (TaskRunner pid=2204243) Training Progress: 38%|███▊ | 38/100 [24:28<32:46, 31.72s/it] (TaskRunner pid=2204243) step:39 - global_seqlen/min:18372 - global_seqlen/max:27526 - global_seqlen/minmax_diff:9154 - global_seqlen/balanced_min:22617 - global_seqlen/balanced_max:22624 - global_seqlen/mean:22619.875 - actor/entropy:0.11347021162509918 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23519669473171234 - training/rollout_probs_diff_mean:0.003448646515607834 - training/rollout_probs_diff_std:0.012663613073527813 - training/rollout_actor_probs_pearson_corr:0.9973973035812378 - self_distillation/success_group_fraction:0.5 - self_distillation/success_sample_fraction:0.5 - self_distillation/correct_sample_fraction:0.44921875 - self_distillation/feedback_available_fraction:0.55078125 - self_distillation/feedback_used_fraction:0.5 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9997061491012573 - rollout_corr/rollout_is_ratio_fraction_high:4.5047072489978746e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.057621717453003 - rollout_corr/rollout_is_min:0.522678017616272 - rollout_corr/rollout_is_std:0.036696698516607285 - rollout_corr/rollout_is_eff_sample_size:0.9986543905048518 - rollout_corr/rollout_is_seq_mean:0.9997634887695312 - rollout_corr/rollout_is_seq_std:0.004171626176685095 - rollout_corr/rollout_is_seq_max:1.012664794921875 - rollout_corr/rollout_is_seq_min:0.983154296875 - rollout_corr/rollout_is_seq_max_deviation:0.016845703125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.110125229228288) - rollout_corr/training_log_ppl:np.float64(0.10175585826596034) - rollout_corr/kl:np.float64(0.0008029047316089333) - rollout_corr/k3_kl:np.float64(0.0012541592355712443) - rollout_corr/rollout_ppl:np.float64(1.1092044822871685) - rollout_corr/rollout_log_ppl:np.float64(0.10095295262544823) - rollout_corr/log_ppl_diff:np.float64(0.0008029056550640234) - rollout_corr/log_ppl_abs_diff:np.float64(0.004041330995505632) - rollout_corr/log_ppl_diff_max:np.float64(0.0866929292678833) - rollout_corr/log_ppl_diff_min:np.float64(-0.020133063197135925) - rollout_corr/ppl_ratio:np.float64(1.000831744633615) - rollout_corr/chi2_token:np.float64(0.0030880908016115427) - rollout_corr/chi2_seq:np.float64(0.28389478800818324) - actor/pg_loss:np.float64(-0.0002478277310729027) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009842569415923208) - actor/ppo_kl:np.float64(-3.4623453878168675e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44921875) - self_distillation/token_reweight_w_mean:np.float64(231575.15462911944) - self_distillation/token_reweight_w_std:np.float64(1744608.5352025703) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999738135375082) - self_distillation/token_reweight_w_clip_frac:np.float64(0.013434967651846819) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.4020931012928486) - perf/mfu/actor:np.float64(0.04147882023505836) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.60799407958984) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002443803707137704 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002443803707137704 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:86.71484375 - response_length/max:288.0 - response_length/min:34.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.71484375 - response_length_non_aborted/max:288.0 - response_length_non_aborted/min:34.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:620.15625 - prompt_length/max:1132.0 - prompt_length/min:273.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010256282985210419 - timing_s/agent_loop/generate_sequences/min:np.float64(0.534924853593111) - timing_s/agent_loop/generate_sequences/max:np.float64(2.1670787297189236) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.026648185783415) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.1670787297189236) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:648 - timing_s/agent_loop/slowest/response_length:280 - timing_s/gen:8.125570883974433 - timing_s/reward:0.07329580187797546 - timing_s/old_log_prob:2.2333226762712 - timing_s/adv:0.613065805286169 - timing_s/update_actor:15.762878144159913 - timing_s/step:26.891811344772577 - timing_s/stop_profile:2.8675422072410583e-05 - timing_per_token_ms/update_actor:0.08710745607656935 - timing_per_token_ms/gen:0.36603319446706756 - timing_per_token_ms/adv:0.003387871314972834 - perf/total_num_tokens:180959 - perf/time_per_step:26.891811344772577 - perf/throughput:841.1435998117328 (TaskRunner pid=2204243) Training Progress: 39%|███▉ | 39/100 [24:55<30:47, 30.28s/it] (TaskRunner pid=2204243) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2204243) validation generation end (TaskRunner pid=2204243) [prompt] user (TaskRunner pid=2204243) Your task is to answer the user's question using available tools. (TaskRunner pid=2204243) You have access to the following tools: (TaskRunner pid=2204243) Name: Axolotl (TaskRunner pid=2204243) Description: Collection of axolotl pictures and facts (TaskRunner pid=2204243) Documentation: (TaskRunner pid=2204243) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2204243) Parameters: {} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{url, source, description} (TaskRunner pid=2204243) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2204243) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2204243) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2204243) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Array[Object{fact, source}] (TaskRunner pid=2204243) (TaskRunner pid=2204243) Use the following format: (TaskRunner pid=2204243) Thought: you should always think about what to do (TaskRunner pid=2204243) Action: the action to take, should be one of the tool names. (TaskRunner pid=2204243) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2204243) (TaskRunner pid=2204243) Begin! (TaskRunner pid=2204243) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2204243) assistant (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: I need to find a random axolotl image. The user is asking for a random picture, so I should use the getRandomAxolotlImage tool. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2204655) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2204243) step:40 - global_seqlen/min:18855 - global_seqlen/max:25592 - global_seqlen/minmax_diff:6737 - global_seqlen/balanced_min:21685 - global_seqlen/balanced_max:21687 - global_seqlen/mean:21685.625 - actor/entropy:0.15498574078083038 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24134781956672668 - training/rollout_probs_diff_mean:0.004059165250509977 - training/rollout_probs_diff_std:0.012976108118891716 - training/rollout_actor_probs_pearson_corr:0.9978162050247192 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.34375 - self_distillation/correct_sample_fraction:0.27734375 - self_distillation/feedback_available_fraction:0.72265625 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000344514846802 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8342734575271606 - rollout_corr/rollout_is_min:0.5350900888442993 - rollout_corr/rollout_is_std:0.0367705300450325 - rollout_corr/rollout_is_eff_sample_size:0.9986498727551798 - rollout_corr/rollout_is_seq_mean:0.9999096393585205 - rollout_corr/rollout_is_seq_std:0.004265214782208204 - rollout_corr/rollout_is_seq_max:1.0155973434448242 - rollout_corr/rollout_is_seq_min:0.9885300397872925 - rollout_corr/rollout_is_seq_max_deviation:0.015597343444824219 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.155925772152841) - rollout_corr/training_log_ppl:np.float64(0.13926204201379733) - rollout_corr/kl:np.float64(0.0012540893715975443) - rollout_corr/k3_kl:np.float64(0.0010578535586053418) - rollout_corr/rollout_ppl:np.float64(1.1543621313758194) - rollout_corr/rollout_log_ppl:np.float64(0.13800795268980437) - rollout_corr/log_ppl_diff:np.float64(0.0012540893239929574) - rollout_corr/log_ppl_abs_diff:np.float64(0.0043306517454766436) - rollout_corr/log_ppl_diff_max:np.float64(0.04409833252429962) - rollout_corr/log_ppl_diff_min:np.float64(-0.015444919466972351) - rollout_corr/ppl_ratio:np.float64(1.0012735622003675) - rollout_corr/chi2_token:np.float64(0.0014706335496157408) - rollout_corr/chi2_seq:np.float64(0.1790581657551229) - actor/pg_loss:np.float64(0.0012572482228279114) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008998411976790521) - actor/ppo_kl:np.float64(0.0005076362810996216) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.27734375) - self_distillation/token_reweight_w_mean:np.float64(273362.1280443887) - self_distillation/token_reweight_w_std:np.float64(2316649.477929814) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000179652357474) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020189722505165264) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.4187011569738388) - perf/mfu/actor:np.float64(0.03948375387379796) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.55215454101562) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5707720588235294) - val-aux/tooluse/reward/std@16:np.float64(0.05399704825636849) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5915735294117647) - val-aux/tooluse/reward/best@2/std:np.float64(0.047841802073095234) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5493970588235294) - val-aux/tooluse/reward/worst@2/std:np.float64(0.04297319660276272) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5708382352941177) - val-aux/tooluse/reward/maj@2/std:np.float64(0.05429929000289233) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6114852941176471) - val-aux/tooluse/reward/best@4/std:np.float64(0.03971257618864854) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5341911764705882) - val-aux/tooluse/reward/worst@4/std:np.float64(0.033105205870473484) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5692941176470586) - val-aux/tooluse/reward/maj@4/std:np.float64(0.04111283282533723) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6286470588235293) - val-aux/tooluse/reward/best@8/std:np.float64(0.028925370310933868) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5215588235294117) - val-aux/tooluse/reward/worst@8/std:np.float64(0.02489246810480766) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5668382352941177) - val-aux/tooluse/reward/maj@8/std:np.float64(0.028498039759103255) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6392352941176471) - val-aux/tooluse/reward/best@16/std:np.float64(0.016143572245395064) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.51125) - val-aux/tooluse/reward/worst@16/std:np.float64(0.018535692394313557) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5646617647058825) - val-aux/tooluse/reward/maj@16/std:np.float64(0.0177160012819154) - val-aux/tooluse/score/mean@16:np.float64(0.5707720588235294) - val-aux/tooluse/score/std@16:np.float64(0.05399704825636849) - val-aux/tooluse/score/best@2/mean:np.float64(0.5915735294117647) - val-aux/tooluse/score/best@2/std:np.float64(0.047841802073095234) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5493970588235294) - val-aux/tooluse/score/worst@2/std:np.float64(0.04297319660276272) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5708382352941177) - val-aux/tooluse/score/maj@2/std:np.float64(0.05429929000289233) - val-aux/tooluse/score/best@4/mean:np.float64(0.6114852941176471) - val-aux/tooluse/score/best@4/std:np.float64(0.03971257618864854) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5341911764705882) - val-aux/tooluse/score/worst@4/std:np.float64(0.033105205870473484) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5692941176470586) - val-aux/tooluse/score/maj@4/std:np.float64(0.04111283282533723) - val-aux/tooluse/score/best@8/mean:np.float64(0.6286470588235293) - val-aux/tooluse/score/best@8/std:np.float64(0.028925370310933868) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5215588235294117) - val-aux/tooluse/score/worst@8/std:np.float64(0.02489246810480766) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5668382352941177) - val-aux/tooluse/score/maj@8/std:np.float64(0.028498039759103255) - val-aux/tooluse/score/best@16/mean:np.float64(0.6392352941176471) - val-aux/tooluse/score/best@16/std:np.float64(0.016143572245395064) - val-aux/tooluse/score/worst@16/mean:np.float64(0.51125) - val-aux/tooluse/score/worst@16/std:np.float64(0.018535692394313557) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5646617647058825) - val-aux/tooluse/score/maj@16/std:np.float64(0.0177160012819154) - val-core/tooluse/acc/mean@16:np.float64(0.5707720588235294) - val-aux/tooluse/acc/std@16:np.float64(0.05399704825636849) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5915735294117647) - val-aux/tooluse/acc/best@2/std:np.float64(0.047841802073095234) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5493970588235294) - val-aux/tooluse/acc/worst@2/std:np.float64(0.04297319660276272) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5708382352941177) - val-aux/tooluse/acc/maj@2/std:np.float64(0.05429929000289233) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6114852941176471) - val-aux/tooluse/acc/best@4/std:np.float64(0.03971257618864854) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5341911764705882) - val-aux/tooluse/acc/worst@4/std:np.float64(0.033105205870473484) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5692941176470586) - val-aux/tooluse/acc/maj@4/std:np.float64(0.04111283282533723) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6286470588235293) - val-aux/tooluse/acc/best@8/std:np.float64(0.028925370310933868) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5215588235294117) - val-aux/tooluse/acc/worst@8/std:np.float64(0.02489246810480766) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5668382352941177) - val-aux/tooluse/acc/maj@8/std:np.float64(0.028498039759103255) - val-core/tooluse/acc/best@16/mean:np.float64(0.6392352941176471) - val-core/tooluse/acc/best@16/std:np.float64(0.016143572245395064) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.51125) - val-aux/tooluse/acc/worst@16/std:np.float64(0.018535692394313557) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5646617647058825) - val-core/tooluse/acc/maj@16/std:np.float64(0.0177160012819154) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.27734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.27734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004095576237887144 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.004095576237887144 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:77.61328125 - response_length/max:297.0 - response_length/min:32.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:77.61328125 - response_length_non_aborted/max:297.0 - response_length_non_aborted/min:32.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:600.0625 - prompt_length/max:923.0 - prompt_length/min:346.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.613934576511383e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.48701853677630424) - timing_s/agent_loop/generate_sequences/max:np.float64(2.1557283271104097) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.95101194914605) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.1557283271104097) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:614 - timing_s/agent_loop/slowest/response_length:297 - timing_s/gen:7.933874558657408 - timing_s/reward:0.07314835675060749 - timing_s/old_log_prob:2.2649934235960245 - timing_s/adv:0.5949201565235853 - timing_s/update_actor:15.509780054911971 - timing_s/step:26.46442878805101 - timing_s/testing:21.072301253676414 - timing_s/save_checkpoint:6.524287695065141 - timing_s/stop_profile:0.00013312511146068573 - timing_per_token_ms/update_actor:0.08940127420187319 - timing_per_token_ms/gen:0.39930920321392155 - timing_per_token_ms/adv:0.003429231095043291 - perf/total_num_tokens:173485 - perf/time_per_step:26.46442878805101 - perf/throughput:819.4253945050689 (TaskRunner pid=2204243) Training Progress: 40%|████ | 40/100 [25:49<37:26, 37.44s/it] (TaskRunner pid=2204243) step:41 - global_seqlen/min:16939 - global_seqlen/max:28738 - global_seqlen/minmax_diff:11799 - global_seqlen/balanced_min:22883 - global_seqlen/balanced_max:22886 - global_seqlen/mean:22884.75 - actor/entropy:0.13451774418354034 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463781476020813 - training/rollout_probs_diff_mean:0.003906734753400087 - training/rollout_probs_diff_std:0.01324495580047369 - training/rollout_actor_probs_pearson_corr:0.9975051283836365 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.3046875 - self_distillation/correct_sample_fraction:0.24609375 - self_distillation/feedback_available_fraction:0.75390625 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:0.99945467710495 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6179554462432861 - rollout_corr/rollout_is_min:0.5510534644126892 - rollout_corr/rollout_is_std:0.03780870512127876 - rollout_corr/rollout_is_eff_sample_size:0.9985709376208617 - rollout_corr/rollout_is_seq_mean:0.9995222687721252 - rollout_corr/rollout_is_seq_std:0.004593297839164734 - rollout_corr/rollout_is_seq_max:1.0143650770187378 - rollout_corr/rollout_is_seq_min:0.9869683980941772 - rollout_corr/rollout_is_seq_max_deviation:0.014365077018737793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.136925593484193) - rollout_corr/training_log_ppl:np.float64(0.122384807540584) - rollout_corr/kl:np.float64(0.0015938217758275641) - rollout_corr/k3_kl:np.float64(0.0012835161193235911) - rollout_corr/rollout_ppl:np.float64(1.1351020499132574) - rollout_corr/rollout_log_ppl:np.float64(0.120790985442909) - rollout_corr/log_ppl_diff:np.float64(0.0015938220831230865) - rollout_corr/log_ppl_abs_diff:np.float64(0.004291237165489292) - rollout_corr/log_ppl_diff_max:np.float64(0.1135956346988678) - rollout_corr/log_ppl_diff_min:np.float64(-0.022269196808338165) - rollout_corr/ppl_ratio:np.float64(1.0016328152269125) - rollout_corr/chi2_token:np.float64(0.0014729148242622614) - rollout_corr/chi2_seq:np.float64(0.07189773162826896) - actor/pg_loss:np.float64(0.0004553523613139987) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000369061111996416) - actor/ppo_kl:np.float64(0.00040975145709865046) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.24609375) - self_distillation/token_reweight_w_mean:np.float64(403308.3239590796) - self_distillation/token_reweight_w_std:np.float64(2701249.198331125) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008678366430104) - self_distillation/token_reweight_w_clip_frac:np.float64(0.009136563829088118) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.7533609392121434) - perf/mfu/actor:np.float64(0.04198118251695549) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.6773910522461) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.24609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.24609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00017224175098817796 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.00017224175098817796 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:82.2109375 - response_length/max:246.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:82.2109375 - response_length_non_aborted/max:246.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:632.9375 - prompt_length/max:1087.0 - prompt_length/min:269.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012574531137943268 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5199193302541971) - timing_s/agent_loop/generate_sequences/max:np.float64(1.847485188394785) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9583117123693228) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.847485188394785) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:544 - timing_s/agent_loop/slowest/response_length:246 - timing_s/gen:7.748473668470979 - timing_s/reward:0.07025398686528206 - timing_s/old_log_prob:2.4495950490236282 - timing_s/adv:0.6306618172675371 - timing_s/update_actor:15.603976730257273 - timing_s/step:26.608383923768997 - timing_s/stop_profile:0.00011967308819293976 - timing_per_token_ms/update_actor:0.08523130430885892 - timing_per_token_ms/gen:0.36816847232115263 - timing_per_token_ms/adv:0.003444771175496439 - perf/total_num_tokens:183078 - perf/time_per_step:26.608383923768997 - perf/throughput:860.0578699391543 (TaskRunner pid=2204243) Training Progress: 41%|████ | 41/100 [26:15<33:38, 34.21s/it] (TaskRunner pid=2204243) step:42 - global_seqlen/min:16352 - global_seqlen/max:23887 - global_seqlen/minmax_diff:7535 - global_seqlen/balanced_min:19779 - global_seqlen/balanced_max:19780 - global_seqlen/mean:19779.875 - actor/entropy:0.11411407589912415 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21511715650558472 - training/rollout_probs_diff_mean:0.003644315991550684 - training/rollout_probs_diff_std:0.012841069139540195 - training/rollout_actor_probs_pearson_corr:0.9972410202026367 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.37109375 - self_distillation/correct_sample_fraction:0.33984375 - self_distillation/feedback_available_fraction:0.66015625 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.0000821352005005 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.907130241394043 - rollout_corr/rollout_is_min:0.6090996861457825 - rollout_corr/rollout_is_std:0.03618556261062622 - rollout_corr/rollout_is_eff_sample_size:0.9986925552318625 - rollout_corr/rollout_is_seq_mean:1.0000348091125488 - rollout_corr/rollout_is_seq_std:0.004476198926568031 - rollout_corr/rollout_is_seq_max:1.014681100845337 - rollout_corr/rollout_is_seq_min:0.9832686185836792 - rollout_corr/rollout_is_seq_max_deviation:0.0167313814163208 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1140988417901099) - rollout_corr/training_log_ppl:np.float64(0.10468776965763027) - rollout_corr/kl:np.float64(0.00042419295387752243) - rollout_corr/k3_kl:np.float64(0.0008959385362383157) - rollout_corr/rollout_ppl:np.float64(1.1135953310877085) - rollout_corr/rollout_log_ppl:np.float64(0.1042635755311494) - rollout_corr/log_ppl_diff:np.float64(0.0004241941264808702) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036938368307346536) - rollout_corr/log_ppl_diff_max:np.float64(0.019276320934295654) - rollout_corr/log_ppl_diff_min:np.float64(-0.016251815482974052) - rollout_corr/ppl_ratio:np.float64(1.000437222654) - rollout_corr/chi2_token:np.float64(0.0028567493427544832) - rollout_corr/chi2_seq:np.float64(0.20289407181553543) - actor/pg_loss:np.float64(0.0003182929940521717) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021898675186093897) - actor/ppo_kl:np.float64(-0.0001667605853015175) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.33984375) - self_distillation/token_reweight_w_mean:np.float64(352286.8143235217) - self_distillation/token_reweight_w_std:np.float64(2238004.3202553447) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999998411629349) - self_distillation/token_reweight_w_clip_frac:np.float64(0.00717775990779046) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.22294482961297035) - perf/mfu/actor:np.float64(0.03634225732150999) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.6547794342041) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.33984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011759292101487517 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0011759292101487517 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:76.40234375 - response_length/max:233.0 - response_length/min:35.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:76.40234375 - response_length_non_aborted/max:233.0 - response_length_non_aborted/min:35.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:541.71875 - prompt_length/max:888.0 - prompt_length/min:229.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013851560652256012 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5243717040866613) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7772518638521433) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9348346871120157) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7772518638521433) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:687 - timing_s/agent_loop/slowest/response_length:233 - timing_s/gen:7.769186474382877 - timing_s/reward:0.10798259265720844 - timing_s/old_log_prob:2.354940064251423 - timing_s/adv:0.5551851354539394 - timing_s/update_actor:15.522776503115892 - timing_s/step:26.404548421502113 - timing_s/stop_profile:0.00015434063971042633 - timing_per_token_ms/update_actor:0.09809703362076284 - timing_per_token_ms/gen:0.3972179801821605 - timing_per_token_ms/adv:0.0035085227753836883 - perf/total_num_tokens:158239 - perf/time_per_step:26.404548421502113 - perf/throughput:749.1086264475775 (TaskRunner pid=2204243) Training Progress: 42%|████▏ | 42/100 [26:42<30:49, 31.88s/it] (TaskRunner pid=2204243) step:43 - global_seqlen/min:17601 - global_seqlen/max:29726 - global_seqlen/minmax_diff:12125 - global_seqlen/balanced_min:22719 - global_seqlen/balanced_max:22720 - global_seqlen/mean:22719.75 - actor/entropy:0.1049850583076477 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2121104598045349 - training/rollout_probs_diff_mean:0.003100998466834426 - training/rollout_probs_diff_std:0.011506113223731518 - training/rollout_actor_probs_pearson_corr:0.9975945353507996 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.3125 - self_distillation/correct_sample_fraction:0.296875 - self_distillation/feedback_available_fraction:0.703125 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.000309705734253 - rollout_corr/rollout_is_ratio_fraction_high:8.806693222140893e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.403346611070447e-05 - rollout_corr/rollout_is_max:3.043858051300049 - rollout_corr/rollout_is_min:0.4731205999851227 - rollout_corr/rollout_is_std:0.03333033621311188 - rollout_corr/rollout_is_eff_sample_size:0.9988907971649656 - rollout_corr/rollout_is_seq_mean:1.0005097389221191 - rollout_corr/rollout_is_seq_std:0.0040773916989564896 - rollout_corr/rollout_is_seq_max:1.0154672861099243 - rollout_corr/rollout_is_seq_min:0.9907342791557312 - rollout_corr/rollout_is_seq_max_deviation:0.015467286109924316 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1044714045710862) - rollout_corr/training_log_ppl:np.float64(0.09725696136956685) - rollout_corr/kl:np.float64(0.00010887079873356242) - rollout_corr/k3_kl:np.float64(0.0007772656189555249) - rollout_corr/rollout_ppl:np.float64(1.1043232576921582) - rollout_corr/rollout_log_ppl:np.float64(0.09714808937496855) - rollout_corr/log_ppl_diff:np.float64(0.00010887199459830299) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034523115973570384) - rollout_corr/log_ppl_diff_max:np.float64(0.01277557760477066) - rollout_corr/log_ppl_diff_min:np.float64(-0.013635165989398956) - rollout_corr/ppl_ratio:np.float64(1.0001191704068333) - rollout_corr/chi2_token:np.float64(0.003161896485835314) - rollout_corr/chi2_seq:np.float64(0.4080053570214659) - actor/pg_loss:np.float64(-0.0001070820726454258) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020541939738905057) - actor/ppo_kl:np.float64(5.767263503742015e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.296875) - self_distillation/token_reweight_w_mean:np.float64(99195.15524136275) - self_distillation/token_reweight_w_std:np.float64(765105.2651277883) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001152083277702) - self_distillation/token_reweight_w_clip_frac:np.float64(0.002422373800072819) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.3116052746772766) - perf/mfu/actor:np.float64(0.041186767956981236) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.66334533691406) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012769704917445779 - critic/advantages/max:0.5 - critic/advantages/min:-0.5 - critic/returns/mean:-0.0012769704917445779 - critic/returns/max:0.5 - critic/returns/min:-0.5 - response_length/mean:88.7109375 - response_length/max:352.0 - response_length/min:39.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:88.7109375 - response_length_non_aborted/max:352.0 - response_length_non_aborted/min:39.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:621.28125 - prompt_length/max:991.0 - prompt_length/min:258.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010633096098899841 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4909998755902052) - timing_s/agent_loop/generate_sequences/max:np.float64(2.5499944984912872) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0431026595033472) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.5499944984912872) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:635 - timing_s/agent_loop/slowest/response_length:352 - timing_s/gen:8.46652709133923 - timing_s/reward:0.06983864866197109 - timing_s/old_log_prob:2.269910551607609 - timing_s/adv:0.6237467359751463 - timing_s/update_actor:15.822451334446669 - timing_s/step:27.327828438952565 - timing_s/stop_profile:0.00011335685849189758 - timing_per_token_ms/update_actor:0.08705229664964771 - timing_per_token_ms/gen:0.3728105280202215 - timing_per_token_ms/adv:0.003431742954781337 - perf/total_num_tokens:181758 - perf/time_per_step:27.327828438952565 - perf/throughput:831.3778041586247 (TaskRunner pid=2204243) Training Progress: 43%|████▎ | 43/100 [27:09<29:00, 30.53s/it] (TaskRunner pid=2204243) step:44 - global_seqlen/min:17415 - global_seqlen/max:31039 - global_seqlen/minmax_diff:13624 - global_seqlen/balanced_min:21380 - global_seqlen/balanced_max:22165 - global_seqlen/mean:21481.375 - actor/entropy:0.13091859221458435 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24394366145133972 - training/rollout_probs_diff_mean:0.0033418694511055946 - training/rollout_probs_diff_std:0.011878211051225662 - training/rollout_actor_probs_pearson_corr:0.9979523420333862 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.3125 - self_distillation/correct_sample_fraction:0.296875 - self_distillation/feedback_available_fraction:0.703125 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.999733030796051 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9363372325897217 - rollout_corr/rollout_is_min:0.5597013831138611 - rollout_corr/rollout_is_std:0.03463786840438843 - rollout_corr/rollout_is_eff_sample_size:0.9988010017767082 - rollout_corr/rollout_is_seq_mean:0.9995107054710388 - rollout_corr/rollout_is_seq_std:0.004361338447779417 - rollout_corr/rollout_is_seq_max:1.0127485990524292 - rollout_corr/rollout_is_seq_min:0.9873204231262207 - rollout_corr/rollout_is_seq_max_deviation:0.0127485990524292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1481413580477238) - rollout_corr/training_log_ppl:np.float64(0.13006715874871588) - rollout_corr/kl:np.float64(4.5177927304962395e-05) - rollout_corr/k3_kl:np.float64(0.0007918616236266551) - rollout_corr/rollout_ppl:np.float64(1.1481290902011096) - rollout_corr/rollout_log_ppl:np.float64(0.13002198007961852) - rollout_corr/log_ppl_diff:np.float64(4.5178669097367674e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.003999868531536777) - rollout_corr/log_ppl_diff_max:np.float64(0.022446148097515106) - rollout_corr/log_ppl_diff_min:np.float64(-0.01722322404384613) - rollout_corr/ppl_ratio:np.float64(1.0000597559846938) - rollout_corr/chi2_token:np.float64(0.0031360944267362356) - rollout_corr/chi2_seq:np.float64(0.24645976652391255) - actor/pg_loss:np.float64(5.800859071314335e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002185421544709243) - actor/ppo_kl:np.float64(-0.001077756299494581) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.296875) - self_distillation/token_reweight_w_mean:np.float64(28065.533878439805) - self_distillation/token_reweight_w_std:np.float64(317684.8927982212) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999411725439131) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0045203372137621045) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.27540536038577557) - perf/mfu/actor:np.float64(0.03952085566348435) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.69012451171875) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008439205586910248 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008439205586910248 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:85.16796875 - response_length/max:1623.0 - response_length/min:32.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:85.16796875 - response_length_non_aborted/max:1623.0 - response_length_non_aborted/min:32.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:586.125 - prompt_length/max:1024.0 - prompt_length/min:253.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.430013597011566e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.41800827719271183) - timing_s/agent_loop/generate_sequences/max:np.float64(8.120173122733831) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9001846894607297) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.120173122733831) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:655 - timing_s/agent_loop/slowest/response_length:1623 - timing_s/gen:14.46633660979569 - timing_s/reward:0.07274330034852028 - timing_s/old_log_prob:2.30580491758883 - timing_s/adv:0.5894836764782667 - timing_s/update_actor:15.506691914051771 - timing_s/step:33.02900003269315 - timing_s/stop_profile:0.00010792911052703857 - timing_per_token_ms/update_actor:0.09023335281174838 - timing_per_token_ms/gen:0.66350211483721 - timing_per_token_ms/adv:0.0034302021895611123 - perf/total_num_tokens:171851 - perf/time_per_step:33.02900003269315 - perf/throughput:650.3792115636882 (TaskRunner pid=2204243) Training Progress: 44%|████▍ | 44/100 [27:42<29:12, 31.30s/it] (TaskRunner pid=2204243) step:45 - global_seqlen/min:15541 - global_seqlen/max:27355 - global_seqlen/minmax_diff:11814 - global_seqlen/balanced_min:21573 - global_seqlen/balanced_max:21576 - global_seqlen/mean:21574.75 - actor/entropy:0.09393709152936935 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1722739338874817 - training/rollout_probs_diff_mean:0.0028341070283204317 - training/rollout_probs_diff_std:0.010932925157248974 - training/rollout_actor_probs_pearson_corr:0.9976241588592529 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.4375 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.578125 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0002236366271973 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8464802503585815 - rollout_corr/rollout_is_min:0.6159732937812805 - rollout_corr/rollout_is_std:0.03361169993877411 - rollout_corr/rollout_is_eff_sample_size:0.9988720042114383 - rollout_corr/rollout_is_seq_mean:1.0003440380096436 - rollout_corr/rollout_is_seq_std:0.004466402810066938 - rollout_corr/rollout_is_seq_max:1.020525574684143 - rollout_corr/rollout_is_seq_min:0.9882777333259583 - rollout_corr/rollout_is_seq_max_deviation:0.020525574684143066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0975356362760067) - rollout_corr/training_log_ppl:np.float64(0.09074880334992486) - rollout_corr/kl:np.float64(0.00026883407277722426) - rollout_corr/k3_kl:np.float64(0.0006511839284337739) - rollout_corr/rollout_ppl:np.float64(1.0972242467105389) - rollout_corr/rollout_log_ppl:np.float64(0.09047996771187172) - rollout_corr/log_ppl_diff:np.float64(0.00026883563805313315) - rollout_corr/log_ppl_abs_diff:np.float64(0.003329489425595966) - rollout_corr/log_ppl_diff_max:np.float64(0.015620946884155273) - rollout_corr/log_ppl_diff_min:np.float64(-0.016770780086517334) - rollout_corr/ppl_ratio:np.float64(1.0002794321626425) - rollout_corr/chi2_token:np.float64(0.002041083760559559) - rollout_corr/chi2_seq:np.float64(0.10593510256148875) - actor/pg_loss:np.float64(-0.00023642322048544884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010863181159947999) - actor/ppo_kl:np.float64(5.170268302734371e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(131549.86720586545) - self_distillation/token_reweight_w_std:np.float64(790943.5514261503) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001925979740918) - self_distillation/token_reweight_w_clip_frac:np.float64(0.003669844794785604) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.18755611032247543) - perf/mfu/actor:np.float64(0.039215177419144584) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.73347091674805) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007487520924769342 - critic/advantages/max:0.5 - critic/advantages/min:-0.5 - critic/returns/mean:0.0007487520924769342 - critic/returns/max:0.5 - critic/returns/min:-0.5 - response_length/mean:70.4296875 - response_length/max:180.0 - response_length/min:35.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:70.4296875 - response_length_non_aborted/max:180.0 - response_length_non_aborted/min:35.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:603.78125 - prompt_length/max:1091.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014632940292358398 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5091425217688084) - timing_s/agent_loop/generate_sequences/max:np.float64(1.5953880678862333) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9114149280358106) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.5953880678862333) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:589 - timing_s/agent_loop/slowest/response_length:171 - timing_s/gen:7.379839904606342 - timing_s/reward:0.0714273676276207 - timing_s/old_log_prob:2.3735286593437195 - timing_s/adv:0.5979645363986492 - timing_s/update_actor:15.77385189756751 - timing_s/step:26.282782211899757 - timing_s/stop_profile:0.00011722184717655182 - timing_per_token_ms/update_actor:0.09139069918288457 - timing_per_token_ms/gen:0.4093089242710118 - timing_per_token_ms/adv:0.003464492846954479 - perf/total_num_tokens:172598 - perf/time_per_step:26.282782211899757 - perf/throughput:820.8700976197202 (TaskRunner pid=2204243) Training Progress: 45%|████▌ | 45/100 [28:09<27:19, 29.81s/it] (TaskRunner pid=2204243) step:46 - global_seqlen/min:14924 - global_seqlen/max:29306 - global_seqlen/minmax_diff:14382 - global_seqlen/balanced_min:23114 - global_seqlen/balanced_max:23116 - global_seqlen/mean:23114.875 - actor/entropy:0.09701564908027649 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22949427366256714 - training/rollout_probs_diff_mean:0.003071932587772608 - training/rollout_probs_diff_std:0.011730343103408813 - training/rollout_actor_probs_pearson_corr:0.9972602128982544 - self_distillation/success_group_fraction:0.25 - self_distillation/success_sample_fraction:0.25 - self_distillation/correct_sample_fraction:0.25 - self_distillation/feedback_available_fraction:0.75 - self_distillation/feedback_used_fraction:0.75 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.00041925907135 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5869412422180176 - rollout_corr/rollout_is_min:0.6071226596832275 - rollout_corr/rollout_is_std:0.03148940950632095 - rollout_corr/rollout_is_eff_sample_size:0.9990102322054077 - rollout_corr/rollout_is_seq_mean:1.0004991292953491 - rollout_corr/rollout_is_seq_std:0.004046741407364607 - rollout_corr/rollout_is_seq_max:1.0201334953308105 - rollout_corr/rollout_is_seq_min:0.985393226146698 - rollout_corr/rollout_is_seq_max_deviation:0.020133495330810547 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1022526612505317) - rollout_corr/training_log_ppl:np.float64(0.09510156522810576) - rollout_corr/kl:np.float64(5.16430548528346e-05) - rollout_corr/k3_kl:np.float64(0.0006735909447670441) - rollout_corr/rollout_ppl:np.float64(1.1021420890465379) - rollout_corr/rollout_log_ppl:np.float64(0.09504992114307242) - rollout_corr/log_ppl_diff:np.float64(5.1644085033331066e-05) - rollout_corr/log_ppl_abs_diff:np.float64(0.003100487323536072) - rollout_corr/log_ppl_diff_max:np.float64(0.013109356164932251) - rollout_corr/log_ppl_diff_min:np.float64(-0.022796165198087692) - rollout_corr/ppl_ratio:np.float64(1.0000616831239313) - rollout_corr/chi2_token:np.float64(0.002731382381170988) - rollout_corr/chi2_seq:np.float64(0.12141061597503722) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(3.662213546107296e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.25) - self_distillation/token_reweight_w_mean:np.float64(0.9999999916180968) - self_distillation/token_reweight_w_std:np.float64(0.0) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999999916180968) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.042447418348367516) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.71398544311523) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.25 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:72.65234375 - response_length/max:175.0 - response_length/min:27.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:72.65234375 - response_length_non_aborted/max:175.0 - response_length_non_aborted/min:27.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:649.6875 - prompt_length/max:1113.0 - prompt_length/min:268.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015606731176376343 - timing_s/agent_loop/generate_sequences/min:np.float64(0.463961374014616) - timing_s/agent_loop/generate_sequences/max:np.float64(1.538517352193594) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9443411030588322) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.538517352193594) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:566 - timing_s/agent_loop/slowest/response_length:175 - timing_s/gen:7.585167666897178 - timing_s/reward:0.07414943166077137 - timing_s/old_log_prob:2.3093747571110725 - timing_s/adv:0.6289361082017422 - timing_s/update_actor:15.775989420711994 - timing_s/step:26.459737364202738 - timing_s/stop_profile:0.00013145990669727325 - timing_per_token_ms/update_actor:0.08531297173742014 - timing_per_token_ms/gen:0.40782663943745245 - timing_per_token_ms/adv:0.003401143788370812 - perf/total_num_tokens:184919 - perf/time_per_step:26.459737364202738 - perf/throughput:873.5867133462939 (TaskRunner pid=2204243) Training Progress: 46%|████▌ | 46/100 [28:35<25:56, 28.82s/it] (TaskRunner pid=2204243) step:47 - global_seqlen/min:17105 - global_seqlen/max:32689 - global_seqlen/minmax_diff:15584 - global_seqlen/balanced_min:21685 - global_seqlen/balanced_max:29151 - global_seqlen/mean:22619.75 - actor/entropy:0.78171306848526 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463563919067383 - training/rollout_probs_diff_mean:0.0031301097478717566 - training/rollout_probs_diff_std:0.010094783268868923 - training/rollout_actor_probs_pearson_corr:0.9996719360351562 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.375 - self_distillation/correct_sample_fraction:0.37109375 - self_distillation/feedback_available_fraction:0.62890625 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999496340751648 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.825261956080794e-05 - rollout_corr/rollout_is_max:1.6338132619857788 - rollout_corr/rollout_is_min:0.484489768743515 - rollout_corr/rollout_is_std:0.03196284547448158 - rollout_corr/rollout_is_eff_sample_size:0.998979181106038 - rollout_corr/rollout_is_seq_mean:0.9999097585678101 - rollout_corr/rollout_is_seq_std:0.003985540010035038 - rollout_corr/rollout_is_seq_max:1.0149580240249634 - rollout_corr/rollout_is_seq_min:0.9855024814605713 - rollout_corr/rollout_is_seq_max_deviation:0.014958024024963379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1510342857800424) - rollout_corr/training_log_ppl:np.float64(0.11676943952625152) - rollout_corr/kl:np.float64(0.0007919220114347425) - rollout_corr/k3_kl:np.float64(0.0006663402650316898) - rollout_corr/rollout_ppl:np.float64(1.1500505520962179) - rollout_corr/rollout_log_ppl:np.float64(0.1159775171909132) - rollout_corr/log_ppl_diff:np.float64(0.0007919223353383131) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032629358101985417) - rollout_corr/log_ppl_diff_max:np.float64(0.01915988326072693) - rollout_corr/log_ppl_diff_min:np.float64(-0.014088243246078491) - rollout_corr/ppl_ratio:np.float64(1.0008019399829209) - rollout_corr/chi2_token:np.float64(0.0010527854319661856) - rollout_corr/chi2_seq:np.float64(0.0232640674803406) - actor/pg_loss:np.float64(0.0002859241794794798) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.00018661713642742495) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37109375) - self_distillation/token_reweight_w_mean:np.float64(43813.83516312181) - self_distillation/token_reweight_w_std:np.float64(258627.98956912814) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999426305294037) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0017608076886972412) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.29779756255447865) - perf/mfu/actor:np.float64(0.03908345854624484) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.751708984375) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.37109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:9.084997145691887e-05 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:9.084997145691887e-05 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:102.1171875 - response_length/max:8192.0 - response_length/min:38.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:102.1171875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:38.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:604.75 - prompt_length/max:974.0 - prompt_length/min:301.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011375918984413147 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5069075990468264) - timing_s/agent_loop/generate_sequences/max:np.float64(46.42898414097726) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0773394289863063) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(46.42898414097726) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:590 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.23416377231479 - timing_s/reward:0.07213537581264973 - timing_s/old_log_prob:2.5689927600324154 - timing_s/adv:0.6200259700417519 - timing_s/update_actor:16.86532241664827 - timing_s/step:72.45371531136334 - timing_s/stop_profile:0.00011219829320907593 - timing_per_token_ms/update_actor:0.09320020345410686 - timing_per_token_ms/gen:1.9980936337049493 - timing_per_token_ms/adv:0.0034263529108508706 - perf/total_num_tokens:180958 - perf/time_per_step:72.45371531136334 - perf/throughput:312.1958605268709 (TaskRunner pid=2204243) Training Progress: 47%|████▋ | 47/100 [29:48<37:01, 41.92s/it] (TaskRunner pid=2204243) step:48 - global_seqlen/min:15968 - global_seqlen/max:32580 - global_seqlen/minmax_diff:16612 - global_seqlen/balanced_min:22661 - global_seqlen/balanced_max:22664 - global_seqlen/mean:22662.625 - actor/entropy:0.12158574908971786 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3477973937988281 - training/rollout_probs_diff_mean:0.0036659413017332554 - training/rollout_probs_diff_std:0.012910273857414722 - training/rollout_actor_probs_pearson_corr:0.9973071813583374 - self_distillation/success_group_fraction:0.28125 - self_distillation/success_sample_fraction:0.28125 - self_distillation/correct_sample_fraction:0.265625 - self_distillation/feedback_available_fraction:0.734375 - self_distillation/feedback_used_fraction:0.71875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9998748302459717 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.198315921006724e-05 - rollout_corr/rollout_is_max:1.6348817348480225 - rollout_corr/rollout_is_min:0.47558870911598206 - rollout_corr/rollout_is_std:0.035987354815006256 - rollout_corr/rollout_is_eff_sample_size:0.9987063475641451 - rollout_corr/rollout_is_seq_mean:0.9998760223388672 - rollout_corr/rollout_is_seq_std:0.004698671400547028 - rollout_corr/rollout_is_seq_max:1.0198969841003418 - rollout_corr/rollout_is_seq_min:0.9845187664031982 - rollout_corr/rollout_is_seq_max_deviation:0.019896984100341797 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1336316326633096) - rollout_corr/training_log_ppl:np.float64(0.12045325426879572) - rollout_corr/kl:np.float64(0.0009278377406900518) - rollout_corr/k3_kl:np.float64(0.0011363851112844436) - rollout_corr/rollout_ppl:np.float64(1.1325111165642738) - rollout_corr/rollout_log_ppl:np.float64(0.11952541557184304) - rollout_corr/log_ppl_diff:np.float64(0.0009278386969526764) - rollout_corr/log_ppl_abs_diff:np.float64(0.004044524135679239) - rollout_corr/log_ppl_diff_max:np.float64(0.038093358278274536) - rollout_corr/log_ppl_diff_min:np.float64(-0.017723526805639267) - rollout_corr/ppl_ratio:np.float64(1.0009480526205152) - rollout_corr/chi2_token:np.float64(0.0024623312056064606) - rollout_corr/chi2_seq:np.float64(0.15918467519804835) - actor/pg_loss:np.float64(0.0003436128608882427) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000147405662573874) - actor/ppo_kl:np.float64(0.00010876671863968568) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.265625) - self_distillation/token_reweight_w_mean:np.float64(417.2280901579652) - self_distillation/token_reweight_w_std:np.float64(2211.922959721065) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003401744179428) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0032765015203040093) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.23359613120555878) - perf/mfu/actor:np.float64(0.042142248014526054) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.78654098510742) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00010396631842013448 - critic/advantages/max:0.5 - critic/advantages/min:-0.5 - critic/returns/mean:-0.00010396631842013448 - critic/returns/max:0.5 - critic/returns/min:-0.5 - response_length/mean:75.14453125 - response_length/max:287.0 - response_length/min:31.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:75.14453125 - response_length_non_aborted/max:287.0 - response_length_non_aborted/min:31.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:633.0625 - prompt_length/max:1499.0 - prompt_length/min:259.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014923512935638428 - timing_s/agent_loop/generate_sequences/min:np.float64(0.47379517555236816) - timing_s/agent_loop/generate_sequences/max:np.float64(2.180079746991396) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9291612271117629) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.180079746991396) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:602 - timing_s/agent_loop/slowest/response_length:280 - timing_s/gen:7.890121741220355 - timing_s/reward:0.07169542089104652 - timing_s/old_log_prob:2.3063189163804054 - timing_s/adv:0.6574728060513735 - timing_s/update_actor:15.559699576348066 - timing_s/step:26.559658244252205 - timing_s/stop_profile:0.00011696480214595795 - timing_per_token_ms/update_actor:0.08582246968493315 - timing_per_token_ms/gen:0.41015344082863 - timing_per_token_ms/adv:0.003626415772948707 - perf/total_num_tokens:181301 - perf/time_per_step:26.559658244252205 - perf/throughput:853.2724627548412 (TaskRunner pid=2204243) Training Progress: 48%|████▊ | 48/100 [30:14<32:21, 37.33s/it] (TaskRunner pid=2204243) step:49 - global_seqlen/min:17611 - global_seqlen/max:26338 - global_seqlen/minmax_diff:8727 - global_seqlen/balanced_min:22086 - global_seqlen/balanced_max:22087 - global_seqlen/mean:22086.5 - actor/entropy:0.10772540420293808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18767410516738892 - training/rollout_probs_diff_mean:0.003266825806349516 - training/rollout_probs_diff_std:0.012206732295453548 - training/rollout_actor_probs_pearson_corr:0.9973509907722473 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.375 - self_distillation/correct_sample_fraction:0.3515625 - self_distillation/feedback_available_fraction:0.6484375 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.000151515007019 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.99300986120943e-05 - rollout_corr/rollout_is_max:1.9518455266952515 - rollout_corr/rollout_is_min:0.48470813035964966 - rollout_corr/rollout_is_std:0.03541133180260658 - rollout_corr/rollout_is_eff_sample_size:0.9987480835796785 - rollout_corr/rollout_is_seq_mean:1.000097393989563 - rollout_corr/rollout_is_seq_std:0.004309747368097305 - rollout_corr/rollout_is_seq_max:1.0218305587768555 - rollout_corr/rollout_is_seq_min:0.9875327348709106 - rollout_corr/rollout_is_seq_max_deviation:0.02183055877685547 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1111758090555668) - rollout_corr/training_log_ppl:np.float64(0.10094921132440504) - rollout_corr/kl:np.float64(0.0008797461489871983) - rollout_corr/k3_kl:np.float64(0.0007332546720443534) - rollout_corr/rollout_ppl:np.float64(1.110091405455023) - rollout_corr/rollout_log_ppl:np.float64(0.1000694645981639) - rollout_corr/log_ppl_diff:np.float64(0.0008797467262411374) - rollout_corr/log_ppl_abs_diff:np.float64(0.0033781125575842452) - rollout_corr/log_ppl_diff_max:np.float64(0.016971111297607422) - rollout_corr/log_ppl_diff_min:np.float64(-0.015144824981689453) - rollout_corr/ppl_ratio:np.float64(1.00089065101929) - rollout_corr/chi2_token:np.float64(0.0011285394430160522) - rollout_corr/chi2_seq:np.float64(0.12210897379554808) - actor/pg_loss:np.float64(7.439963519573212e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003152230492560193) - actor/ppo_kl:np.float64(0.00038846359332467273) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3515625) - self_distillation/token_reweight_w_mean:np.float64(229084.46303244028) - self_distillation/token_reweight_w_std:np.float64(1525211.7124935784) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003775488585234) - self_distillation/token_reweight_w_clip_frac:np.float64(0.004672748138546012) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.46622749231755733) - perf/mfu/actor:np.float64(0.041222799509389015) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.6617774963379) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.3515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0009861193830147386 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0009861193830147386 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:78.234375 - response_length/max:269.0 - response_length/min:39.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:78.234375 - response_length_non_aborted/max:269.0 - response_length_non_aborted/min:39.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:611.96875 - prompt_length/max:1026.0 - prompt_length/min:309.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016329996287822723 - timing_s/agent_loop/generate_sequences/min:np.float64(0.46982434019446373) - timing_s/agent_loop/generate_sequences/max:np.float64(1.8211136907339096) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.8732757365141879) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.8211136907339096) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:757 - timing_s/agent_loop/slowest/response_length:269 - timing_s/gen:7.626596696674824 - timing_s/reward:0.08472305722534657 - timing_s/old_log_prob:2.384451065212488 - timing_s/adv:0.5974102355539799 - timing_s/update_actor:15.378535756841302 - timing_s/step:26.165667632594705 - timing_s/stop_profile:0.00012943334877490997 - timing_per_token_ms/update_actor:0.08703583499446099 - timing_per_token_ms/gen:0.38079671942654403 - timing_per_token_ms/adv:0.0033810825365833195 - perf/total_num_tokens:176692 - perf/time_per_step:26.165667632594705 - perf/throughput:844.1022912209867 (TaskRunner pid=2204243) Training Progress: 49%|████▉ | 49/100 [30:41<28:54, 34.00s/it] (TaskRunner pid=2204243) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2204243) validation generation end (TaskRunner pid=2204243) [prompt] user (TaskRunner pid=2204243) Your task is to answer the user's question using available tools. (TaskRunner pid=2204243) You have access to the following tools: (TaskRunner pid=2204243) Name: Axolotl (TaskRunner pid=2204243) Description: Collection of axolotl pictures and facts (TaskRunner pid=2204243) Documentation: (TaskRunner pid=2204243) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2204243) Parameters: {} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{url, source, description} (TaskRunner pid=2204243) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2204243) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2204243) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2204243) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Array[Object{fact, source}] (TaskRunner pid=2204243) (TaskRunner pid=2204243) Use the following format: (TaskRunner pid=2204243) Thought: you should always think about what to do (TaskRunner pid=2204243) Action: the action to take, should be one of the tool names. (TaskRunner pid=2204243) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2204243) (TaskRunner pid=2204243) Begin! (TaskRunner pid=2204243) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2204243) assistant (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: I need to find a random axolotl image. The user asked for a random picture, so I should use the getRandomAxolotlImage tool. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2204655) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2204243) step:50 - global_seqlen/min:19086 - global_seqlen/max:24016 - global_seqlen/minmax_diff:4930 - global_seqlen/balanced_min:21347 - global_seqlen/balanced_max:21348 - global_seqlen/mean:21347.125 - actor/entropy:0.09960741549730301 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18463638424873352 - training/rollout_probs_diff_mean:0.003469532122835517 - training/rollout_probs_diff_std:0.012408807873725891 - training/rollout_actor_probs_pearson_corr:0.9969720840454102 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.42578125 - self_distillation/correct_sample_fraction:0.30859375 - self_distillation/feedback_available_fraction:0.69140625 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:1.0004281997680664 - rollout_corr/rollout_is_ratio_fraction_high:5.208062066230923e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.0745599269866943 - rollout_corr/rollout_is_min:0.5543735027313232 - rollout_corr/rollout_is_std:0.03567127510905266 - rollout_corr/rollout_is_eff_sample_size:0.9987302472487798 - rollout_corr/rollout_is_seq_mean:1.0004222393035889 - rollout_corr/rollout_is_seq_std:0.004516529850661755 - rollout_corr/rollout_is_seq_max:1.0310603380203247 - rollout_corr/rollout_is_seq_min:0.9876805543899536 - rollout_corr/rollout_is_seq_max_deviation:0.031060338020324707 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1024314020760357) - rollout_corr/training_log_ppl:np.float64(0.09458866953718825) - rollout_corr/kl:np.float64(0.000829289464764571) - rollout_corr/k3_kl:np.float64(0.0014336688870209002) - rollout_corr/rollout_ppl:np.float64(1.1014101514592767) - rollout_corr/rollout_log_ppl:np.float64(0.09375937961158343) - rollout_corr/log_ppl_diff:np.float64(0.000829289921966847) - rollout_corr/log_ppl_abs_diff:np.float64(0.005084052187157795) - rollout_corr/log_ppl_diff_max:np.float64(0.03320774435997009) - rollout_corr/log_ppl_diff_min:np.float64(-0.017315220087766647) - rollout_corr/ppl_ratio:np.float64(1.000856357626617) - rollout_corr/chi2_token:np.float64(0.003805949352681637) - rollout_corr/chi2_seq:np.float64(0.33349762205034494) - actor/pg_loss:np.float64(0.0009683704702183604) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008467895167996176) - actor/ppo_kl:np.float64(0.0005741024954204477) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.30859375) - self_distillation/token_reweight_w_mean:np.float64(746090.7167412583) - self_distillation/token_reweight_w_std:np.float64(5518735.194646507) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020667829085141) - self_distillation/token_reweight_w_clip_frac:np.float64(0.025049185554962605) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.530059278011322) - perf/mfu/actor:np.float64(0.03911523240564726) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.69704055786133) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5533088235294118) - val-aux/tooluse/reward/std@16:np.float64(0.04515779690563993) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5710147058823529) - val-aux/tooluse/reward/best@2/std:np.float64(0.03588961948367756) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5353235294117648) - val-aux/tooluse/reward/worst@2/std:np.float64(0.040513636891242835) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5520294117647059) - val-aux/tooluse/reward/maj@2/std:np.float64(0.044709639111387464) - val-aux/tooluse/reward/best@4/mean:np.float64(0.5858235294117646) - val-aux/tooluse/reward/best@4/std:np.float64(0.025344883418784707) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.52) - val-aux/tooluse/reward/worst@4/std:np.float64(0.03148284787596885) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5541470588235294) - val-aux/tooluse/reward/maj@4/std:np.float64(0.03654184430860969) - val-aux/tooluse/reward/best@8/mean:np.float64(0.5959558823529413) - val-aux/tooluse/reward/best@8/std:np.float64(0.014481264369154362) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5086029411764706) - val-aux/tooluse/reward/worst@8/std:np.float64(0.024538229665776196) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5536617647058824) - val-aux/tooluse/reward/maj@8/std:np.float64(0.02741865282207127) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6009558823529411) - val-aux/tooluse/reward/best@16/std:np.float64(0.006431212784135987) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.4974852941176471) - val-aux/tooluse/reward/worst@16/std:np.float64(0.019308495171943028) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5523676470588236) - val-aux/tooluse/reward/maj@16/std:np.float64(0.02329050968428219) - val-aux/tooluse/score/mean@16:np.float64(0.5533088235294118) - val-aux/tooluse/score/std@16:np.float64(0.04515779690563993) - val-aux/tooluse/score/best@2/mean:np.float64(0.5710147058823529) - val-aux/tooluse/score/best@2/std:np.float64(0.03588961948367756) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5353235294117648) - val-aux/tooluse/score/worst@2/std:np.float64(0.040513636891242835) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5520294117647059) - val-aux/tooluse/score/maj@2/std:np.float64(0.044709639111387464) - val-aux/tooluse/score/best@4/mean:np.float64(0.5858235294117646) - val-aux/tooluse/score/best@4/std:np.float64(0.025344883418784707) - val-aux/tooluse/score/worst@4/mean:np.float64(0.52) - val-aux/tooluse/score/worst@4/std:np.float64(0.03148284787596885) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5541470588235294) - val-aux/tooluse/score/maj@4/std:np.float64(0.03654184430860969) - val-aux/tooluse/score/best@8/mean:np.float64(0.5959558823529413) - val-aux/tooluse/score/best@8/std:np.float64(0.014481264369154362) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5086029411764706) - val-aux/tooluse/score/worst@8/std:np.float64(0.024538229665776196) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5536617647058824) - val-aux/tooluse/score/maj@8/std:np.float64(0.02741865282207127) - val-aux/tooluse/score/best@16/mean:np.float64(0.6009558823529411) - val-aux/tooluse/score/best@16/std:np.float64(0.006431212784135987) - val-aux/tooluse/score/worst@16/mean:np.float64(0.4974852941176471) - val-aux/tooluse/score/worst@16/std:np.float64(0.019308495171943028) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5523676470588236) - val-aux/tooluse/score/maj@16/std:np.float64(0.02329050968428219) - val-core/tooluse/acc/mean@16:np.float64(0.5533088235294118) - val-aux/tooluse/acc/std@16:np.float64(0.04515779690563993) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5710147058823529) - val-aux/tooluse/acc/best@2/std:np.float64(0.03588961948367756) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5353235294117648) - val-aux/tooluse/acc/worst@2/std:np.float64(0.040513636891242835) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5520294117647059) - val-aux/tooluse/acc/maj@2/std:np.float64(0.044709639111387464) - val-aux/tooluse/acc/best@4/mean:np.float64(0.5858235294117646) - val-aux/tooluse/acc/best@4/std:np.float64(0.025344883418784707) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.52) - val-aux/tooluse/acc/worst@4/std:np.float64(0.03148284787596885) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5541470588235294) - val-aux/tooluse/acc/maj@4/std:np.float64(0.03654184430860969) - val-aux/tooluse/acc/best@8/mean:np.float64(0.5959558823529413) - val-aux/tooluse/acc/best@8/std:np.float64(0.014481264369154362) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5086029411764706) - val-aux/tooluse/acc/worst@8/std:np.float64(0.024538229665776196) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5536617647058824) - val-aux/tooluse/acc/maj@8/std:np.float64(0.02741865282207127) - val-core/tooluse/acc/best@16/mean:np.float64(0.6009558823529411) - val-core/tooluse/acc/best@16/std:np.float64(0.006431212784135987) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.4974852941176471) - val-aux/tooluse/acc/worst@16/std:np.float64(0.019308495171943028) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5523676470588236) - val-core/tooluse/acc/maj@16/std:np.float64(0.02329050968428219) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.30859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004394302610307932 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004394302610307932 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:75.00390625 - response_length/max:200.0 - response_length/min:34.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:75.00390625 - response_length_non_aborted/max:200.0 - response_length_non_aborted/min:34.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:592.09375 - prompt_length/max:911.0 - prompt_length/min:352.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012982077896595 - timing_s/agent_loop/generate_sequences/min:np.float64(0.43292769975960255) - timing_s/agent_loop/generate_sequences/max:np.float64(1.6602819338440895) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.860448016654118) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.6602819338440895) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:487 - timing_s/agent_loop/slowest/response_length:200 - timing_s/gen:7.281735476106405 - timing_s/reward:0.07058124803006649 - timing_s/old_log_prob:2.244998227804899 - timing_s/adv:0.5783395282924175 - timing_s/update_actor:15.651438953354955 - timing_s/step:25.914235873147845 - timing_s/testing:19.678959732875228 - timing_s/save_checkpoint:6.6768900994211435 - timing_s/stop_profile:0.00012764893472194672 - timing_per_token_ms/update_actor:0.09164840085816564 - timing_per_token_ms/gen:0.3792373041042865 - timing_per_token_ms/adv:0.0033865188420713417 - perf/total_num_tokens:170777 - perf/time_per_step:25.914235873147845 - perf/throughput:823.7605424483978 (TaskRunner pid=2204243) Training Progress: 50%|█████ | 50/100 [31:33<32:55, 39.50s/it] (TaskRunner pid=2204243) step:51 - global_seqlen/min:18667 - global_seqlen/max:30325 - global_seqlen/minmax_diff:11658 - global_seqlen/balanced_min:22731 - global_seqlen/balanced_max:22735 - global_seqlen/mean:22733.25 - actor/entropy:0.10003570467233658 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413557469844818 - training/rollout_probs_diff_mean:0.0036957848351448774 - training/rollout_probs_diff_std:0.014095666818320751 - training/rollout_actor_probs_pearson_corr:0.9962520599365234 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.34375 - self_distillation/correct_sample_fraction:0.30078125 - self_distillation/feedback_available_fraction:0.69921875 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0007126331329346 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.758541226387024 - rollout_corr/rollout_is_min:0.5677290558815002 - rollout_corr/rollout_is_std:0.03851236030459404 - rollout_corr/rollout_is_eff_sample_size:0.9985210152132482 - rollout_corr/rollout_is_seq_mean:1.0006033182144165 - rollout_corr/rollout_is_seq_std:0.004394259303808212 - rollout_corr/rollout_is_seq_max:1.0190714597702026 - rollout_corr/rollout_is_seq_min:0.988793134689331 - rollout_corr/rollout_is_seq_max_deviation:0.019071459770202637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1000283984467387) - rollout_corr/training_log_ppl:np.float64(0.09261283368687145) - rollout_corr/kl:np.float64(0.001417233828422404) - rollout_corr/k3_kl:np.float64(0.002070765299912576) - rollout_corr/rollout_ppl:np.float64(1.0984417693689466) - rollout_corr/rollout_log_ppl:np.float64(0.09119559949613176) - rollout_corr/log_ppl_diff:np.float64(0.0014172341907396913) - rollout_corr/log_ppl_abs_diff:np.float64(0.005220359220402315) - rollout_corr/log_ppl_diff_max:np.float64(0.046716831624507904) - rollout_corr/log_ppl_diff_min:np.float64(-0.029736503958702087) - rollout_corr/ppl_ratio:np.float64(1.0014550217892975) - rollout_corr/chi2_token:np.float64(0.0056394843850284815) - rollout_corr/chi2_seq:np.float64(0.44010692019946873) - actor/pg_loss:np.float64(0.0007794767152518034) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007243816380650969) - actor/ppo_kl:np.float64(0.0013004957165207998) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.30078125) - self_distillation/token_reweight_w_mean:np.float64(76629.52096982277) - self_distillation/token_reweight_w_std:np.float64(769435.0793446852) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002005817368627) - self_distillation/token_reweight_w_clip_frac:np.float64(0.010744694111053832) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.42179466038942337) - perf/mfu/actor:np.float64(0.04238834535845978) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.95610809326172) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.30078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005651434184983373 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0005651434184983373 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:80.3515625 - response_length/max:505.0 - response_length/min:35.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:80.3515625 - response_length_non_aborted/max:505.0 - response_length_non_aborted/min:35.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:630.0625 - prompt_length/max:1233.0 - prompt_length/min:226.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011558644473552704 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5104469377547503) - timing_s/agent_loop/generate_sequences/max:np.float64(3.10017566755414) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9291960202826886) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.10017566755414) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:332 - timing_s/agent_loop/slowest/response_length:505 - timing_s/gen:8.625935746356845 - timing_s/reward:0.0697182435542345 - timing_s/old_log_prob:2.395427107810974 - timing_s/adv:0.6585645042359829 - timing_s/update_actor:15.51681711524725 - timing_s/step:27.366128016263247 - timing_s/stop_profile:3.660842776298523e-05 - timing_per_token_ms/update_actor:0.08532005495940555 - timing_per_token_ms/gen:0.41934544221472264 - timing_per_token_ms/adv:0.0036211524102140197 - perf/total_num_tokens:181866 - perf/time_per_step:27.366128016263247 - perf/throughput:830.7075807907497 (TaskRunner pid=2204243) Training Progress: 51%|█████ | 51/100 [32:00<29:17, 35.87s/it] (TaskRunner pid=2204243) step:52 - global_seqlen/min:16841 - global_seqlen/max:24035 - global_seqlen/minmax_diff:7194 - global_seqlen/balanced_min:21012 - global_seqlen/balanced_max:21015 - global_seqlen/mean:21013.375 - actor/entropy:0.11270956695079803 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2957986295223236 - training/rollout_probs_diff_mean:0.004261509515345097 - training/rollout_probs_diff_std:0.014947227202355862 - training/rollout_actor_probs_pearson_corr:0.9962414503097534 - self_distillation/success_group_fraction:0.25 - self_distillation/success_sample_fraction:0.25 - self_distillation/correct_sample_fraction:0.25 - self_distillation/feedback_available_fraction:0.75 - self_distillation/feedback_used_fraction:0.75 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999476671218872 - rollout_corr/rollout_is_ratio_fraction_high:4.5169159420765936e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001355074782622978 - rollout_corr/rollout_is_max:2.2578909397125244 - rollout_corr/rollout_is_min:0.3513765037059784 - rollout_corr/rollout_is_std:0.04257437214255333 - rollout_corr/rollout_is_eff_sample_size:0.9981905834660311 - rollout_corr/rollout_is_seq_mean:0.9998025298118591 - rollout_corr/rollout_is_seq_std:0.0045767975971102715 - rollout_corr/rollout_is_seq_max:1.0130218267440796 - rollout_corr/rollout_is_seq_min:0.9801503419876099 - rollout_corr/rollout_is_seq_max_deviation:0.019849658012390137 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1176359057426453) - rollout_corr/training_log_ppl:np.float64(0.1078356985544815) - rollout_corr/kl:np.float64(0.0013219092068439409) - rollout_corr/k3_kl:np.float64(0.0011961325544049828) - rollout_corr/rollout_ppl:np.float64(1.1161169209517539) - rollout_corr/rollout_log_ppl:np.float64(0.10651378861621197) - rollout_corr/log_ppl_diff:np.float64(0.0013219099382695276) - rollout_corr/log_ppl_abs_diff:np.float64(0.004351939845946617) - rollout_corr/log_ppl_diff_max:np.float64(0.03677331656217575) - rollout_corr/log_ppl_diff_min:np.float64(-0.023302670568227768) - rollout_corr/ppl_ratio:np.float64(1.001341824652627) - rollout_corr/chi2_token:np.float64(0.0021924192551523447) - rollout_corr/chi2_seq:np.float64(0.15908730146475136) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0002422864043296613) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.25) - self_distillation/token_reweight_w_mean:np.float64(0.9999999932479113) - self_distillation/token_reweight_w_std:np.float64(0.0) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999999932479113) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.038616766650180254) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.0467071533203) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.25 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:86.48046875 - response_length/max:449.0 - response_length/min:38.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.48046875 - response_length_non_aborted/max:449.0 - response_length_non_aborted/min:38.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:570.1875 - prompt_length/max:1224.0 - prompt_length/min:256.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.120171070098877e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.41120051220059395) - timing_s/agent_loop/generate_sequences/max:np.float64(2.8605230636894703) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9114399397658417) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.8605230636894703) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:667 - timing_s/agent_loop/slowest/response_length:449 - timing_s/gen:8.681461775675416 - timing_s/reward:0.07274099439382553 - timing_s/old_log_prob:2.270609974861145 - timing_s/adv:0.5926898028701544 - timing_s/update_actor:15.549352053552866 - timing_s/step:27.25665437616408 - timing_s/stop_profile:0.00012969225645065308 - timing_per_token_ms/update_actor:0.09249675536148326 - timing_per_token_ms/gen:0.3921343229448221 - timing_per_token_ms/adv:0.003525669977277296 - perf/total_num_tokens:168107 - perf/time_per_step:27.25665437616408 - perf/throughput:770.9447649002799 (TaskRunner pid=2204243) Training Progress: 52%|█████▏ | 52/100 [32:28<26:38, 33.30s/it] (TaskRunner pid=2204243) step:53 - global_seqlen/min:17343 - global_seqlen/max:26152 - global_seqlen/minmax_diff:8809 - global_seqlen/balanced_min:20031 - global_seqlen/balanced_max:20034 - global_seqlen/mean:20032.125 - actor/entropy:0.08245602250099182 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19511407613754272 - training/rollout_probs_diff_mean:0.0031950678676366806 - training/rollout_probs_diff_std:0.012576941400766373 - training/rollout_actor_probs_pearson_corr:0.9965688586235046 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.4375 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.5859375 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000741481781006 - rollout_corr/rollout_is_ratio_fraction_high:5.08724624523893e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001526173873571679 - rollout_corr/rollout_is_max:2.5879404544830322 - rollout_corr/rollout_is_min:0.12148968875408173 - rollout_corr/rollout_is_std:0.03586125746369362 - rollout_corr/rollout_is_eff_sample_size:0.9987157408360959 - rollout_corr/rollout_is_seq_mean:1.0000253915786743 - rollout_corr/rollout_is_seq_std:0.004540097899734974 - rollout_corr/rollout_is_seq_max:1.0284918546676636 - rollout_corr/rollout_is_seq_min:0.9841364026069641 - rollout_corr/rollout_is_seq_max_deviation:0.028491854667663574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0846671643666923) - rollout_corr/training_log_ppl:np.float64(0.07857233920003637) - rollout_corr/kl:np.float64(0.0008492294873008177) - rollout_corr/k3_kl:np.float64(0.0011417816812362425) - rollout_corr/rollout_ppl:np.float64(1.0836734119802713) - rollout_corr/rollout_log_ppl:np.float64(0.07772310877408017) - rollout_corr/log_ppl_diff:np.float64(0.0008492304259561934) - rollout_corr/log_ppl_abs_diff:np.float64(0.004029584815725684) - rollout_corr/log_ppl_diff_max:np.float64(0.04583638906478882) - rollout_corr/log_ppl_diff_min:np.float64(-0.016166560351848602) - rollout_corr/ppl_ratio:np.float64(1.000867866910994) - rollout_corr/chi2_token:np.float64(0.002919555176049471) - rollout_corr/chi2_seq:np.float64(0.15523555362597108) - actor/pg_loss:np.float64(0.000842868466861546) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007537693963968195) - actor/ppo_kl:np.float64(0.0001411033300016129) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(208808.62719406863) - self_distillation/token_reweight_w_std:np.float64(1298977.9899588337) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000635378528386) - self_distillation/token_reweight_w_clip_frac:np.float64(0.011950318730669096) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.41743914037942886) - perf/mfu/actor:np.float64(0.037221966521849635) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.74567413330078) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034974818117916584 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034974818117916584 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:76.78515625 - response_length/max:220.0 - response_length/min:36.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:76.78515625 - response_length_non_aborted/max:220.0 - response_length_non_aborted/min:36.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:549.21875 - prompt_length/max:924.0 - prompt_length/min:244.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.202584624290466e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5270705576986074) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7523805722594261) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9445158711532713) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7523805722594261) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:573 - timing_s/agent_loop/slowest/response_length:220 - timing_s/gen:7.585309384390712 - timing_s/reward:0.07015729323029518 - timing_s/old_log_prob:2.2739233281463385 - timing_s/adv:0.5626175310462713 - timing_s/update_actor:15.378771299496293 - timing_s/step:25.957886477932334 - timing_s/stop_profile:0.00012243352830410004 - timing_per_token_ms/update_actor:0.09596317976435534 - timing_per_token_ms/gen:0.3858833689978487 - timing_per_token_ms/adv:0.003510720474277388 - perf/total_num_tokens:160257 - perf/time_per_step:25.957886477932334 - perf/throughput:771.7163343413948 (TaskRunner pid=2204243) Training Progress: 53%|█████▎ | 53/100 [32:54<24:22, 31.11s/it] (TaskRunner pid=2204243) step:54 - global_seqlen/min:16396 - global_seqlen/max:28110 - global_seqlen/minmax_diff:11714 - global_seqlen/balanced_min:22955 - global_seqlen/balanced_max:23930 - global_seqlen/mean:23079.875 - actor/entropy:0.08941730856895447 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105570673942566 - training/rollout_probs_diff_mean:0.0035335146822035313 - training/rollout_probs_diff_std:0.013440735638141632 - training/rollout_actor_probs_pearson_corr:0.9961168169975281 - self_distillation/success_group_fraction:0.5 - self_distillation/success_sample_fraction:0.49609375 - self_distillation/correct_sample_fraction:0.3984375 - self_distillation/feedback_available_fraction:0.6015625 - self_distillation/feedback_used_fraction:0.5 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.0001165866851807 - rollout_corr/rollout_is_ratio_fraction_high:4.2046838643727824e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.2046838643727824e-05 - rollout_corr/rollout_is_max:2.3978540897369385 - rollout_corr/rollout_is_min:0.45853883028030396 - rollout_corr/rollout_is_std:0.03660807013511658 - rollout_corr/rollout_is_eff_sample_size:0.9986619994387993 - rollout_corr/rollout_is_seq_mean:0.9999159574508667 - rollout_corr/rollout_is_seq_std:0.004038548097014427 - rollout_corr/rollout_is_seq_max:1.0239872932434082 - rollout_corr/rollout_is_seq_min:0.9883003830909729 - rollout_corr/rollout_is_seq_max_deviation:0.023987293243408203 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0861194231547415) - rollout_corr/training_log_ppl:np.float64(0.08049156603010488) - rollout_corr/kl:np.float64(0.001744110675147681) - rollout_corr/k3_kl:np.float64(0.0010414716397103008) - rollout_corr/rollout_ppl:np.float64(1.0841814293526113) - rollout_corr/rollout_log_ppl:np.float64(0.07874745477693068) - rollout_corr/log_ppl_diff:np.float64(0.0017441112531741965) - rollout_corr/log_ppl_abs_diff:np.float64(0.003491563295938249) - rollout_corr/log_ppl_diff_max:np.float64(0.022219013422727585) - rollout_corr/log_ppl_diff_min:np.float64(-0.008661642670631409) - rollout_corr/ppl_ratio:np.float64(1.0017570310737938) - rollout_corr/chi2_token:np.float64(0.0007168902084231377) - rollout_corr/chi2_seq:np.float64(0.04462135746143758) - actor/pg_loss:np.float64(0.0013959958450868726) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012390575413974148) - actor/ppo_kl:np.float64(0.0009985840585717831) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3984375) - self_distillation/token_reweight_w_mean:np.float64(594186.7411197773) - self_distillation/token_reweight_w_std:np.float64(4250993.840979477) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007701043505222) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02545608614309458) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.659911647439003) - perf/mfu/actor:np.float64(0.04265401847770788) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.92128372192383) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.3984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03787369281053543 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03787369281053543 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:92.90234375 - response_length/max:2244.0 - response_length/min:34.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:92.90234375 - response_length_non_aborted/max:2244.0 - response_length_non_aborted/min:34.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:628.34375 - prompt_length/max:1261.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011267140507698059 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4639764055609703) - timing_s/agent_loop/generate_sequences/max:np.float64(10.925770100206137) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0394952135538915) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.925770100206137) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:581 - timing_s/agent_loop/slowest/response_length:2244 - timing_s/gen:16.895026741549373 - timing_s/reward:0.07279234752058983 - timing_s/old_log_prob:2.38123189099133 - timing_s/adv:0.628468407317996 - timing_s/update_actor:15.728637207299471 - timing_s/step:35.79242232069373 - timing_s/stop_profile:0.00011715851724147797 - timing_per_token_ms/update_actor:0.08518588817800936 - timing_per_token_ms/gen:0.7103824892380849 - timing_per_token_ms/adv:0.0034037684742551466 - perf/total_num_tokens:184639 - perf/time_per_step:35.79242232069373 - perf/throughput:644.8257341514477 (TaskRunner pid=2204243) Training Progress: 54%|█████▍ | 54/100 [33:29<24:56, 32.53s/it] (TaskRunner pid=2204243) step:55 - global_seqlen/min:16143 - global_seqlen/max:24382 - global_seqlen/minmax_diff:8239 - global_seqlen/balanced_min:19716 - global_seqlen/balanced_max:22359 - global_seqlen/mean:20047.375 - actor/entropy:0.08437657356262207 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45647868514060974 - training/rollout_probs_diff_mean:0.0033371378667652607 - training/rollout_probs_diff_std:0.013518329709768295 - training/rollout_actor_probs_pearson_corr:0.9961479306221008 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.3125 - self_distillation/correct_sample_fraction:0.26953125 - self_distillation/feedback_available_fraction:0.73046875 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9996286034584045 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.089633047347888e-05 - rollout_corr/rollout_is_max:1.842090129852295 - rollout_corr/rollout_is_min:0.41273799538612366 - rollout_corr/rollout_is_std:0.03544498234987259 - rollout_corr/rollout_is_eff_sample_size:0.9987443973524001 - rollout_corr/rollout_is_seq_mean:0.9994367361068726 - rollout_corr/rollout_is_seq_std:0.0042431349866092205 - rollout_corr/rollout_is_seq_max:1.0120738744735718 - rollout_corr/rollout_is_seq_min:0.9865037798881531 - rollout_corr/rollout_is_seq_max_deviation:0.013496220111846924 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.0993408006615937) - rollout_corr/training_log_ppl:np.float64(0.09299615051168075) - rollout_corr/kl:np.float64(0.003286844284742685) - rollout_corr/k3_kl:np.float64(0.002238221012687447) - rollout_corr/rollout_ppl:np.float64(1.0957516594789922) - rollout_corr/rollout_log_ppl:np.float64(0.0897093057501479) - rollout_corr/log_ppl_diff:np.float64(0.003286844779722742) - rollout_corr/log_ppl_abs_diff:np.float64(0.006130468575065606) - rollout_corr/log_ppl_diff_max:np.float64(0.04959423094987869) - rollout_corr/log_ppl_diff_min:np.float64(-0.013323470950126648) - rollout_corr/ppl_ratio:np.float64(1.003330314066261) - rollout_corr/chi2_token:np.float64(0.002536744810640812) - rollout_corr/chi2_seq:np.float64(0.10854817647486925) - actor/pg_loss:np.float64(0.00029866164550185204) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006199879626365146) - actor/ppo_kl:np.float64(0.002030772887010812) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.26953125) - self_distillation/token_reweight_w_mean:np.float64(369197.9254463576) - self_distillation/token_reweight_w_std:np.float64(2408731.874646828) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004046217072755) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006205351659446023) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.37380824238061905) - perf/mfu/actor:np.float64(0.03673496007752844) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.87210178375244) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.26953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.26953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00028819317230954766 - critic/advantages/max:0.75 - critic/advantages/min:-0.375 - critic/returns/mean:0.00028819317230954766 - critic/returns/max:0.75 - critic/returns/min:-0.375 - response_length/mean:96.57421875 - response_length/max:3526.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:96.57421875 - response_length_non_aborted/max:3526.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:529.90625 - prompt_length/max:980.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010949932038784027 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5257433410733938) - timing_s/agent_loop/generate_sequences/max:np.float64(17.014353832229972) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0430916920304298) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.014353832229972) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:542 - timing_s/agent_loop/slowest/response_length:3526 - timing_s/gen:22.855054510757327 - timing_s/reward:0.06988451443612576 - timing_s/old_log_prob:2.3120052199810743 - timing_s/adv:0.5623201113194227 - timing_s/update_actor:15.721988135948777 - timing_s/step:41.611597852781415 - timing_s/stop_profile:0.00011841952800750732 - timing_per_token_ms/update_actor:0.09803021677369717 - timing_per_token_ms/gen:0.9244450313779609 - timing_per_token_ms/adv:0.00350619539540353 - perf/total_num_tokens:160379 - perf/time_per_step:41.611597852781415 - perf/throughput:481.77373699818133 (TaskRunner pid=2204243) Training Progress: 55%|█████▌ | 55/100 [34:11<26:27, 35.27s/it] (TaskRunner pid=2204243) step:56 - global_seqlen/min:18742 - global_seqlen/max:30237 - global_seqlen/minmax_diff:11495 - global_seqlen/balanced_min:23119 - global_seqlen/balanced_max:23120 - global_seqlen/mean:23119.875 - actor/entropy:0.11057615280151367 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163905024528503 - training/rollout_probs_diff_mean:0.0035216338001191616 - training/rollout_probs_diff_std:0.012724548578262329 - training/rollout_actor_probs_pearson_corr:0.9972032308578491 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.3125 - self_distillation/correct_sample_fraction:0.30859375 - self_distillation/feedback_available_fraction:0.69140625 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999288320541382 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.350382267963141e-05 - rollout_corr/rollout_is_max:1.940208911895752 - rollout_corr/rollout_is_min:0.44323068857192993 - rollout_corr/rollout_is_std:0.03610971197485924 - rollout_corr/rollout_is_eff_sample_size:0.998697548963349 - rollout_corr/rollout_is_seq_mean:0.9998021125793457 - rollout_corr/rollout_is_seq_std:0.0037325036246329546 - rollout_corr/rollout_is_seq_max:1.0156078338623047 - rollout_corr/rollout_is_seq_min:0.9880214333534241 - rollout_corr/rollout_is_seq_max_deviation:0.015607833862304688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1046981760300696) - rollout_corr/training_log_ppl:np.float64(0.09688017570806551) - rollout_corr/kl:np.float64(0.0014025520569020955) - rollout_corr/k3_kl:np.float64(0.0010570474939601837) - rollout_corr/rollout_ppl:np.float64(1.1031573805958033) - rollout_corr/rollout_log_ppl:np.float64(0.0954776229291383) - rollout_corr/log_ppl_diff:np.float64(0.0014025527789272019) - rollout_corr/log_ppl_abs_diff:np.float64(0.004000807298325526) - rollout_corr/log_ppl_diff_max:np.float64(0.043913260102272034) - rollout_corr/log_ppl_diff_min:np.float64(-0.019757740199565887) - rollout_corr/ppl_ratio:np.float64(1.0014231519307941) - rollout_corr/chi2_token:np.float64(0.001416476909071207) - rollout_corr/chi2_seq:np.float64(0.21909841243177652) - actor/pg_loss:np.float64(0.00016872480046004057) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(7.931676373118535e-05) - actor/ppo_kl:np.float64(0.0005879578359326842) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.30859375) - self_distillation/token_reweight_w_mean:np.float64(14813.701483787503) - self_distillation/token_reweight_w_std:np.float64(164370.77270257816) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999380588997155) - self_distillation/token_reweight_w_clip_frac:np.float64(0.002431772925774567) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.07515872456133366) - perf/mfu/actor:np.float64(0.04308993241234514) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.98887252807617) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.30859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015813536010682583 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015813536010682583 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:93.55859375 - response_length/max:302.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:93.55859375 - response_length_non_aborted/max:302.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:628.9375 - prompt_length/max:1219.0 - prompt_length/min:254.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010671280324459076 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5345313623547554) - timing_s/agent_loop/generate_sequences/max:np.float64(2.253320286050439) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0835376217437442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.253320286050439) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:815 - timing_s/agent_loop/slowest/response_length:275 - timing_s/gen:8.067085301503539 - timing_s/reward:0.07412604615092278 - timing_s/old_log_prob:2.4735532235354185 - timing_s/adv:0.6355291809886694 - timing_s/update_actor:15.284198155626655 - timing_s/step:26.621995747089386 - timing_s/stop_profile:0.00011903978884220123 - timing_per_token_ms/update_actor:0.08263560116364521 - timing_per_token_ms/gen:0.3368162206798689 - timing_per_token_ms/adv:0.0034360543741514035 - perf/total_num_tokens:184959 - perf/time_per_step:26.621995747089386 - perf/throughput:868.4501049297825 (TaskRunner pid=2204243) Training Progress: 56%|█████▌ | 56/100 [34:38<23:58, 32.69s/it] (TaskRunner pid=2204243) step:57 - global_seqlen/min:21309 - global_seqlen/max:25155 - global_seqlen/minmax_diff:3846 - global_seqlen/balanced_min:22713 - global_seqlen/balanced_max:22715 - global_seqlen/mean:22714.5 - actor/entropy:0.10051295906305313 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016381561756134 - training/rollout_probs_diff_mean:0.003623584285378456 - training/rollout_probs_diff_std:0.013785067945718765 - training/rollout_actor_probs_pearson_corr:0.9963996410369873 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.34375 - self_distillation/correct_sample_fraction:0.30859375 - self_distillation/feedback_available_fraction:0.69140625 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0001078844070435 - rollout_corr/rollout_is_ratio_fraction_high:4.4762757170246914e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.0926899909973145 - rollout_corr/rollout_is_min:0.5154091119766235 - rollout_corr/rollout_is_std:0.03944816812872887 - rollout_corr/rollout_is_eff_sample_size:0.9984466163170869 - rollout_corr/rollout_is_seq_mean:1.0000673532485962 - rollout_corr/rollout_is_seq_std:0.004448513966053724 - rollout_corr/rollout_is_seq_max:1.01644766330719 - rollout_corr/rollout_is_seq_min:0.9889145493507385 - rollout_corr/rollout_is_seq_max_deviation:0.01644766330718994 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1085894485004246) - rollout_corr/training_log_ppl:np.float64(0.10126656437205384) - rollout_corr/kl:np.float64(0.0008975799961685738) - rollout_corr/k3_kl:np.float64(0.0009016328989197575) - rollout_corr/rollout_ppl:np.float64(1.1076103881932795) - rollout_corr/rollout_log_ppl:np.float64(0.10036898346515954) - rollout_corr/log_ppl_diff:np.float64(0.0008975809068942908) - rollout_corr/log_ppl_abs_diff:np.float64(0.003569827491446631) - rollout_corr/log_ppl_diff_max:np.float64(0.03016757220029831) - rollout_corr/log_ppl_diff_min:np.float64(-0.011988222599029541) - rollout_corr/ppl_ratio:np.float64(1.0009092008695006) - rollout_corr/chi2_token:np.float64(0.001822149846702814) - rollout_corr/chi2_seq:np.float64(0.22556268447078764) - actor/pg_loss:np.float64(0.0007208599708974361) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022642006661044434) - actor/ppo_kl:np.float64(0.000185914718059621) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.30859375) - self_distillation/token_reweight_w_mean:np.float64(190896.0683354116) - self_distillation/token_reweight_w_std:np.float64(1571404.9968573537) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001445405650884) - self_distillation/token_reweight_w_clip_frac:np.float64(0.00989003761060303) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.25426503270864487) - perf/mfu/actor:np.float64(0.04166996744512609) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.83833694458008) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.30859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0044650849886238575 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0044650849886238575 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:87.265625 - response_length/max:368.0 - response_length/min:45.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:87.265625 - response_length_non_aborted/max:368.0 - response_length_non_aborted/min:45.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:622.5625 - prompt_length/max:1228.0 - prompt_length/min:351.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001737792044878006 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6398835349828005) - timing_s/agent_loop/generate_sequences/max:np.float64(2.4508363008499146) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0763657611532835) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.4508363008499146) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:658 - timing_s/agent_loop/slowest/response_length:368 - timing_s/gen:8.058409651741385 - timing_s/reward:0.07317380420863628 - timing_s/old_log_prob:2.331144340336323 - timing_s/adv:0.6200122404843569 - timing_s/update_actor:15.68521218560636 - timing_s/step:26.854476736858487 - timing_s/stop_profile:0.00012129917740821838 - timing_per_token_ms/update_actor:0.08631717727446322 - timing_per_token_ms/gen:0.36071663615673166 - timing_per_token_ms/adv:0.0034119848581542454 - perf/total_num_tokens:181716 - perf/time_per_step:26.854476736858487 - perf/throughput:845.8366261452319 (TaskRunner pid=2204243) Training Progress: 57%|█████▋ | 57/100 [35:05<22:11, 30.97s/it] (TaskRunner pid=2204243) step:58 - global_seqlen/min:15349 - global_seqlen/max:29133 - global_seqlen/minmax_diff:13784 - global_seqlen/balanced_min:21181 - global_seqlen/balanced_max:21187 - global_seqlen/mean:21183.25 - actor/entropy:0.10371923446655273 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21512344479560852 - training/rollout_probs_diff_mean:0.003614909714087844 - training/rollout_probs_diff_std:0.013131417334079742 - training/rollout_actor_probs_pearson_corr:0.9967510104179382 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.4375 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.5859375 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9997174143791199 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7097834348678589 - rollout_corr/rollout_is_min:0.50865238904953 - rollout_corr/rollout_is_std:0.034970108419656754 - rollout_corr/rollout_is_eff_sample_size:0.9987778122136148 - rollout_corr/rollout_is_seq_mean:0.9996785521507263 - rollout_corr/rollout_is_seq_std:0.0039123836904764175 - rollout_corr/rollout_is_seq_max:1.0112284421920776 - rollout_corr/rollout_is_seq_min:0.9894210696220398 - rollout_corr/rollout_is_seq_max_deviation:0.011228442192077637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1056648031808436) - rollout_corr/training_log_ppl:np.float64(0.09856435639812844) - rollout_corr/kl:np.float64(0.0014258211778184204) - rollout_corr/k3_kl:np.float64(0.0013744421184540556) - rollout_corr/rollout_ppl:np.float64(1.1039966172538698) - rollout_corr/rollout_log_ppl:np.float64(0.09713853454741184) - rollout_corr/log_ppl_diff:np.float64(0.001425821850716602) - rollout_corr/log_ppl_abs_diff:np.float64(0.004259307308529969) - rollout_corr/log_ppl_diff_max:np.float64(0.044669464230537415) - rollout_corr/log_ppl_diff_min:np.float64(-0.02541404776275158) - rollout_corr/ppl_ratio:np.float64(1.0014505225699395) - rollout_corr/chi2_token:np.float64(0.002054498763754964) - rollout_corr/chi2_seq:np.float64(0.23966963402926922) - actor/pg_loss:np.float64(0.000830475240945816) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000853122506669024) - actor/ppo_kl:np.float64(0.0004935391127816047) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(102965.48446710757) - self_distillation/token_reweight_w_std:np.float64(736743.0680832942) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995958062354475) - self_distillation/token_reweight_w_clip_frac:np.float64(0.009628775995224714) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.33479520305991173) - perf/mfu/actor:np.float64(0.03901556139253888) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.70441436767578) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034388916101306677 - critic/advantages/max:0.5 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0034388916101306677 - critic/returns/max:0.5 - critic/returns/min:-0.75 - response_length/mean:78.9453125 - response_length/max:168.0 - response_length/min:39.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:78.9453125 - response_length_non_aborted/max:168.0 - response_length_non_aborted/min:39.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:583.03125 - prompt_length/max:1263.0 - prompt_length/min:256.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014735758304595947 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5729048009961843) - timing_s/agent_loop/generate_sequences/max:np.float64(1.5363880097866058) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9929611482293694) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.5363880097866058) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1263 - timing_s/agent_loop/slowest/response_length:168 - timing_s/gen:7.412254769355059 - timing_s/reward:0.06059943325817585 - timing_s/old_log_prob:2.2101074289530516 - timing_s/adv:0.6132335178554058 - timing_s/update_actor:15.665855368599296 - timing_s/step:25.99549888446927 - timing_s/stop_profile:5.347840487957001e-05 - timing_per_token_ms/update_actor:0.09244246851049352 - timing_per_token_ms/gen:0.3667617401956981 - timing_per_token_ms/adv:0.0036186227199285154 - perf/total_num_tokens:169466 - perf/time_per_step:25.99549888446927 - perf/throughput:814.8814567531036 (TaskRunner pid=2204243) Training Progress: 58%|█████▊ | 58/100 [35:31<20:38, 29.49s/it] (TaskRunner pid=2204243) step:59 - global_seqlen/min:15417 - global_seqlen/max:24124 - global_seqlen/minmax_diff:8707 - global_seqlen/balanced_min:21085 - global_seqlen/balanced_max:21089 - global_seqlen/mean:21087.5 - actor/entropy:0.1323874443769455 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3147658407688141 - training/rollout_probs_diff_mean:0.004064080771058798 - training/rollout_probs_diff_std:0.013299784623086452 - training/rollout_actor_probs_pearson_corr:0.997320294380188 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.43359375 - self_distillation/correct_sample_fraction:0.37890625 - self_distillation/feedback_available_fraction:0.62109375 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9994232058525085 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.531448212219402e-05 - rollout_corr/rollout_is_max:1.6135103702545166 - rollout_corr/rollout_is_min:0.4973900616168976 - rollout_corr/rollout_is_std:0.036343347281217575 - rollout_corr/rollout_is_eff_sample_size:0.9986792388587119 - rollout_corr/rollout_is_seq_mean:0.9995426535606384 - rollout_corr/rollout_is_seq_std:0.003795828204602003 - rollout_corr/rollout_is_seq_max:1.00856351852417 - rollout_corr/rollout_is_seq_min:0.9863641262054443 - rollout_corr/rollout_is_seq_max_deviation:0.013635873794555664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1360691818408668) - rollout_corr/training_log_ppl:np.float64(0.12421963129781943) - rollout_corr/kl:np.float64(0.0014597514572329828) - rollout_corr/k3_kl:np.float64(0.0009712452897899793) - rollout_corr/rollout_ppl:np.float64(1.1343398070894182) - rollout_corr/rollout_log_ppl:np.float64(0.12275987809789513) - rollout_corr/log_ppl_diff:np.float64(0.0014597531999243074) - rollout_corr/log_ppl_abs_diff:np.float64(0.004079278282915766) - rollout_corr/log_ppl_diff_max:np.float64(0.028037577867507935) - rollout_corr/log_ppl_diff_min:np.float64(-0.010827913880348206) - rollout_corr/ppl_ratio:np.float64(1.001476097619161) - rollout_corr/chi2_token:np.float64(0.0008270400576293468) - rollout_corr/chi2_seq:np.float64(0.10522137978114188) - actor/pg_loss:np.float64(0.0007836048025637865) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(6.734913768013939e-05) - actor/ppo_kl:np.float64(0.00035485127421708285) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.37890625) - self_distillation/token_reweight_w_mean:np.float64(171313.1193060847) - self_distillation/token_reweight_w_std:np.float64(1310334.1709769317) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000733888708055) - self_distillation/token_reweight_w_clip_frac:np.float64(0.013617925156722777) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.7427506223320961) - perf/mfu/actor:np.float64(0.03926540640048928) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.79670333862305) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.37890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.37890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0030757205095142126 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0030757205095142126 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:86.203125 - response_length/max:265.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.203125 - response_length_non_aborted/max:265.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:572.78125 - prompt_length/max:1271.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.734261453151703e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.361585883423686) - timing_s/agent_loop/generate_sequences/max:np.float64(1.7599963750690222) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.9170911096734926) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.7599963750690222) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:439 - timing_s/agent_loop/slowest/response_length:198 - timing_s/gen:7.732312185689807 - timing_s/reward:0.07021494396030903 - timing_s/old_log_prob:2.25281467102468 - timing_s/adv:0.6098823770880699 - timing_s/update_actor:15.3917680028826 - timing_s/step:26.144965978339314 - timing_s/stop_profile:0.00012209452688694 - timing_per_token_ms/update_actor:0.09123751039053112 - timing_per_token_ms/gen:0.3503857252895508 - timing_per_token_ms/adv:0.00361518895724997 - perf/total_num_tokens:168700 - perf/time_per_step:26.144965978339314 - perf/throughput:806.5606211716113 (TaskRunner pid=2204243) Training Progress: 59%|█████▉ | 59/100 [35:57<19:28, 28.50s/it] (TaskRunner pid=2204243) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2204243) validation generation end (TaskRunner pid=2204243) [prompt] user (TaskRunner pid=2204243) Your task is to answer the user's question using available tools. (TaskRunner pid=2204243) You have access to the following tools: (TaskRunner pid=2204243) Name: Axolotl (TaskRunner pid=2204243) Description: Collection of axolotl pictures and facts (TaskRunner pid=2204243) Documentation: (TaskRunner pid=2204243) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2204243) Parameters: {} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{url, source, description} (TaskRunner pid=2204243) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2204243) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2204243) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2204243) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Array[Object{fact, source}] (TaskRunner pid=2204243) (TaskRunner pid=2204243) Use the following format: (TaskRunner pid=2204243) Thought: you should always think about what to do (TaskRunner pid=2204243) Action: the action to take, should be one of the tool names. (TaskRunner pid=2204243) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2204243) (TaskRunner pid=2204243) Begin! (TaskRunner pid=2204243) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2204243) assistant (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: I need to find a random axolotl image. The user hasn't specified any particular criteria, so I should use the getRandomAxolotlImage tool. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2204655) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2204243) step:60 - global_seqlen/min:16803 - global_seqlen/max:25459 - global_seqlen/minmax_diff:8656 - global_seqlen/balanced_min:21424 - global_seqlen/balanced_max:21430 - global_seqlen/mean:21427.875 - actor/entropy:0.11457356810569763 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977370262146 - training/rollout_probs_diff_mean:0.0038266906049102545 - training/rollout_probs_diff_std:0.01311736460775137 - training/rollout_actor_probs_pearson_corr:0.9971461892127991 - self_distillation/success_group_fraction:0.46875 - self_distillation/success_sample_fraction:0.46875 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.578125 - self_distillation/feedback_used_fraction:0.53125 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9997825026512146 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00014035087951924652 - rollout_corr/rollout_is_max:1.9132329225540161 - rollout_corr/rollout_is_min:0.167610764503479 - rollout_corr/rollout_is_std:0.037565141916275024 - rollout_corr/rollout_is_eff_sample_size:0.9985903136141516 - rollout_corr/rollout_is_seq_mean:0.9996955990791321 - rollout_corr/rollout_is_seq_std:0.00393352285027504 - rollout_corr/rollout_is_seq_max:1.0112698078155518 - rollout_corr/rollout_is_seq_min:0.9865787029266357 - rollout_corr/rollout_is_seq_max_deviation:0.013421297073364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1216912264935672) - rollout_corr/training_log_ppl:np.float64(0.11247390051721595) - rollout_corr/kl:np.float64(0.0012605203389510766) - rollout_corr/k3_kl:np.float64(0.00102121454250792) - rollout_corr/rollout_ppl:np.float64(1.120270253624767) - rollout_corr/rollout_log_ppl:np.float64(0.11121337938675424) - rollout_corr/log_ppl_diff:np.float64(0.0012605211304617114) - rollout_corr/log_ppl_abs_diff:np.float64(0.0040084450811264105) - rollout_corr/log_ppl_diff_max:np.float64(0.01973031461238861) - rollout_corr/log_ppl_diff_min:np.float64(-0.014343395829200745) - rollout_corr/ppl_ratio:np.float64(1.001275957096368) - rollout_corr/chi2_token:np.float64(0.0014870327431708574) - rollout_corr/chi2_seq:np.float64(0.12056928570382297) - actor/pg_loss:np.float64(0.0012248143320903182) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024261497674160637) - actor/ppo_kl:np.float64(0.00022513904845489208) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(269118.3681596515) - self_distillation/token_reweight_w_std:np.float64(2210808.522484349) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998694392852485) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020179529674351215) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.4099989980459213) - perf/mfu/actor:np.float64(0.03995723938708147) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.7181625366211) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.6102941176470589) - val-aux/tooluse/reward/std@16:np.float64(0.08465206310162904) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6442205882352942) - val-aux/tooluse/reward/best@2/std:np.float64(0.0645000387411443) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5765147058823529) - val-aux/tooluse/reward/worst@2/std:np.float64(0.0790814081159696) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.6105735294117648) - val-aux/tooluse/reward/maj@2/std:np.float64(0.08374069098239675) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6669705882352941) - val-aux/tooluse/reward/best@4/std:np.float64(0.043758007819239005) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.543485294117647) - val-aux/tooluse/reward/worst@4/std:np.float64(0.06631034594826848) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.6194558823529411) - val-aux/tooluse/reward/maj@4/std:np.float64(0.06345137373730864) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6833676470588236) - val-aux/tooluse/reward/best@8/std:np.float64(0.02920175801262831) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5157941176470588) - val-aux/tooluse/reward/worst@8/std:np.float64(0.04725929692466077) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.6261176470588237) - val-aux/tooluse/reward/maj@8/std:np.float64(0.04082545607991657) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6961029411764705) - val-aux/tooluse/reward/best@16/std:np.float64(0.02057139059378335) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.498485294117647) - val-aux/tooluse/reward/worst@16/std:np.float64(0.024637893547394887) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.6295882352941176) - val-aux/tooluse/reward/maj@16/std:np.float64(0.021155186336700844) - val-aux/tooluse/score/mean@16:np.float64(0.6102941176470589) - val-aux/tooluse/score/std@16:np.float64(0.08465206310162904) - val-aux/tooluse/score/best@2/mean:np.float64(0.6442205882352942) - val-aux/tooluse/score/best@2/std:np.float64(0.0645000387411443) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5765147058823529) - val-aux/tooluse/score/worst@2/std:np.float64(0.0790814081159696) - val-aux/tooluse/score/maj@2/mean:np.float64(0.6105735294117648) - val-aux/tooluse/score/maj@2/std:np.float64(0.08374069098239675) - val-aux/tooluse/score/best@4/mean:np.float64(0.6669705882352941) - val-aux/tooluse/score/best@4/std:np.float64(0.043758007819239005) - val-aux/tooluse/score/worst@4/mean:np.float64(0.543485294117647) - val-aux/tooluse/score/worst@4/std:np.float64(0.06631034594826848) - val-aux/tooluse/score/maj@4/mean:np.float64(0.6194558823529411) - val-aux/tooluse/score/maj@4/std:np.float64(0.06345137373730864) - val-aux/tooluse/score/best@8/mean:np.float64(0.6833676470588236) - val-aux/tooluse/score/best@8/std:np.float64(0.02920175801262831) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5157941176470588) - val-aux/tooluse/score/worst@8/std:np.float64(0.04725929692466077) - val-aux/tooluse/score/maj@8/mean:np.float64(0.6261176470588237) - val-aux/tooluse/score/maj@8/std:np.float64(0.04082545607991657) - val-aux/tooluse/score/best@16/mean:np.float64(0.6961029411764705) - val-aux/tooluse/score/best@16/std:np.float64(0.02057139059378335) - val-aux/tooluse/score/worst@16/mean:np.float64(0.498485294117647) - val-aux/tooluse/score/worst@16/std:np.float64(0.024637893547394887) - val-aux/tooluse/score/maj@16/mean:np.float64(0.6295882352941176) - val-aux/tooluse/score/maj@16/std:np.float64(0.021155186336700844) - val-core/tooluse/acc/mean@16:np.float64(0.6102941176470589) - val-aux/tooluse/acc/std@16:np.float64(0.08465206310162904) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6442205882352942) - val-aux/tooluse/acc/best@2/std:np.float64(0.0645000387411443) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5765147058823529) - val-aux/tooluse/acc/worst@2/std:np.float64(0.0790814081159696) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.6105735294117648) - val-aux/tooluse/acc/maj@2/std:np.float64(0.08374069098239675) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6669705882352941) - val-aux/tooluse/acc/best@4/std:np.float64(0.043758007819239005) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.543485294117647) - val-aux/tooluse/acc/worst@4/std:np.float64(0.06631034594826848) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.6194558823529411) - val-aux/tooluse/acc/maj@4/std:np.float64(0.06345137373730864) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6833676470588236) - val-aux/tooluse/acc/best@8/std:np.float64(0.02920175801262831) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5157941176470588) - val-aux/tooluse/acc/worst@8/std:np.float64(0.04725929692466077) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.6261176470588237) - val-aux/tooluse/acc/maj@8/std:np.float64(0.04082545607991657) - val-core/tooluse/acc/best@16/mean:np.float64(0.6961029411764705) - val-core/tooluse/acc/best@16/std:np.float64(0.02057139059378335) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.498485294117647) - val-aux/tooluse/acc/worst@16/std:np.float64(0.024637893547394887) - val-core/tooluse/acc/maj@16/mean:np.float64(0.6295882352941176) - val-core/tooluse/acc/maj@16/std:np.float64(0.021155186336700844) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003058479633182287 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003058479633182287 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:83.49609375 - response_length/max:231.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:83.49609375 - response_length_non_aborted/max:231.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:586.125 - prompt_length/max:990.0 - prompt_length/min:300.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.294436752796173e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.49717358872294426) - timing_s/agent_loop/generate_sequences/max:np.float64(1.9063139501959085) - timing_s/agent_loop/generate_sequences/mean:np.float64(0.955458436968911) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1.9063139501959085) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:351 - timing_s/agent_loop/slowest/response_length:231 - timing_s/gen:7.886375110596418 - timing_s/reward:0.06999804265797138 - timing_s/old_log_prob:2.4437901563942432 - timing_s/adv:0.5851743035018444 - timing_s/update_actor:15.340492064133286 - timing_s/step:26.414382552728057 - timing_s/testing:20.233499640598893 - timing_s/save_checkpoint:6.8538718819618225 - timing_s/stop_profile:0.00013783201575279236 - timing_per_token_ms/update_actor:0.08948911210358754 - timing_per_token_ms/gen:0.36895322154837046 - timing_per_token_ms/adv:0.003413627713328109 - perf/total_num_tokens:171423 - perf/time_per_step:26.414382552728057 - perf/throughput:811.2199843106666 (TaskRunner pid=2204243) Training Progress: 60%|██████ | 60/100 [36:50<24:00, 36.01s/it] (TaskRunner pid=2204243) step:61 - global_seqlen/min:18748 - global_seqlen/max:29832 - global_seqlen/minmax_diff:11084 - global_seqlen/balanced_min:22132 - global_seqlen/balanced_max:22156 - global_seqlen/mean:22148.625 - actor/entropy:0.17683996260166168 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1846364438533783 - training/rollout_probs_diff_mean:0.004533935803920031 - training/rollout_probs_diff_std:0.013547694310545921 - training/rollout_actor_probs_pearson_corr:0.9977143406867981 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.3359375 - self_distillation/correct_sample_fraction:0.2890625 - self_distillation/feedback_available_fraction:0.7109375 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:1.0001479387283325 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8288978338241577 - rollout_corr/rollout_is_min:0.5262149572372437 - rollout_corr/rollout_is_std:0.03821249306201935 - rollout_corr/rollout_is_eff_sample_size:0.9985424099195641 - rollout_corr/rollout_is_seq_mean:1.000368595123291 - rollout_corr/rollout_is_seq_std:0.0043691121973097324 - rollout_corr/rollout_is_seq_max:1.0186423063278198 - rollout_corr/rollout_is_seq_min:0.9839122295379639 - rollout_corr/rollout_is_seq_max_deviation:0.018642306327819824 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.170066851656884) - rollout_corr/training_log_ppl:np.float64(0.14713505309191532) - rollout_corr/kl:np.float64(0.0009053573143491533) - rollout_corr/k3_kl:np.float64(0.0008859551338389338) - rollout_corr/rollout_ppl:np.float64(1.1688813669607043) - rollout_corr/rollout_log_ppl:np.float64(0.14622969441552414) - rollout_corr/log_ppl_diff:np.float64(0.0009053586763911881) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037377206826931797) - rollout_corr/log_ppl_diff_max:np.float64(0.024098269641399384) - rollout_corr/log_ppl_diff_min:np.float64(-0.019215740263462067) - rollout_corr/ppl_ratio:np.float64(1.000918622361496) - rollout_corr/chi2_token:np.float64(0.001646691933274269) - rollout_corr/chi2_seq:np.float64(0.21492237015627325) - actor/pg_loss:np.float64(0.0003205802058801055) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003644490607257467) - actor/ppo_kl:np.float64(0.0005418468742490745) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.2890625) - self_distillation/token_reweight_w_mean:np.float64(169581.68458764977) - self_distillation/token_reweight_w_std:np.float64(1377475.0991191864) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010959387291223) - self_distillation/token_reweight_w_clip_frac:np.float64(0.009479727392317727) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.21386374160647392) - perf/mfu/actor:np.float64(0.04122693972449444) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.8953399658203) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:0 - critic/score/mean:0.2890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014827304985374212 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:0.0014827304985374212 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:100.76953125 - response_length/max:498.0 - response_length/min:38.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:100.76953125 - response_length_non_aborted/max:498.0 - response_length_non_aborted/min:38.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:591.375 - prompt_length/max:1283.0 - prompt_length/min:247.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.378790855407715e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5083952154964209) - timing_s/agent_loop/generate_sequences/max:np.float64(3.26853902079165) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.112087765162869) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.26853902079165) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1283 - timing_s/agent_loop/slowest/response_length:493 - timing_s/gen:8.953279837965965 - timing_s/reward:0.06862030178308487 - timing_s/old_log_prob:2.372952690348029 - timing_s/adv:0.6411861088126898 - timing_s/update_actor:15.339752214029431 - timing_s/step:27.475288659334183 - timing_s/stop_profile:0.00011019781231880188 - timing_per_token_ms/update_actor:0.08657282457731254 - timing_per_token_ms/gen:0.34706670690258423 - timing_per_token_ms/adv:0.003618656399735253 - perf/total_num_tokens:177189 - perf/time_per_step:27.475288659334183 - perf/throughput:806.1289282387737 (TaskRunner pid=2204243) Training Progress: 61%|██████ | 61/100 [37:18<21:45, 33.46s/it] (TaskRunner pid=2204243) step:62 - global_seqlen/min:20402 - global_seqlen/max:27707 - global_seqlen/minmax_diff:7305 - global_seqlen/balanced_min:23467 - global_seqlen/balanced_max:23475 - global_seqlen/mean:23471.5 - actor/entropy:0.15909716486930847 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2498505711555481 - training/rollout_probs_diff_mean:0.004238549154251814 - training/rollout_probs_diff_std:0.013025213964283466 - training/rollout_actor_probs_pearson_corr:0.9977805614471436 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58203125 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.5078125 - self_distillation/feedback_used_fraction:0.40625 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:0.9997345209121704 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.957934856414795 - rollout_corr/rollout_is_min:0.536312460899353 - rollout_corr/rollout_is_std:0.037953462451696396 - rollout_corr/rollout_is_eff_sample_size:0.9985608338735662 - rollout_corr/rollout_is_seq_mean:0.9997060298919678 - rollout_corr/rollout_is_seq_std:0.004011985845863819 - rollout_corr/rollout_is_seq_max:1.0190627574920654 - rollout_corr/rollout_is_seq_min:0.9886996150016785 - rollout_corr/rollout_is_seq_max_deviation:0.01906275749206543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1599841760471463) - rollout_corr/training_log_ppl:np.float64(0.14427279867959442) - rollout_corr/kl:np.float64(0.0009269902941468899) - rollout_corr/k3_kl:np.float64(0.0007238646723156661) - rollout_corr/rollout_ppl:np.float64(1.1589470193721354) - rollout_corr/rollout_log_ppl:np.float64(0.14334580661306973) - rollout_corr/log_ppl_diff:np.float64(0.00092699206652469) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032036463562690187) - rollout_corr/log_ppl_diff_max:np.float64(0.01514461636543274) - rollout_corr/log_ppl_diff_min:np.float64(-0.017682358622550964) - rollout_corr/ppl_ratio:np.float64(1.000936521915719) - rollout_corr/chi2_token:np.float64(0.0010468875989317894) - rollout_corr/chi2_seq:np.float64(0.2756866791751236) - actor/pg_loss:np.float64(0.001126691116951406) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005847402617291664) - actor/ppo_kl:np.float64(-6.567377641886196e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(414431.05384494225) - self_distillation/token_reweight_w_std:np.float64(3463046.3314629286) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001482950290665) - self_distillation/token_reweight_w_clip_frac:np.float64(0.025261769915232435) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.42992889508605003) - perf/mfu/actor:np.float64(0.04343468186702431) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.83296966552734) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006008652504533529 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006008652504533529 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:97.515625 - response_length/max:411.0 - response_length/min:38.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:97.515625 - response_length_non_aborted/max:411.0 - response_length_non_aborted/min:38.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:635.96875 - prompt_length/max:1125.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001500975340604782 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5518545098602772) - timing_s/agent_loop/generate_sequences/max:np.float64(2.8701058216392994) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1486372877043323) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.8701058216392994) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:457 - timing_s/agent_loop/slowest/response_length:411 - timing_s/gen:8.748721081763506 - timing_s/reward:0.07462143152952194 - timing_s/old_log_prob:2.3898277021944523 - timing_s/adv:0.6286297962069511 - timing_s/update_actor:15.471786139532924 - timing_s/step:27.40148391202092 - timing_s/stop_profile:0.00011436082422733307 - timing_per_token_ms/update_actor:0.08239666265222144 - timing_per_token_ms/gen:0.35045349630521977 - timing_per_token_ms/adv:0.003347835652850005 - perf/total_num_tokens:187772 - perf/time_per_step:27.40148391202092 - perf/throughput:856.5776975933462 (TaskRunner pid=2204243) Training Progress: 62%|██████▏ | 62/100 [37:45<20:03, 31.66s/it] (TaskRunner pid=2204243) step:63 - global_seqlen/min:19242 - global_seqlen/max:26121 - global_seqlen/minmax_diff:6879 - global_seqlen/balanced_min:22042 - global_seqlen/balanced_max:22044 - global_seqlen/mean:22042.625 - actor/entropy:0.16857299208641052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1849110722541809 - training/rollout_probs_diff_mean:0.004358360078185797 - training/rollout_probs_diff_std:0.013025011867284775 - training/rollout_actor_probs_pearson_corr:0.9979040026664734 - self_distillation/success_group_fraction:0.46875 - self_distillation/success_sample_fraction:0.46875 - self_distillation/correct_sample_fraction:0.4296875 - self_distillation/feedback_available_fraction:0.5703125 - self_distillation/feedback_used_fraction:0.53125 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0002260208129883 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.37426169810351e-05 - rollout_corr/rollout_is_max:1.8803716897964478 - rollout_corr/rollout_is_min:0.47978681325912476 - rollout_corr/rollout_is_std:0.0380655862390995 - rollout_corr/rollout_is_eff_sample_size:0.9985535830750532 - rollout_corr/rollout_is_seq_mean:1.0001977682113647 - rollout_corr/rollout_is_seq_std:0.004199532326310873 - rollout_corr/rollout_is_seq_max:1.0182735919952393 - rollout_corr/rollout_is_seq_min:0.9848820567131042 - rollout_corr/rollout_is_seq_max_deviation:0.018273591995239258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1631165510043502) - rollout_corr/training_log_ppl:np.float64(0.14702915714769915) - rollout_corr/kl:np.float64(0.0006015916769328555) - rollout_corr/k3_kl:np.float64(0.0011502239240958056) - rollout_corr/rollout_ppl:np.float64(1.1624473198316991) - rollout_corr/rollout_log_ppl:np.float64(0.14642756380453648) - rollout_corr/log_ppl_diff:np.float64(0.0006015933431626763) - rollout_corr/log_ppl_abs_diff:np.float64(0.003732067798409844) - rollout_corr/log_ppl_diff_max:np.float64(0.02512134611606598) - rollout_corr/log_ppl_diff_min:np.float64(-0.018030375242233276) - rollout_corr/ppl_ratio:np.float64(1.0006158670876175) - rollout_corr/chi2_token:np.float64(0.00374963553622365) - rollout_corr/chi2_seq:np.float64(0.2979479634668678) - actor/pg_loss:np.float64(0.0008234501583501697) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008168562326318352) - actor/ppo_kl:np.float64(7.368436053400984e-05) - actor/pg_clipfrac_lower:np.float64(4.340277882874943e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4296875) - self_distillation/token_reweight_w_mean:np.float64(121465.32093760255) - self_distillation/token_reweight_w_std:np.float64(847494.297212487) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004239520058036) - self_distillation/token_reweight_w_clip_frac:np.float64(0.021297515399055555) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.37713877856731415) - perf/mfu/actor:np.float64(0.04074059361832152) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.95700788497925) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:0 - critic/score/mean:0.4296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004757009737659246 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004757009737659246 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:89.30078125 - response_length/max:311.0 - response_length/min:43.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:89.30078125 - response_length_non_aborted/max:311.0 - response_length_non_aborted/min:43.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:599.53125 - prompt_length/max:977.0 - prompt_length/min:262.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010108016431331635 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6113905571401119) - timing_s/agent_loop/generate_sequences/max:np.float64(2.294467255473137) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.105186572560342) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.294467255473137) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:631 - timing_s/agent_loop/slowest/response_length:311 - timing_s/gen:8.053372075781226 - timing_s/reward:0.07245118916034698 - timing_s/old_log_prob:2.2529285326600075 - timing_s/adv:0.5971449576318264 - timing_s/update_actor:15.516113428398967 - timing_s/step:26.580770069733262 - timing_s/stop_profile:0.00013327598571777344 - timing_per_token_ms/update_actor:0.08798925620473382 - timing_per_token_ms/gen:0.35227558181099805 - timing_per_token_ms/adv:0.0033863081054991543 - perf/total_num_tokens:176341 - perf/time_per_step:26.580770069733262 - perf/throughput:829.2696164246681 (TaskRunner pid=2204243) Training Progress: 63%|██████▎ | 63/100 [38:12<18:35, 30.15s/it] (TaskRunner pid=2204243) step:64 - global_seqlen/min:19045 - global_seqlen/max:24890 - global_seqlen/minmax_diff:5845 - global_seqlen/balanced_min:21327 - global_seqlen/balanced_max:21332 - global_seqlen/mean:21329.75 - actor/entropy:0.17216283082962036 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2294939160346985 - training/rollout_probs_diff_mean:0.0041274866089224815 - training/rollout_probs_diff_std:0.012461678124964237 - training/rollout_actor_probs_pearson_corr:0.9981242418289185 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.37109375 - self_distillation/correct_sample_fraction:0.2890625 - self_distillation/feedback_available_fraction:0.7109375 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9997236132621765 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.110858132364228e-05 - rollout_corr/rollout_is_max:1.5971819162368774 - rollout_corr/rollout_is_min:0.3082539141178131 - rollout_corr/rollout_is_std:0.036094024777412415 - rollout_corr/rollout_is_eff_sample_size:0.9986980245594285 - rollout_corr/rollout_is_seq_mean:0.999690055847168 - rollout_corr/rollout_is_seq_std:0.0039053531363606453 - rollout_corr/rollout_is_seq_max:1.012115716934204 - rollout_corr/rollout_is_seq_min:0.9882854223251343 - rollout_corr/rollout_is_seq_max_deviation:0.012115716934204102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1879927553236485) - rollout_corr/training_log_ppl:np.float64(0.1668556657168665) - rollout_corr/kl:np.float64(0.0014975151318452617) - rollout_corr/k3_kl:np.float64(0.0009576225180731868) - rollout_corr/rollout_ppl:np.float64(1.1862593442201614) - rollout_corr/rollout_log_ppl:np.float64(0.16535814944654703) - rollout_corr/log_ppl_diff:np.float64(0.0014975162703194655) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037381610527518205) - rollout_corr/log_ppl_diff_max:np.float64(0.022669300436973572) - rollout_corr/log_ppl_diff_min:np.float64(-0.02294468879699707) - rollout_corr/ppl_ratio:np.float64(1.0015117079019547) - rollout_corr/chi2_token:np.float64(0.0009261495433747768) - rollout_corr/chi2_seq:np.float64(0.2422764648217708) - actor/pg_loss:np.float64(0.0015214221784844995) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003768960596062243) - actor/ppo_kl:np.float64(0.00047191293657533606) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.2890625) - self_distillation/token_reweight_w_mean:np.float64(94150.79080042709) - self_distillation/token_reweight_w_std:np.float64(834364.9323378133) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0003909405786544) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03359128799638711) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.4504476487636566) - perf/mfu/actor:np.float64(0.03935514658336046) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.87870407104492) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:0 - critic/score/mean:0.2890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.2890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013306193985044956 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.013306193985044956 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:109.8671875 - response_length/max:545.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:109.8671875 - response_length_non_aborted/max:545.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:556.6875 - prompt_length/max:939.0 - prompt_length/min:245.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018357671797275543 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5570401679724455) - timing_s/agent_loop/generate_sequences/max:np.float64(3.6798063814640045) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2474529358369182) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.6798063814640045) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:551 - timing_s/agent_loop/slowest/response_length:545 - timing_s/gen:9.543762193992734 - timing_s/reward:0.07194927521049976 - timing_s/old_log_prob:2.373839283362031 - timing_s/adv:0.592408686876297 - timing_s/update_actor:15.518252087756991 - timing_s/step:28.18789092078805 - timing_s/stop_profile:0.00011622905731201172 - timing_per_token_ms/update_actor:0.09094253383043045 - timing_per_token_ms/gen:0.3393217021258883 - timing_per_token_ms/adv:0.0034717277914432716 - perf/total_num_tokens:170638 - perf/time_per_step:28.18789092078805 - perf/throughput:756.6990400218167 (TaskRunner pid=2204243) Training Progress: 64%|██████▍ | 64/100 [38:40<17:44, 29.58s/it] (TaskRunner pid=2204243) step:65 - global_seqlen/min:17552 - global_seqlen/max:23753 - global_seqlen/minmax_diff:6201 - global_seqlen/balanced_min:20674 - global_seqlen/balanced_max:20676 - global_seqlen/mean:20675.0 - actor/entropy:0.18921928107738495 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3331897258758545 - training/rollout_probs_diff_mean:0.004135925322771072 - training/rollout_probs_diff_std:0.0123396385461092 - training/rollout_actor_probs_pearson_corr:0.998322069644928 - self_distillation/success_group_fraction:0.5 - self_distillation/success_sample_fraction:0.48828125 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.5859375 - self_distillation/feedback_used_fraction:0.5 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:0.9997037053108215 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.17527761682868e-05 - rollout_corr/rollout_is_max:1.5995460748672485 - rollout_corr/rollout_is_min:0.385733962059021 - rollout_corr/rollout_is_std:0.03636876121163368 - rollout_corr/rollout_is_eff_sample_size:0.9986784065971809 - rollout_corr/rollout_is_seq_mean:0.9995953440666199 - rollout_corr/rollout_is_seq_std:0.004801814444363117 - rollout_corr/rollout_is_seq_max:1.0165265798568726 - rollout_corr/rollout_is_seq_min:0.9844603538513184 - rollout_corr/rollout_is_seq_max_deviation:0.01652657985687256 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2023144564591348) - rollout_corr/training_log_ppl:np.float64(0.17772714045713656) - rollout_corr/kl:np.float64(0.0014629138687922705) - rollout_corr/k3_kl:np.float64(0.0008300077757716906) - rollout_corr/rollout_ppl:np.float64(1.2004908896051347) - rollout_corr/rollout_log_ppl:np.float64(0.17626422486500815) - rollout_corr/log_ppl_diff:np.float64(0.001462915592128411) - rollout_corr/log_ppl_abs_diff:np.float64(0.0038804516079835594) - rollout_corr/log_ppl_diff_max:np.float64(0.01740705966949463) - rollout_corr/log_ppl_diff_min:np.float64(-0.015998616814613342) - rollout_corr/ppl_ratio:np.float64(1.0014764133375138) - rollout_corr/chi2_token:np.float64(0.0005918422248214483) - rollout_corr/chi2_seq:np.float64(0.12960048066452146) - actor/pg_loss:np.float64(0.0005147648043930531) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036167971120448783) - actor/ppo_kl:np.float64(0.00033357571657433027) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(262630.31874683476) - self_distillation/token_reweight_w_std:np.float64(1912057.586982216) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014394370373338) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02761238341918215) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.26377786323428154) - perf/mfu/actor:np.float64(0.03860376557749216) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.8223648071289) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0038679284043610096 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0038679284043610096 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:95.5625 - response_length/max:642.0 - response_length/min:35.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:95.5625 - response_length_non_aborted/max:642.0 - response_length_non_aborted/min:35.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:550.53125 - prompt_length/max:945.0 - prompt_length/min:308.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015504658222198486 - timing_s/agent_loop/generate_sequences/min:np.float64(0.45710465870797634) - timing_s/agent_loop/generate_sequences/max:np.float64(3.9321485739201307) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.082073192264943) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.9321485739201307) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:580 - timing_s/agent_loop/slowest/response_length:642 - timing_s/gen:9.764921706169844 - timing_s/reward:0.0703295785933733 - timing_s/old_log_prob:2.3131796065717936 - timing_s/adv:0.594145180657506 - timing_s/update_actor:15.360387997701764 - timing_s/step:28.19119462557137 - timing_s/stop_profile:0.00012121722102165222 - timing_per_token_ms/update_actor:0.09286812574184863 - timing_per_token_ms/gen:0.3991547460010564 - timing_per_token_ms/adv:0.0035921715880139417 - perf/total_num_tokens:165400 - perf/time_per_step:28.19119462557137 - perf/throughput:733.3850258777732 (TaskRunner pid=2204243) Training Progress: 65%|██████▌ | 65/100 [39:09<17:01, 29.18s/it] (TaskRunner pid=2204243) step:66 - global_seqlen/min:17174 - global_seqlen/max:28212 - global_seqlen/minmax_diff:11038 - global_seqlen/balanced_min:22604 - global_seqlen/balanced_max:22611 - global_seqlen/mean:22608.125 - actor/entropy:0.20968422293663025 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3234601616859436 - training/rollout_probs_diff_mean:0.00455645052716136 - training/rollout_probs_diff_std:0.012609071098268032 - training/rollout_actor_probs_pearson_corr:0.998332142829895 - self_distillation/success_group_fraction:0.40625 - self_distillation/success_sample_fraction:0.39453125 - self_distillation/correct_sample_fraction:0.28125 - self_distillation/feedback_available_fraction:0.71875 - self_distillation/feedback_used_fraction:0.59375 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:0.9999441504478455 - rollout_corr/rollout_is_ratio_fraction_high:3.936852954211645e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.2054405212402344 - rollout_corr/rollout_is_min:0.6073460578918457 - rollout_corr/rollout_is_std:0.03690482676029205 - rollout_corr/rollout_is_eff_sample_size:0.9986397674104652 - rollout_corr/rollout_is_seq_mean:1.0000721216201782 - rollout_corr/rollout_is_seq_std:0.003896591253578663 - rollout_corr/rollout_is_seq_max:1.0156590938568115 - rollout_corr/rollout_is_seq_min:0.9887583255767822 - rollout_corr/rollout_is_seq_max_deviation:0.015659093856811523 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2270207703113556) - rollout_corr/training_log_ppl:np.float64(0.1987948477762984) - rollout_corr/kl:np.float64(0.001155567901170329) - rollout_corr/k3_kl:np.float64(0.000888615789470748) - rollout_corr/rollout_ppl:np.float64(1.2255423944443464) - rollout_corr/rollout_log_ppl:np.float64(0.1976392794167623) - rollout_corr/log_ppl_diff:np.float64(0.0011555683595361188) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037456003919942304) - rollout_corr/log_ppl_diff_max:np.float64(0.018168896436691284) - rollout_corr/log_ppl_diff_min:np.float64(-0.017258822917938232) - rollout_corr/ppl_ratio:np.float64(1.0011675609275699) - rollout_corr/chi2_token:np.float64(0.0011328028049319983) - rollout_corr/chi2_seq:np.float64(0.12879195273853838) - actor/pg_loss:np.float64(0.0010045708622783422) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045344509271671996) - actor/ppo_kl:np.float64(0.0005702014591832949) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.28125) - self_distillation/token_reweight_w_mean:np.float64(574780.558464298) - self_distillation/token_reweight_w_std:np.float64(4772806.540433234) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018667599651963) - self_distillation/token_reweight_w_clip_frac:np.float64(0.026583102444419637) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.2858910672366619) - perf/mfu/actor:np.float64(0.04206193888173963) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.87303924560547) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:0 - critic/score/mean:0.28125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004315774887800217 - critic/advantages/max:0.875 - critic/advantages/min:-0.375 - critic/returns/mean:0.004315774887800217 - critic/returns/max:0.875 - critic/returns/min:-0.375 - response_length/mean:99.22265625 - response_length/max:338.0 - response_length/min:42.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:99.22265625 - response_length_non_aborted/max:338.0 - response_length_non_aborted/min:42.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:607.28125 - prompt_length/max:1165.0 - prompt_length/min:239.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000130530446767807 - timing_s/agent_loop/generate_sequences/min:np.float64(0.608036782592535) - timing_s/agent_loop/generate_sequences/max:np.float64(2.5336991529911757) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.169929622374184) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.5336991529911757) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:717 - timing_s/agent_loop/slowest/response_length:338 - timing_s/gen:8.392959600314498 - timing_s/reward:0.07544131577014923 - timing_s/old_log_prob:2.2662572748959064 - timing_s/adv:0.6118465233594179 - timing_s/update_actor:15.534270694479346 - timing_s/step:26.968531526625156 - timing_s/stop_profile:0.00010806508362293243 - timing_per_token_ms/update_actor:0.08588876064733003 - timing_per_token_ms/gen:0.3304184717260934 - timing_per_token_ms/adv:0.0033828906828818066 - perf/total_num_tokens:180865 - perf/time_per_step:26.968531526625156 - perf/throughput:838.3150182901035 (TaskRunner pid=2204243) Training Progress: 66%|██████▌ | 66/100 [39:36<16:10, 28.53s/it] (TaskRunner pid=2204243) step:67 - global_seqlen/min:18006 - global_seqlen/max:30261 - global_seqlen/minmax_diff:12255 - global_seqlen/balanced_min:23824 - global_seqlen/balanced_max:23960 - global_seqlen/mean:23848.25 - actor/entropy:0.15897494554519653 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22936123609542847 - training/rollout_probs_diff_mean:0.0036624132189899683 - training/rollout_probs_diff_std:0.011706416495144367 - training/rollout_actor_probs_pearson_corr:0.9982317686080933 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.33984375 - self_distillation/correct_sample_fraction:0.29296875 - self_distillation/feedback_available_fraction:0.70703125 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.999773383140564 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8682467937469482 - rollout_corr/rollout_is_min:0.6065937280654907 - rollout_corr/rollout_is_std:0.033457957208156586 - rollout_corr/rollout_is_eff_sample_size:0.9988814005998093 - rollout_corr/rollout_is_seq_mean:1.0000083446502686 - rollout_corr/rollout_is_seq_std:0.00415265467017889 - rollout_corr/rollout_is_seq_max:1.0149999856948853 - rollout_corr/rollout_is_seq_min:0.9884124398231506 - rollout_corr/rollout_is_seq_max_deviation:0.014999985694885254 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.182903200853616) - rollout_corr/training_log_ppl:np.float64(0.16291830602131085) - rollout_corr/kl:np.float64(0.0012190366105215844) - rollout_corr/k3_kl:np.float64(0.0006799885535286876) - rollout_corr/rollout_ppl:np.float64(1.1814202656969428) - rollout_corr/rollout_log_ppl:np.float64(0.1616992673079949) - rollout_corr/log_ppl_diff:np.float64(0.0012190387133159675) - rollout_corr/log_ppl_abs_diff:np.float64(0.0033827177903731354) - rollout_corr/log_ppl_diff_max:np.float64(0.016148462891578674) - rollout_corr/log_ppl_diff_min:np.float64(-0.00940769910812378) - rollout_corr/ppl_ratio:np.float64(1.0012285171542317) - rollout_corr/chi2_token:np.float64(0.00023990776389837265) - rollout_corr/chi2_seq:np.float64(0.013280826853588223) - actor/pg_loss:np.float64(0.0001726325135678053) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003200081664544996) - actor/ppo_kl:np.float64(0.0005977377616792623) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.29296875) - self_distillation/token_reweight_w_mean:np.float64(226851.71611470752) - self_distillation/token_reweight_w_std:np.float64(1812229.1514175457) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002479723189026) - self_distillation/token_reweight_w_clip_frac:np.float64(0.007520618062699214) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.19119691103696823) - perf/mfu/actor:np.float64(0.04406968158563203) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.85897064208984) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:0 - critic/score/mean:0.29296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.29296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0010344889014959335 - critic/advantages/max:0.875 - critic/advantages/min:-0.25 - critic/returns/mean:0.0010344889014959335 - critic/returns/max:0.875 - critic/returns/min:-0.25 - response_length/mean:109.9765625 - response_length/max:1360.0 - response_length/min:34.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:109.9765625 - response_length_non_aborted/max:1360.0 - response_length_non_aborted/min:34.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:635.28125 - prompt_length/max:1258.0 - prompt_length/min:244.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.919703006744385e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4477618783712387) - timing_s/agent_loop/generate_sequences/max:np.float64(7.133308483287692) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0619034790652222) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.133308483287692) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:658 - timing_s/agent_loop/slowest/response_length:1360 - timing_s/gen:13.418079048395157 - timing_s/reward:0.07516939006745815 - timing_s/old_log_prob:2.2874904349446297 - timing_s/adv:0.6309217792004347 - timing_s/update_actor:15.556731447577477 - timing_s/step:32.056258948519826 - timing_s/stop_profile:0.00012564100325107574 - timing_per_token_ms/update_actor:0.08154021494018154 - timing_per_token_ms/gen:0.47659583179637555 - timing_per_token_ms/adv:0.0033069605694360945 - perf/total_num_tokens:190786 - perf/time_per_step:32.056258948519826 - perf/throughput:743.9498800623825 (TaskRunner pid=2204243) Training Progress: 67%|██████▋ | 67/100 [40:08<16:16, 29.60s/it] (TaskRunner pid=2204243) step:68 - global_seqlen/min:19197 - global_seqlen/max:29330 - global_seqlen/minmax_diff:10133 - global_seqlen/balanced_min:21769 - global_seqlen/balanced_max:21774 - global_seqlen/mean:21771.125 - actor/entropy:0.16915389895439148 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17915785312652588 - training/rollout_probs_diff_mean:0.00364634464494884 - training/rollout_probs_diff_std:0.011112609878182411 - training/rollout_actor_probs_pearson_corr:0.9985078573226929 - self_distillation/success_group_fraction:0.28125 - self_distillation/success_sample_fraction:0.28125 - self_distillation/correct_sample_fraction:0.28125 - self_distillation/feedback_available_fraction:0.71875 - self_distillation/feedback_used_fraction:0.71875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0001206398010254 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.627697467803955 - rollout_corr/rollout_is_min:0.6462766528129578 - rollout_corr/rollout_is_std:0.03299422934651375 - rollout_corr/rollout_is_eff_sample_size:0.9989128024713322 - rollout_corr/rollout_is_seq_mean:1.0001252889633179 - rollout_corr/rollout_is_seq_std:0.003706379095092416 - rollout_corr/rollout_is_seq_max:1.0112172365188599 - rollout_corr/rollout_is_seq_min:0.9876261353492737 - rollout_corr/rollout_is_seq_max_deviation:0.012373864650726318 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1799099128693342) - rollout_corr/training_log_ppl:np.float64(0.16039785902830772) - rollout_corr/kl:np.float64(0.00048031557349048626) - rollout_corr/k3_kl:np.float64(0.0006000358300415343) - rollout_corr/rollout_ppl:np.float64(1.179317635949701) - rollout_corr/rollout_log_ppl:np.float64(0.15991754146671155) - rollout_corr/log_ppl_diff:np.float64(0.00048031756159616634) - rollout_corr/log_ppl_abs_diff:np.float64(0.002982226164021995) - rollout_corr/log_ppl_diff_max:np.float64(0.018945112824440002) - rollout_corr/log_ppl_diff_min:np.float64(-0.011037811636924744) - rollout_corr/ppl_ratio:np.float64(1.000488493591547) - rollout_corr/chi2_token:np.float64(0.0014690356329083443) - rollout_corr/chi2_seq:np.float64(0.17614812194369733) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(5.6858210182753055e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.28125) - self_distillation/token_reweight_w_mean:np.float64(0.9999999920837581) - self_distillation/token_reweight_w_std:np.float64(0.0) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999999920837581) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.040839027349956335) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.8672218322754) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:0 - critic/score/mean:0.28125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:95.31640625 - response_length/max:232.0 - response_length/min:36.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:95.31640625 - response_length_non_aborted/max:232.0 - response_length_non_aborted/min:36.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:585.03125 - prompt_length/max:1262.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010986253619194031 - timing_s/agent_loop/generate_sequences/min:np.float64(0.44508632086217403) - timing_s/agent_loop/generate_sequences/max:np.float64(2.0003224052488804) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.05117017844168) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.0003224052488804) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:538 - timing_s/agent_loop/slowest/response_length:200 - timing_s/gen:7.965172305703163 - timing_s/reward:0.07240584678947926 - timing_s/old_log_prob:2.427291888743639 - timing_s/adv:0.6243052911013365 - timing_s/update_actor:15.348970513790846 - timing_s/step:26.526268865913153 - timing_s/stop_profile:0.00011803023517131805 - timing_per_token_ms/update_actor:0.08812687971907082 - timing_per_token_ms/gen:0.32642810973743547 - timing_per_token_ms/adv:0.003584479965443543 - perf/total_num_tokens:174169 - perf/time_per_step:26.526268865913153 - perf/throughput:820.7383069986289 (TaskRunner pid=2204243) Training Progress: 68%|██████▊ | 68/100 [40:34<15:18, 28.69s/it] (TaskRunner pid=2204243) step:69 - global_seqlen/min:16524 - global_seqlen/max:30559 - global_seqlen/minmax_diff:14035 - global_seqlen/balanced_min:24085 - global_seqlen/balanced_max:24090 - global_seqlen/mean:24087.5 - actor/entropy:0.20297829806804657 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29786932468414307 - training/rollout_probs_diff_mean:0.0041356380097568035 - training/rollout_probs_diff_std:0.012082680128514767 - training/rollout_actor_probs_pearson_corr:0.9984449744224548 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.375 - self_distillation/correct_sample_fraction:0.29296875 - self_distillation/feedback_available_fraction:0.70703125 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9998466372489929 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9674241542816162 - rollout_corr/rollout_is_min:0.589870274066925 - rollout_corr/rollout_is_std:0.03635154664516449 - rollout_corr/rollout_is_eff_sample_size:0.9986798333320833 - rollout_corr/rollout_is_seq_mean:0.9997987747192383 - rollout_corr/rollout_is_seq_std:0.0035299560986459255 - rollout_corr/rollout_is_seq_max:1.0096718072891235 - rollout_corr/rollout_is_seq_min:0.9885482788085938 - rollout_corr/rollout_is_seq_max_deviation:0.01145172119140625 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2195711270906031) - rollout_corr/training_log_ppl:np.float64(0.19257877549534896) - rollout_corr/kl:np.float64(0.0009204127151409836) - rollout_corr/k3_kl:np.float64(0.001296327421457022) - rollout_corr/rollout_ppl:np.float64(1.2183606382459402) - rollout_corr/rollout_log_ppl:np.float64(0.19165836123283952) - rollout_corr/log_ppl_diff:np.float64(0.0009204142625094391) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037502322738873772) - rollout_corr/log_ppl_diff_max:np.float64(0.024910718202590942) - rollout_corr/log_ppl_diff_min:np.float64(-0.026864521205425262) - rollout_corr/ppl_ratio:np.float64(1.0009357139933854) - rollout_corr/chi2_token:np.float64(0.0035917784553021193) - rollout_corr/chi2_seq:np.float64(0.6056221991311759) - actor/pg_loss:np.float64(0.0013994482578709722) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010189479871769436) - actor/ppo_kl:np.float64(6.697692964507951e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.29296875) - self_distillation/token_reweight_w_mean:np.float64(198652.77768032183) - self_distillation/token_reweight_w_std:np.float64(1629332.3844376116) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997963055502623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041180176398484036) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.5828979983925819) - perf/mfu/actor:np.float64(0.04453443621465142) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.84906768798828) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:0 - critic/score/mean:0.29296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.29296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-1.3957123883301392e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-1.3957123883301392e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:104.953125 - response_length/max:454.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:104.953125 - response_length_non_aborted/max:454.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:647.78125 - prompt_length/max:1521.0 - prompt_length/min:348.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010402314364910126 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4764007702469826) - timing_s/agent_loop/generate_sequences/max:np.float64(2.9652316365391016) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.122635073159472) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.9652316365391016) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:867 - timing_s/agent_loop/slowest/response_length:454 - timing_s/gen:8.804267661646008 - timing_s/reward:0.07646027021110058 - timing_s/old_log_prob:2.2781725712120533 - timing_s/adv:0.6520331110805273 - timing_s/update_actor:15.58453724347055 - timing_s/step:27.48270800150931 - timing_s/stop_profile:0.00012394413352012634 - timing_per_token_ms/update_actor:0.08087460946274286 - timing_per_token_ms/gen:0.32768600795169006 - timing_per_token_ms/adv:0.003383669491855357 - perf/total_num_tokens:192700 - perf/time_per_step:27.48270800150931 - perf/throughput:876.4602090404319 (TaskRunner pid=2204243) Training Progress: 69%|██████▉ | 69/100 [41:02<14:38, 28.35s/it] (TaskRunner pid=2204243) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2204243) validation generation end (TaskRunner pid=2204243) [prompt] user (TaskRunner pid=2204243) Your task is to answer the user's question using available tools. (TaskRunner pid=2204243) You have access to the following tools: (TaskRunner pid=2204243) Name: Axolotl (TaskRunner pid=2204243) Description: Collection of axolotl pictures and facts (TaskRunner pid=2204243) Documentation: (TaskRunner pid=2204243) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2204243) Parameters: {} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{url, source, description} (TaskRunner pid=2204243) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2204243) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2204243) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2204243) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Array[Object{fact, source}] (TaskRunner pid=2204243) (TaskRunner pid=2204243) Use the following format: (TaskRunner pid=2204243) Thought: you should always think about what to do (TaskRunner pid=2204243) Action: the action to take, should be one of the tool names. (TaskRunner pid=2204243) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2204243) (TaskRunner pid=2204243) Begin! (TaskRunner pid=2204243) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2204243) assistant (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: I need to find a random axolotl image. The user hasn't specified any particular criteria, so I should use the getRandomAxolotlImage tool. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2204655) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2204243) step:70 - global_seqlen/min:18136 - global_seqlen/max:28626 - global_seqlen/minmax_diff:10490 - global_seqlen/balanced_min:23806 - global_seqlen/balanced_max:23809 - global_seqlen/mean:23807.375 - actor/entropy:0.1836181879043579 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21924197673797607 - training/rollout_probs_diff_mean:0.0037344321608543396 - training/rollout_probs_diff_std:0.011251681484282017 - training/rollout_actor_probs_pearson_corr:0.9985505938529968 - self_distillation/success_group_fraction:0.53125 - self_distillation/success_sample_fraction:0.53125 - self_distillation/correct_sample_fraction:0.51171875 - self_distillation/feedback_available_fraction:0.48828125 - self_distillation/feedback_used_fraction:0.46875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999583959579468 - rollout_corr/rollout_is_ratio_fraction_high:3.9690414268989116e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.3013765811920166 - rollout_corr/rollout_is_min:0.5136908292770386 - rollout_corr/rollout_is_std:0.033298131078481674 - rollout_corr/rollout_is_eff_sample_size:0.998892224506785 - rollout_corr/rollout_is_seq_mean:0.999870777130127 - rollout_corr/rollout_is_seq_std:0.00424517085775733 - rollout_corr/rollout_is_seq_max:1.0353556871414185 - rollout_corr/rollout_is_seq_min:0.9887574911117554 - rollout_corr/rollout_is_seq_max_deviation:0.03535568714141846 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1923476299270988) - rollout_corr/training_log_ppl:np.float64(0.1674999686292722) - rollout_corr/kl:np.float64(0.001711104008053077) - rollout_corr/k3_kl:np.float64(0.0012346766571030843) - rollout_corr/rollout_ppl:np.float64(1.190187608357519) - rollout_corr/rollout_log_ppl:np.float64(0.16578886295610573) - rollout_corr/log_ppl_diff:np.float64(0.0017111056731664576) - rollout_corr/log_ppl_abs_diff:np.float64(0.004042080974613782) - rollout_corr/log_ppl_diff_max:np.float64(0.02568104863166809) - rollout_corr/log_ppl_diff_min:np.float64(-0.015813075006008148) - rollout_corr/ppl_ratio:np.float64(1.0017276084981859) - rollout_corr/chi2_token:np.float64(0.0012146567460149527) - rollout_corr/chi2_seq:np.float64(0.09416058892384171) - actor/pg_loss:np.float64(0.0003779985709115863) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044372407501214184) - actor/ppo_kl:np.float64(0.0009813297224887663) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.51171875) - self_distillation/token_reweight_w_mean:np.float64(124132.25338875502) - self_distillation/token_reweight_w_std:np.float64(913681.4291177337) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995329540688545) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014685297675896436) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.23002920299768448) - perf/mfu/actor:np.float64(0.044148276739960406) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.80029296875) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5946691176470589) - val-aux/tooluse/reward/std@16:np.float64(0.0798580594829434) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6286470588235293) - val-aux/tooluse/reward/best@2/std:np.float64(0.06415072107768015) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5620588235294117) - val-aux/tooluse/reward/worst@2/std:np.float64(0.07187329244573848) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5957205882352941) - val-aux/tooluse/reward/maj@2/std:np.float64(0.07992022278558766) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6513970588235294) - val-aux/tooluse/reward/best@4/std:np.float64(0.04372500853747388) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5329705882352942) - val-aux/tooluse/reward/worst@4/std:np.float64(0.05636448793027157) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.6012205882352941) - val-aux/tooluse/reward/maj@4/std:np.float64(0.06485408775996664) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6670735294117648) - val-aux/tooluse/reward/best@8/std:np.float64(0.02957882610876418) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5096323529411765) - val-aux/tooluse/reward/worst@8/std:np.float64(0.038190130728107195) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.6061029411764707) - val-aux/tooluse/reward/maj@8/std:np.float64(0.0499031384225905) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6785735294117647) - val-aux/tooluse/reward/best@16/std:np.float64(0.020828451075017008) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.4947352941176471) - val-aux/tooluse/reward/worst@16/std:np.float64(0.021230822023229987) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.6106617647058823) - val-aux/tooluse/reward/maj@16/std:np.float64(0.03810105405597047) - val-aux/tooluse/score/mean@16:np.float64(0.5946691176470589) - val-aux/tooluse/score/std@16:np.float64(0.0798580594829434) - val-aux/tooluse/score/best@2/mean:np.float64(0.6286470588235293) - val-aux/tooluse/score/best@2/std:np.float64(0.06415072107768015) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5620588235294117) - val-aux/tooluse/score/worst@2/std:np.float64(0.07187329244573848) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5957205882352941) - val-aux/tooluse/score/maj@2/std:np.float64(0.07992022278558766) - val-aux/tooluse/score/best@4/mean:np.float64(0.6513970588235294) - val-aux/tooluse/score/best@4/std:np.float64(0.04372500853747388) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5329705882352942) - val-aux/tooluse/score/worst@4/std:np.float64(0.05636448793027157) - val-aux/tooluse/score/maj@4/mean:np.float64(0.6012205882352941) - val-aux/tooluse/score/maj@4/std:np.float64(0.06485408775996664) - val-aux/tooluse/score/best@8/mean:np.float64(0.6670735294117648) - val-aux/tooluse/score/best@8/std:np.float64(0.02957882610876418) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5096323529411765) - val-aux/tooluse/score/worst@8/std:np.float64(0.038190130728107195) - val-aux/tooluse/score/maj@8/mean:np.float64(0.6061029411764707) - val-aux/tooluse/score/maj@8/std:np.float64(0.0499031384225905) - val-aux/tooluse/score/best@16/mean:np.float64(0.6785735294117647) - val-aux/tooluse/score/best@16/std:np.float64(0.020828451075017008) - val-aux/tooluse/score/worst@16/mean:np.float64(0.4947352941176471) - val-aux/tooluse/score/worst@16/std:np.float64(0.021230822023229987) - val-aux/tooluse/score/maj@16/mean:np.float64(0.6106617647058823) - val-aux/tooluse/score/maj@16/std:np.float64(0.03810105405597047) - val-core/tooluse/acc/mean@16:np.float64(0.5946691176470589) - val-aux/tooluse/acc/std@16:np.float64(0.0798580594829434) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6286470588235293) - val-aux/tooluse/acc/best@2/std:np.float64(0.06415072107768015) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5620588235294117) - val-aux/tooluse/acc/worst@2/std:np.float64(0.07187329244573848) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5957205882352941) - val-aux/tooluse/acc/maj@2/std:np.float64(0.07992022278558766) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6513970588235294) - val-aux/tooluse/acc/best@4/std:np.float64(0.04372500853747388) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5329705882352942) - val-aux/tooluse/acc/worst@4/std:np.float64(0.05636448793027157) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.6012205882352941) - val-aux/tooluse/acc/maj@4/std:np.float64(0.06485408775996664) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6670735294117648) - val-aux/tooluse/acc/best@8/std:np.float64(0.02957882610876418) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5096323529411765) - val-aux/tooluse/acc/worst@8/std:np.float64(0.038190130728107195) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.6061029411764707) - val-aux/tooluse/acc/maj@8/std:np.float64(0.0499031384225905) - val-core/tooluse/acc/best@16/mean:np.float64(0.6785735294117647) - val-core/tooluse/acc/best@16/std:np.float64(0.020828451075017008) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.4947352941176471) - val-aux/tooluse/acc/worst@16/std:np.float64(0.021230822023229987) - val-core/tooluse/acc/maj@16/mean:np.float64(0.6106617647058823) - val-core/tooluse/acc/maj@16/std:np.float64(0.03810105405597047) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:0 - critic/score/mean:0.51171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.51171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016471522394567728 - critic/advantages/max:0.375 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016471522394567728 - critic/returns/max:0.375 - critic/returns/min:-0.875 - response_length/mean:98.41796875 - response_length/max:433.0 - response_length/min:39.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:98.41796875 - response_length_non_aborted/max:433.0 - response_length_non_aborted/min:39.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:645.5625 - prompt_length/max:1228.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012632831931114197 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5998907629400492) - timing_s/agent_loop/generate_sequences/max:np.float64(2.9571366477757692) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.173424483378767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.9571366477757692) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1010 - timing_s/agent_loop/slowest/response_length:433 - timing_s/gen:8.699137950316072 - timing_s/reward:0.07445470802485943 - timing_s/old_log_prob:2.2356281019747257 - timing_s/adv:0.6057741101831198 - timing_s/update_actor:15.194010950624943 - timing_s/step:26.89719434455037 - timing_s/testing:21.24201563373208 - timing_s/save_checkpoint:6.66465063393116 - timing_s/stop_profile:0.00016132928431034088 - timing_per_token_ms/update_actor:0.07977575725287302 - timing_per_token_ms/gen:0.34527239334455534 - timing_per_token_ms/adv:0.003180601127713155 - perf/total_num_tokens:190459 - perf/time_per_step:26.89719434455037 - perf/throughput:885.1248459237014 (TaskRunner pid=2204243) Training Progress: 70%|███████ | 70/100 [41:57<18:09, 36.31s/it] (TaskRunner pid=2204243) step:71 - global_seqlen/min:19490 - global_seqlen/max:24969 - global_seqlen/minmax_diff:5479 - global_seqlen/balanced_min:22312 - global_seqlen/balanced_max:24889 - global_seqlen/mean:22713.0 - actor/entropy:0.13971230387687683 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23184734582901 - training/rollout_probs_diff_mean:0.003360356669872999 - training/rollout_probs_diff_std:0.011199713684618473 - training/rollout_actor_probs_pearson_corr:0.9981712698936462 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.61328125 - self_distillation/correct_sample_fraction:0.484375 - self_distillation/feedback_available_fraction:0.515625 - self_distillation/feedback_used_fraction:0.375 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:1.0002855062484741 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9838473796844482 - rollout_corr/rollout_is_min:0.5107617378234863 - rollout_corr/rollout_is_std:0.0320243276655674 - rollout_corr/rollout_is_eff_sample_size:0.9989760879987939 - rollout_corr/rollout_is_seq_mean:1.000514030456543 - rollout_corr/rollout_is_seq_std:0.004211563616991043 - rollout_corr/rollout_is_seq_max:1.0244622230529785 - rollout_corr/rollout_is_seq_min:0.9845895767211914 - rollout_corr/rollout_is_seq_max_deviation:0.024462223052978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1774482005275786) - rollout_corr/training_log_ppl:np.float64(0.15947280398540897) - rollout_corr/kl:np.float64(0.0007086538114968732) - rollout_corr/k3_kl:np.float64(0.0010046848543510123) - rollout_corr/rollout_ppl:np.float64(1.1765439701266587) - rollout_corr/rollout_log_ppl:np.float64(0.15876415105958586) - rollout_corr/log_ppl_diff:np.float64(0.0007086529258231167) - rollout_corr/log_ppl_abs_diff:np.float64(0.003673080867883982) - rollout_corr/log_ppl_diff_max:np.float64(0.01353137195110321) - rollout_corr/log_ppl_diff_min:np.float64(-0.02259892225265503) - rollout_corr/ppl_ratio:np.float64(1.0007204895373434) - rollout_corr/chi2_token:np.float64(0.0026063655968755484) - rollout_corr/chi2_seq:np.float64(0.21057552751153708) - actor/pg_loss:np.float64(0.0013483036309480667) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039533938365821086) - actor/ppo_kl:np.float64(0.000602565654877818) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.484375) - self_distillation/token_reweight_w_mean:np.float64(496224.6731502523) - self_distillation/token_reweight_w_std:np.float64(4082958.567067202) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000827145529911) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03326115601885249) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.44041093438863754) - perf/mfu/actor:np.float64(0.041161158642668755) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.01330947875977) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:0 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.07906116545200348 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.07906116545200348 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:124.65625 - response_length/max:3617.0 - response_length/min:38.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:124.65625 - response_length_non_aborted/max:3617.0 - response_length_non_aborted/min:38.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:585.125 - prompt_length/max:880.0 - prompt_length/min:389.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.69738107919693e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.45664551109075546) - timing_s/agent_loop/generate_sequences/max:np.float64(17.583325112238526) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2127331683514058) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.583325112238526) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:536 - timing_s/agent_loop/slowest/response_length:3617 - timing_s/gen:23.330108407884836 - timing_s/reward:0.06837156042456627 - timing_s/old_log_prob:2.4029543083161116 - timing_s/adv:0.6026344485580921 - timing_s/update_actor:15.92515530437231 - timing_s/step:42.43415713682771 - timing_s/stop_profile:0.00012646056711673737 - timing_per_token_ms/update_actor:0.0876433942256214 - timing_per_token_ms/gen:0.7310763477025832 - timing_per_token_ms/adv:0.003316572274457866 - perf/total_num_tokens:181704 - perf/time_per_step:42.43415713682771 - perf/throughput:535.2527664627011 (TaskRunner pid=2204243) Training Progress: 71%|███████ | 71/100 [42:39<18:26, 38.16s/it] (TaskRunner pid=2204243) step:72 - global_seqlen/min:16670 - global_seqlen/max:31302 - global_seqlen/minmax_diff:14632 - global_seqlen/balanced_min:24347 - global_seqlen/balanced_max:24352 - global_seqlen/mean:24350.25 - actor/entropy:0.17336490750312805 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31862419843673706 - training/rollout_probs_diff_mean:0.0039044052828103304 - training/rollout_probs_diff_std:0.012037508189678192 - training/rollout_actor_probs_pearson_corr:0.9982341527938843 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.375 - self_distillation/correct_sample_fraction:0.34375 - self_distillation/feedback_available_fraction:0.65625 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9995772838592529 - rollout_corr/rollout_is_ratio_fraction_high:3.723285408341326e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.4298923015594482 - rollout_corr/rollout_is_min:0.5584025979042053 - rollout_corr/rollout_is_std:0.03514183685183525 - rollout_corr/rollout_is_eff_sample_size:0.9987653259718028 - rollout_corr/rollout_is_seq_mean:0.999469518661499 - rollout_corr/rollout_is_seq_std:0.003958587069064379 - rollout_corr/rollout_is_seq_max:1.0143871307373047 - rollout_corr/rollout_is_seq_min:0.9847607612609863 - rollout_corr/rollout_is_seq_max_deviation:0.015239238739013672 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1789607214741409) - rollout_corr/training_log_ppl:np.float64(0.15941029245732352) - rollout_corr/kl:np.float64(0.0014580503966783454) - rollout_corr/k3_kl:np.float64(0.0008553113256155598) - rollout_corr/rollout_ppl:np.float64(1.1772252516821027) - rollout_corr/rollout_log_ppl:np.float64(0.15795224096655147) - rollout_corr/log_ppl_diff:np.float64(0.0014580514907720499) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036441966294660233) - rollout_corr/log_ppl_diff_max:np.float64(0.04131127893924713) - rollout_corr/log_ppl_diff_min:np.float64(-0.011709451675415039) - rollout_corr/ppl_ratio:np.float64(1.001472253818065) - rollout_corr/chi2_token:np.float64(0.000416038790717721) - rollout_corr/chi2_seq:np.float64(0.09685749630443752) - actor/pg_loss:np.float64(0.00030853983480483294) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00048680295185477007) - actor/ppo_kl:np.float64(0.0003038988050922953) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.34375) - self_distillation/token_reweight_w_mean:np.float64(86406.2217199707) - self_distillation/token_reweight_w_std:np.float64(787074.5188532125) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0002374455798417) - self_distillation/token_reweight_w_clip_frac:np.float64(0.016149164614034817) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.2884107492864132) - perf/mfu/actor:np.float64(0.044097635923031384) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.9966583251953) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:0 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018476804252713919 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018476804252713919 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:104.9140625 - response_length/max:274.0 - response_length/min:45.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:104.9140625 - response_length_non_aborted/max:274.0 - response_length_non_aborted/min:45.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:656.03125 - prompt_length/max:1504.0 - prompt_length/min:275.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011811591684818268 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5577939935028553) - timing_s/agent_loop/generate_sequences/max:np.float64(2.1868883557617664) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2234121287474409) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.1868883557617664) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:934 - timing_s/agent_loop/slowest/response_length:256 - timing_s/gen:8.080066932365298 - timing_s/reward:0.07572477124631405 - timing_s/old_log_prob:2.3795893676579 - timing_s/adv:0.6499445158988237 - timing_s/update_actor:15.856424532830715 - timing_s/step:27.128469137474895 - timing_s/stop_profile:0.00011540204286575317 - timing_per_token_ms/update_actor:0.08139764752328373 - timing_per_token_ms/gen:0.3008439545895189 - timing_per_token_ms/adv:0.003336436565840308 - perf/total_num_tokens:194802 - perf/time_per_step:27.128469137474895 - perf/throughput:897.5902722930613 (TaskRunner pid=2204243) Training Progress: 72%|███████▏ | 72/100 [43:06<16:16, 34.86s/it] (TaskRunner pid=2204243) step:73 - global_seqlen/min:17169 - global_seqlen/max:26248 - global_seqlen/minmax_diff:9079 - global_seqlen/balanced_min:23356 - global_seqlen/balanced_max:23365 - global_seqlen/mean:23362.625 - actor/entropy:0.16339124739170074 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105686902999878 - training/rollout_probs_diff_mean:0.004083090461790562 - training/rollout_probs_diff_std:0.012518049217760563 - training/rollout_actor_probs_pearson_corr:0.9980318546295166 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.34375 - self_distillation/correct_sample_fraction:0.28515625 - self_distillation/feedback_available_fraction:0.71484375 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9998648762702942 - rollout_corr/rollout_is_ratio_fraction_high:3.646839832072146e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.646839832072146e-05 - rollout_corr/rollout_is_max:2.055161714553833 - rollout_corr/rollout_is_min:0.4297546446323395 - rollout_corr/rollout_is_std:0.03553064167499542 - rollout_corr/rollout_is_eff_sample_size:0.9987388086081045 - rollout_corr/rollout_is_seq_mean:0.99981689453125 - rollout_corr/rollout_is_seq_std:0.0035779119934886694 - rollout_corr/rollout_is_seq_max:1.0140897035598755 - rollout_corr/rollout_is_seq_min:0.990483283996582 - rollout_corr/rollout_is_seq_max_deviation:0.014089703559875488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1769403284415603) - rollout_corr/training_log_ppl:np.float64(0.15965253425383708) - rollout_corr/kl:np.float64(0.000889868779545111) - rollout_corr/k3_kl:np.float64(0.0008503772052108616) - rollout_corr/rollout_ppl:np.float64(1.1758585586212575) - rollout_corr/rollout_log_ppl:np.float64(0.1587626645778073) - rollout_corr/log_ppl_diff:np.float64(0.0008898696760297753) - rollout_corr/log_ppl_abs_diff:np.float64(0.003167667229718063) - rollout_corr/log_ppl_diff_max:np.float64(0.014793917536735535) - rollout_corr/log_ppl_diff_min:np.float64(-0.01035696268081665) - rollout_corr/ppl_ratio:np.float64(1.0008984378073364) - rollout_corr/chi2_token:np.float64(0.0021289775613695383) - rollout_corr/chi2_seq:np.float64(0.21070270310156047) - actor/pg_loss:np.float64(0.0009087058715522289) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003231324371881783) - actor/ppo_kl:np.float64(7.436231013357997e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.28515625) - self_distillation/token_reweight_w_mean:np.float64(397097.7051075774) - self_distillation/token_reweight_w_std:np.float64(3088084.429281376) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004388359375298) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014315580134280026) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.31481652706861496) - perf/mfu/actor:np.float64(0.04247338507046719) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.04314661026) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:0 - critic/score/mean:0.28515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.28515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:1.36756498250179e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.5 - critic/returns/mean:1.36756498250179e-05 - critic/returns/max:0.75 - critic/returns/min:-0.5 - response_length/mean:107.11328125 - response_length/max:546.0 - response_length/min:42.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:107.11328125 - response_length_non_aborted/max:546.0 - response_length_non_aborted/min:42.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:622.96875 - prompt_length/max:907.0 - prompt_length/min:354.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017004460096359253 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6583534255623817) - timing_s/agent_loop/generate_sequences/max:np.float64(3.4240427371114492) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2717665863820002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.4240427371114492) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:722 - timing_s/agent_loop/slowest/response_length:546 - timing_s/gen:9.500233164057136 - timing_s/reward:0.07463196665048599 - timing_s/old_log_prob:2.4434218294918537 - timing_s/adv:0.6019022837281227 - timing_s/update_actor:15.857062119990587 - timing_s/step:28.56592097878456 - timing_s/stop_profile:0.00011659227311611176 - timing_per_token_ms/update_actor:0.08484204001043648 - timing_per_token_ms/gen:0.34645830436735114 - timing_per_token_ms/adv:0.003220433725491692 - perf/total_num_tokens:186901 - perf/time_per_step:28.56592097878456 - perf/throughput:817.8495283716228 (TaskRunner pid=2204243) Training Progress: 73%|███████▎ | 73/100 [43:35<14:51, 33.00s/it] (TaskRunner pid=2204243) step:74 - global_seqlen/min:19521 - global_seqlen/max:25920 - global_seqlen/minmax_diff:6399 - global_seqlen/balanced_min:22515 - global_seqlen/balanced_max:22525 - global_seqlen/mean:22522.5 - actor/entropy:0.20618578791618347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27729904651641846 - training/rollout_probs_diff_mean:0.004290976095944643 - training/rollout_probs_diff_std:0.01236980315297842 - training/rollout_actor_probs_pearson_corr:0.998401939868927 - self_distillation/success_group_fraction:0.3125 - self_distillation/success_sample_fraction:0.3125 - self_distillation/correct_sample_fraction:0.3046875 - self_distillation/feedback_available_fraction:0.6953125 - self_distillation/feedback_used_fraction:0.6875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999351501464844 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.831534119555727e-05 - rollout_corr/rollout_is_max:1.9132237434387207 - rollout_corr/rollout_is_min:0.3921355903148651 - rollout_corr/rollout_is_std:0.035967472940683365 - rollout_corr/rollout_is_eff_sample_size:0.9987077743788828 - rollout_corr/rollout_is_seq_mean:1.000075101852417 - rollout_corr/rollout_is_seq_std:0.003660839283838868 - rollout_corr/rollout_is_seq_max:1.0144089460372925 - rollout_corr/rollout_is_seq_min:0.983939528465271 - rollout_corr/rollout_is_seq_max_deviation:0.016060471534729004 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2141870078630745) - rollout_corr/training_log_ppl:np.float64(0.1866758709657006) - rollout_corr/kl:np.float64(0.0008737989317744166) - rollout_corr/k3_kl:np.float64(0.0009263143030375787) - rollout_corr/rollout_ppl:np.float64(1.21305015636608) - rollout_corr/rollout_log_ppl:np.float64(0.18580207211198285) - rollout_corr/log_ppl_diff:np.float64(0.0008737988537177444) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035511332389432937) - rollout_corr/log_ppl_diff_max:np.float64(0.020244911313056946) - rollout_corr/log_ppl_diff_min:np.float64(-0.016882598400115967) - rollout_corr/ppl_ratio:np.float64(1.0008854367770255) - rollout_corr/chi2_token:np.float64(0.001916322624310851) - rollout_corr/chi2_seq:np.float64(0.24254979263059795) - actor/pg_loss:np.float64(0.0006697163917124271) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003125329349131789) - actor/ppo_kl:np.float64(0.0003027935115795799) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3046875) - self_distillation/token_reweight_w_mean:np.float64(83162.0599368345) - self_distillation/token_reweight_w_std:np.float64(491112.5860763433) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995260804425925) - self_distillation/token_reweight_w_clip_frac:np.float64(0.006346230220515281) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.14878704026341438) - perf/mfu/actor:np.float64(0.0418206068655697) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.02629470825195) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:0 - critic/score/mean:0.3046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005123650771565735 - critic/advantages/max:0.25 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0005123650771565735 - critic/returns/max:0.25 - critic/returns/min:-0.75 - response_length/mean:114.359375 - response_length/max:651.0 - response_length/min:45.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:114.359375 - response_length_non_aborted/max:651.0 - response_length_non_aborted/min:45.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:589.46875 - prompt_length/max:973.0 - prompt_length/min:380.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015323609113693237 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6179168373346329) - timing_s/agent_loop/generate_sequences/max:np.float64(4.011843465268612) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2877205274271546) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.011843465268612) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:704 - timing_s/agent_loop/slowest/response_length:651 - timing_s/gen:9.80706818588078 - timing_s/reward:0.07173065654933453 - timing_s/old_log_prob:2.2883298583328724 - timing_s/adv:0.5773394219577312 - timing_s/update_actor:15.463554060086608 - timing_s/step:28.295814787968993 - timing_s/stop_profile:0.00011878646910190582 - timing_per_token_ms/update_actor:0.08582281085629154 - timing_per_token_ms/gen:0.334986616541904 - timing_per_token_ms/adv:0.0032042369961024046 - perf/total_num_tokens:180180 - perf/time_per_step:28.295814787968993 - perf/throughput:795.9657698062213 (TaskRunner pid=2204243) Training Progress: 74%|███████▍ | 74/100 [44:03<13:41, 31.60s/it] (TaskRunner pid=2204243) step:75 - global_seqlen/min:19343 - global_seqlen/max:27582 - global_seqlen/minmax_diff:8239 - global_seqlen/balanced_min:23602 - global_seqlen/balanced_max:23612 - global_seqlen/mean:23607.625 - actor/entropy:0.1933431774377823 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44308218359947205 - training/rollout_probs_diff_mean:0.004375780466943979 - training/rollout_probs_diff_std:0.01272844523191452 - training/rollout_actor_probs_pearson_corr:0.9981529116630554 - self_distillation/success_group_fraction:0.53125 - self_distillation/success_sample_fraction:0.53125 - self_distillation/correct_sample_fraction:0.44921875 - self_distillation/feedback_available_fraction:0.55078125 - self_distillation/feedback_used_fraction:0.46875 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.999764621257782 - rollout_corr/rollout_is_ratio_fraction_high:3.3239157346542925e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.647831469308585e-05 - rollout_corr/rollout_is_max:2.1365387439727783 - rollout_corr/rollout_is_min:0.30766159296035767 - rollout_corr/rollout_is_std:0.03733116760849953 - rollout_corr/rollout_is_eff_sample_size:0.9986076694367252 - rollout_corr/rollout_is_seq_mean:0.9996073246002197 - rollout_corr/rollout_is_seq_std:0.004047990310937166 - rollout_corr/rollout_is_seq_max:1.0112133026123047 - rollout_corr/rollout_is_seq_min:0.9819174408912659 - rollout_corr/rollout_is_seq_max_deviation:0.01808255910873413 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1908653886057436) - rollout_corr/training_log_ppl:np.float64(0.1701184309204109) - rollout_corr/kl:np.float64(0.0014101347805564046) - rollout_corr/k3_kl:np.float64(0.001469150591314694) - rollout_corr/rollout_ppl:np.float64(1.1891403603367507) - rollout_corr/rollout_log_ppl:np.float64(0.16870829428808065) - rollout_corr/log_ppl_diff:np.float64(0.00141013663233025) - rollout_corr/log_ppl_abs_diff:np.float64(0.004865684175456408) - rollout_corr/log_ppl_diff_max:np.float64(0.03036680817604065) - rollout_corr/log_ppl_diff_min:np.float64(-0.018799765035510063) - rollout_corr/ppl_ratio:np.float64(1.0014335417654365) - rollout_corr/chi2_token:np.float64(0.0029236783739179373) - rollout_corr/chi2_seq:np.float64(0.41502745589241385) - actor/pg_loss:np.float64(0.0020789949921891093) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009934747354236606) - actor/ppo_kl:np.float64(0.0003302215436757905) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44921875) - self_distillation/token_reweight_w_mean:np.float64(452176.73929213313) - self_distillation/token_reweight_w_std:np.float64(3426749.02583762) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999885531142354) - self_distillation/token_reweight_w_clip_frac:np.float64(0.039197944483021274) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.56168482452631) - perf/mfu/actor:np.float64(0.04382752303846317) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.0500831604004) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:0 - critic/score/mean:0.44921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011513212695717812 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011513212695717812 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:117.51953125 - response_length/max:830.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:117.51953125 - response_length_non_aborted/max:830.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:620.21875 - prompt_length/max:1264.0 - prompt_length/min:380.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001175031065940857 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5387511476874352) - timing_s/agent_loop/generate_sequences/max:np.float64(4.762413555756211) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2340314836546895) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.762413555756211) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:720 - timing_s/agent_loop/slowest/response_length:829 - timing_s/gen:10.806806728243828 - timing_s/reward:0.07272440008819103 - timing_s/old_log_prob:2.2803980875760317 - timing_s/adv:0.6138144880533218 - timing_s/update_actor:15.485302640125155 - timing_s/step:29.348086923360825 - timing_s/stop_profile:0.00012330897152423859 - timing_per_token_ms/update_actor:0.08199312002014791 - timing_per_token_ms/gen:0.3592091317348788 - timing_per_token_ms/adv:0.0032500859788591706 - perf/total_num_tokens:188861 - perf/time_per_step:29.348086923360825 - perf/throughput:804.4008136424229 (TaskRunner pid=2204243) Training Progress: 75%|███████▌ | 75/100 [44:33<12:53, 30.94s/it] (TaskRunner pid=2204243) step:76 - global_seqlen/min:16194 - global_seqlen/max:24835 - global_seqlen/minmax_diff:8641 - global_seqlen/balanced_min:21983 - global_seqlen/balanced_max:21988 - global_seqlen/mean:21986.25 - actor/entropy:0.1794392466545105 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17766550183296204 - training/rollout_probs_diff_mean:0.004363747779279947 - training/rollout_probs_diff_std:0.012615064159035683 - training/rollout_actor_probs_pearson_corr:0.9981122612953186 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.375 - self_distillation/correct_sample_fraction:0.29296875 - self_distillation/feedback_available_fraction:0.70703125 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000191926956177 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6041940450668335 - rollout_corr/rollout_is_min:0.5898853540420532 - rollout_corr/rollout_is_std:0.03630068153142929 - rollout_corr/rollout_is_eff_sample_size:0.9986839946654996 - rollout_corr/rollout_is_seq_mean:0.999951958656311 - rollout_corr/rollout_is_seq_std:0.003703543683513999 - rollout_corr/rollout_is_seq_max:1.0117768049240112 - rollout_corr/rollout_is_seq_min:0.9891529679298401 - rollout_corr/rollout_is_seq_max_deviation:0.01177680492401123 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1879223263822496) - rollout_corr/training_log_ppl:np.float64(0.16701738664414734) - rollout_corr/kl:np.float64(0.0012198199164084045) - rollout_corr/k3_kl:np.float64(0.0011120540672919788) - rollout_corr/rollout_ppl:np.float64(1.186505121178925) - rollout_corr/rollout_log_ppl:np.float64(0.16579756570718018) - rollout_corr/log_ppl_diff:np.float64(0.001219820936967153) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037314699511625804) - rollout_corr/log_ppl_diff_max:np.float64(0.01797012984752655) - rollout_corr/log_ppl_diff_min:np.float64(-0.02454441785812378) - rollout_corr/ppl_ratio:np.float64(1.0012336145155132) - rollout_corr/chi2_token:np.float64(0.002160495612770319) - rollout_corr/chi2_seq:np.float64(0.40214965119957924) - actor/pg_loss:np.float64(0.002658140263520181) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007551920461992268) - actor/ppo_kl:np.float64(0.0005032984873878377) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.29296875) - self_distillation/token_reweight_w_mean:np.float64(536897.8268922176) - self_distillation/token_reweight_w_std:np.float64(4194788.892871575) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006419797427952) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03821892134146765) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.7488630264997482) - perf/mfu/actor:np.float64(0.040748089437767296) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.02930641174316) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:0 - critic/score/mean:0.29296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.29296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034345875028520823 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034345875028520823 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:113.1640625 - response_length/max:320.0 - response_length/min:46.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:113.1640625 - response_length_non_aborted/max:320.0 - response_length_non_aborted/min:46.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:573.90625 - prompt_length/max:928.0 - prompt_length/min:254.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001046936959028244 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6192603353410959) - timing_s/agent_loop/generate_sequences/max:np.float64(2.5699364002794027) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.3146521197049879) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.5699364002794027) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:716 - timing_s/agent_loop/slowest/response_length:320 - timing_s/gen:8.681765150278807 - timing_s/reward:0.08284247107803822 - timing_s/old_log_prob:2.2532359957695007 - timing_s/adv:0.576120350509882 - timing_s/update_actor:15.589469177648425 - timing_s/step:27.27052005007863 - timing_s/stop_profile:0.00011876411736011505 - timing_per_token_ms/update_actor:0.08863192437118896 - timing_per_token_ms/gen:0.2996812271411393 - timing_per_token_ms/adv:0.003275458243844914 - perf/total_num_tokens:175890 - perf/time_per_step:27.27052005007863 - perf/throughput:806.2277492187615 (TaskRunner pid=2204243) Training Progress: 76%|███████▌ | 76/100 [45:00<11:56, 29.85s/it] (TaskRunner pid=2204243) step:77 - global_seqlen/min:19382 - global_seqlen/max:28171 - global_seqlen/minmax_diff:8789 - global_seqlen/balanced_min:23484 - global_seqlen/balanced_max:23875 - global_seqlen/mean:23578.75 - actor/entropy:0.1791856437921524 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22026300430297852 - training/rollout_probs_diff_mean:0.0040746270678937435 - training/rollout_probs_diff_std:0.012526131235063076 - training/rollout_actor_probs_pearson_corr:0.9981771111488342 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.4375 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.625 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000211000442505 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.458053834037855e-05 - rollout_corr/rollout_is_max:1.712790846824646 - rollout_corr/rollout_is_min:0.4282894730567932 - rollout_corr/rollout_is_std:0.03662922978401184 - rollout_corr/rollout_is_eff_sample_size:0.9986602160845024 - rollout_corr/rollout_is_seq_mean:1.0003334283828735 - rollout_corr/rollout_is_seq_std:0.004470841959118843 - rollout_corr/rollout_is_seq_max:1.0139907598495483 - rollout_corr/rollout_is_seq_min:0.9837765097618103 - rollout_corr/rollout_is_seq_max_deviation:0.016223490238189697 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1862867465242743) - rollout_corr/training_log_ppl:np.float64(0.1658716400997946) - rollout_corr/kl:np.float64(0.000996613987428674) - rollout_corr/k3_kl:np.float64(0.0014110473224135944) - rollout_corr/rollout_ppl:np.float64(1.184983007144183) - rollout_corr/rollout_log_ppl:np.float64(0.164875024187495) - rollout_corr/log_ppl_diff:np.float64(0.0009966159122996032) - rollout_corr/log_ppl_abs_diff:np.float64(0.004256505169905722) - rollout_corr/log_ppl_diff_max:np.float64(0.032035380601882935) - rollout_corr/log_ppl_diff_min:np.float64(-0.01583130657672882) - rollout_corr/ppl_ratio:np.float64(1.0010148785077035) - rollout_corr/chi2_token:np.float64(0.003662694012746215) - rollout_corr/chi2_seq:np.float64(0.2312209859956056) - actor/pg_loss:np.float64(0.001877392758615315) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040770312853055657) - actor/ppo_kl:np.float64(0.0006109042182842472) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(284907.1694545767) - self_distillation/token_reweight_w_std:np.float64(2452854.050770485) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993604340124875) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03068707522470504) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.3736233711242676) - perf/mfu/actor:np.float64(0.04337399296973363) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.95214080810547) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005965142627246678 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0005965142627246678 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:112.9609375 - response_length/max:1175.0 - response_length/min:25.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:112.9609375 - response_length_non_aborted/max:1175.0 - response_length_non_aborted/min:25.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:623.875 - prompt_length/max:881.0 - prompt_length/min:341.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011204369366168976 - timing_s/agent_loop/generate_sequences/min:np.float64(0.43923107348382473) - timing_s/agent_loop/generate_sequences/max:np.float64(6.271953346207738) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1829631632645032) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.271953346207738) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:538 - timing_s/agent_loop/slowest/response_length:1175 - timing_s/gen:12.217393962666392 - timing_s/reward:0.07459044642746449 - timing_s/old_log_prob:2.2774024587124586 - timing_s/adv:0.5947028715163469 - timing_s/update_actor:15.595375303179026 - timing_s/step:30.846084889024496 - timing_s/stop_profile:0.0001142546534538269 - timing_per_token_ms/update_actor:0.08267706782155027 - timing_per_token_ms/gen:0.422484057080932 - timing_per_token_ms/adv:0.0031527480862871596 - perf/total_num_tokens:188630 - perf/time_per_step:30.846084889024496 - perf/throughput:764.4000878824553 (TaskRunner pid=2204243) Training Progress: 77%|███████▋ | 77/100 [45:31<11:33, 30.17s/it] (TaskRunner pid=2204243) step:78 - global_seqlen/min:18595 - global_seqlen/max:30959 - global_seqlen/minmax_diff:12364 - global_seqlen/balanced_min:23238 - global_seqlen/balanced_max:25363 - global_seqlen/mean:23551.5 - actor/entropy:0.17319826781749725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932348847389221 - training/rollout_probs_diff_mean:0.0038828898686915636 - training/rollout_probs_diff_std:0.011737355031073093 - training/rollout_actor_probs_pearson_corr:0.9983370304107666 - self_distillation/success_group_fraction:0.25 - self_distillation/success_sample_fraction:0.25 - self_distillation/correct_sample_fraction:0.25 - self_distillation/feedback_available_fraction:0.75 - self_distillation/feedback_used_fraction:0.75 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.000113606452942 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8666751384735107 - rollout_corr/rollout_is_min:0.5988900065422058 - rollout_corr/rollout_is_std:0.03391006961464882 - rollout_corr/rollout_is_eff_sample_size:0.9988516657990258 - rollout_corr/rollout_is_seq_mean:1.00006902217865 - rollout_corr/rollout_is_seq_std:0.003983448725193739 - rollout_corr/rollout_is_seq_max:1.0137455463409424 - rollout_corr/rollout_is_seq_min:0.988329291343689 - rollout_corr/rollout_is_seq_max_deviation:0.013745546340942383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1899874950759113) - rollout_corr/training_log_ppl:np.float64(0.16942024025775027) - rollout_corr/kl:np.float64(0.0006121923132553775) - rollout_corr/k3_kl:np.float64(0.0009290384985055355) - rollout_corr/rollout_ppl:np.float64(1.1892074048519135) - rollout_corr/rollout_log_ppl:np.float64(0.16880804611719213) - rollout_corr/log_ppl_diff:np.float64(0.000612194140558131) - rollout_corr/log_ppl_abs_diff:np.float64(0.0033112192031694576) - rollout_corr/log_ppl_diff_max:np.float64(0.03236883878707886) - rollout_corr/log_ppl_diff_min:np.float64(-0.015657424926757812) - rollout_corr/ppl_ratio:np.float64(1.0006236843764782) - rollout_corr/chi2_token:np.float64(0.0027093940880149603) - rollout_corr/chi2_seq:np.float64(0.18730456940829754) - actor/pg_loss:np.float64(0.0) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(4.156407724575928e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.25) - self_distillation/token_reweight_w_mean:np.float64(0.9999999930150807) - self_distillation/token_reweight_w_std:np.float64(0.0) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999999930150807) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.0) - perf/mfu/actor:np.float64(0.04272200474647107) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(145.97413635253906) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:0 - critic/score/mean:0.25 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.25 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0 - critic/advantages/max:0.0 - critic/advantages/min:0.0 - critic/returns/mean:0.0 - critic/returns/max:0.0 - critic/returns/min:0.0 - response_length/mean:133.890625 - response_length/max:3387.0 - response_length/min:39.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:133.890625 - response_length_non_aborted/max:3387.0 - response_length_non_aborted/min:39.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:602.09375 - prompt_length/max:1095.0 - prompt_length/min:355.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016301311552524567 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5600714478641748) - timing_s/agent_loop/generate_sequences/max:np.float64(17.028423136100173) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.3896567677002167) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.028423136100173) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:493 - timing_s/agent_loop/slowest/response_length:3387 - timing_s/gen:23.34828830882907 - timing_s/reward:0.07651215977966785 - timing_s/old_log_prob:2.481689067557454 - timing_s/adv:0.6034424118697643 - timing_s/update_actor:15.874576034024358 - timing_s/step:42.47145488485694 - timing_s/stop_profile:0.00012134946882724762 - timing_per_token_ms/update_actor:0.0842545911832811 - timing_per_token_ms/gen:0.6811847446851753 - timing_per_token_ms/adv:0.0032027812022045535 - perf/total_num_tokens:188412 - perf/time_per_step:42.47145488485694 - perf/throughput:554.5253880247275 (TaskRunner pid=2204243) Training Progress: 78%|███████▊ | 78/100 [46:13<12:25, 33.87s/it] (TaskRunner pid=2204243) step:79 - global_seqlen/min:17639 - global_seqlen/max:30699 - global_seqlen/minmax_diff:13060 - global_seqlen/balanced_min:20371 - global_seqlen/balanced_max:27810 - global_seqlen/mean:21302.375 - actor/entropy:0.12300966680049896 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3583569824695587 - training/rollout_probs_diff_mean:0.0030804690904915333 - training/rollout_probs_diff_std:0.01145958062261343 - training/rollout_actor_probs_pearson_corr:0.9979829788208008 - self_distillation/success_group_fraction:0.46875 - self_distillation/success_sample_fraction:0.46484375 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.578125 - self_distillation/feedback_used_fraction:0.53125 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9998621940612793 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:8.833142055664212e-05 - rollout_corr/rollout_is_max:1.6468292474746704 - rollout_corr/rollout_is_min:0.39613887667655945 - rollout_corr/rollout_is_std:0.03225614130496979 - rollout_corr/rollout_is_eff_sample_size:0.9989605037882843 - rollout_corr/rollout_is_seq_mean:0.9999085068702698 - rollout_corr/rollout_is_seq_std:0.00402069604024291 - rollout_corr/rollout_is_seq_max:1.0173876285552979 - rollout_corr/rollout_is_seq_min:0.9876928925514221 - rollout_corr/rollout_is_seq_max_deviation:0.01738762855529785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1608892101794481) - rollout_corr/training_log_ppl:np.float64(0.14488916451955447) - rollout_corr/kl:np.float64(0.0006675544648047094) - rollout_corr/k3_kl:np.float64(0.0008047550561727235) - rollout_corr/rollout_ppl:np.float64(1.1600501500070095) - rollout_corr/rollout_log_ppl:np.float64(0.14422160830145003) - rollout_corr/log_ppl_diff:np.float64(0.000667556218104437) - rollout_corr/log_ppl_abs_diff:np.float64(0.003304619516711682) - rollout_corr/log_ppl_diff_max:np.float64(0.018346011638641357) - rollout_corr/log_ppl_diff_min:np.float64(-0.01784604787826538) - rollout_corr/ppl_ratio:np.float64(1.0006778293754905) - rollout_corr/chi2_token:np.float64(0.001853058347478509) - rollout_corr/chi2_seq:np.float64(0.27634171093814075) - actor/pg_loss:np.float64(0.0005190407391637564) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005710039240511833) - actor/ppo_kl:np.float64(-9.167133173804132e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(144093.8724760895) - self_distillation/token_reweight_w_std:np.float64(1129560.9346376678) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004591008182615) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02232855127658695) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.3326127380132675) - perf/mfu/actor:np.float64(0.0365605301843239) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.15404224395752) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.20817507803440094 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.20817507803440094 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:132.66796875 - response_length/max:8192.0 - response_length/min:40.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:132.66796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:533.03125 - prompt_length/max:923.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.981170296669006e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5808947421610355) - timing_s/agent_loop/generate_sequences/max:np.float64(46.98064439184964) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.3612120854304521) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(46.98064439184964) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:547 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:52.867963548749685 - timing_s/reward:0.07146033644676208 - timing_s/old_log_prob:2.543733850121498 - timing_s/adv:0.566445691511035 - timing_s/update_actor:17.2258820515126 - timing_s/step:73.36468144506216 - timing_s/stop_profile:0.00010889023542404175 - timing_per_token_ms/update_actor:0.1010795864986451 - timing_per_token_ms/gen:1.556634088530156 - timing_per_token_ms/adv:0.003323841188547257 - perf/total_num_tokens:170419 - perf/time_per_step:73.36468144506216 - perf/throughput:290.3628091938477 (TaskRunner pid=2204243) Training Progress: 79%|███████▉ | 79/100 [47:27<16:00, 45.74s/it] (TaskRunner pid=2204243) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2204243) validation generation end (TaskRunner pid=2204243) [prompt] user (TaskRunner pid=2204243) Your task is to answer the user's question using available tools. (TaskRunner pid=2204243) You have access to the following tools: (TaskRunner pid=2204243) Name: Axolotl (TaskRunner pid=2204243) Description: Collection of axolotl pictures and facts (TaskRunner pid=2204243) Documentation: (TaskRunner pid=2204243) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2204243) Parameters: {} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{url, source, description} (TaskRunner pid=2204243) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2204243) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2204243) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2204243) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Array[Object{fact, source}] (TaskRunner pid=2204243) (TaskRunner pid=2204243) Use the following format: (TaskRunner pid=2204243) Thought: you should always think about what to do (TaskRunner pid=2204243) Action: the action to take, should be one of the tool names. (TaskRunner pid=2204243) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2204243) (TaskRunner pid=2204243) Begin! (TaskRunner pid=2204243) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2204243) assistant (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: I need to find a random axolotl image. The user is asking for a random picture, so I should use the getRandomAxolotlImage tool. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2204655) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2204243) step:80 - global_seqlen/min:19917 - global_seqlen/max:30244 - global_seqlen/minmax_diff:10327 - global_seqlen/balanced_min:24488 - global_seqlen/balanced_max:24501 - global_seqlen/mean:24496.875 - actor/entropy:0.17567157745361328 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4042512774467468 - training/rollout_probs_diff_mean:0.003904949873685837 - training/rollout_probs_diff_std:0.012644829228520393 - training/rollout_actor_probs_pearson_corr:0.99812912940979 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.37109375 - self_distillation/correct_sample_fraction:0.26953125 - self_distillation/feedback_available_fraction:0.73046875 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.0000805854797363 - rollout_corr/rollout_is_ratio_fraction_high:0.00014569188351742923 - rollout_corr/rollout_is_ratio_fraction_low:2.913837852247525e-05 - rollout_corr/rollout_is_max:2.4427950382232666 - rollout_corr/rollout_is_min:0.484937459230423 - rollout_corr/rollout_is_std:0.037461068481206894 - rollout_corr/rollout_is_eff_sample_size:0.9985986348236627 - rollout_corr/rollout_is_seq_mean:1.000091552734375 - rollout_corr/rollout_is_seq_std:0.003862826619297266 - rollout_corr/rollout_is_seq_max:1.0156095027923584 - rollout_corr/rollout_is_seq_min:0.9902510046958923 - rollout_corr/rollout_is_seq_max_deviation:0.015609502792358398 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1763388719409704) - rollout_corr/training_log_ppl:np.float64(0.1576913361714105) - rollout_corr/kl:np.float64(0.0010720672920108143) - rollout_corr/k3_kl:np.float64(0.0009125643200906808) - rollout_corr/rollout_ppl:np.float64(1.175038946326822) - rollout_corr/rollout_log_ppl:np.float64(0.15661926797110937) - rollout_corr/log_ppl_diff:np.float64(0.0010720682003011461) - rollout_corr/log_ppl_abs_diff:np.float64(0.0031145907159952912) - rollout_corr/log_ppl_diff_max:np.float64(0.015575125813484192) - rollout_corr/log_ppl_diff_min:np.float64(-0.012317121028900146) - rollout_corr/ppl_ratio:np.float64(1.0010812666732818) - rollout_corr/chi2_token:np.float64(0.0015714741311967373) - rollout_corr/chi2_seq:np.float64(0.2864194030407816) - actor/pg_loss:np.float64(0.0021724561229348183) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001126089211084036) - actor/ppo_kl:np.float64(0.0004655882661346311) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.26953125) - self_distillation/token_reweight_w_mean:np.float64(323723.9415737605) - self_distillation/token_reweight_w_std:np.float64(2843199.1033894382) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000941570615396) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03495625717914663) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.4897511675953865) - perf/mfu/actor:np.float64(0.04451835617570689) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.08455276489258) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.5928308823529411) - val-aux/tooluse/reward/std@16:np.float64(0.03677377920354384) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6079411764705882) - val-aux/tooluse/reward/best@2/std:np.float64(0.029904280081493432) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5772352941176471) - val-aux/tooluse/reward/worst@2/std:np.float64(0.033138138814584515) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5920588235294117) - val-aux/tooluse/reward/maj@2/std:np.float64(0.03660220780839206) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6202352941176471) - val-aux/tooluse/reward/best@4/std:np.float64(0.01901702261333825) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5629852941176471) - val-aux/tooluse/reward/worst@4/std:np.float64(0.02422065970246017) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5942352941176471) - val-aux/tooluse/reward/maj@4/std:np.float64(0.0314536949620504) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6267058823529412) - val-aux/tooluse/reward/best@8/std:np.float64(0.010388173629343581) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5540735294117647) - val-aux/tooluse/reward/worst@8/std:np.float64(0.014092800427817442) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5963382352941177) - val-aux/tooluse/reward/maj@8/std:np.float64(0.026000548995860767) - val-aux/tooluse/reward/best@16/mean:np.float64(0.630529411764706) - val-aux/tooluse/reward/best@16/std:np.float64(0.005294767203959937) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5494558823529412) - val-aux/tooluse/reward/worst@16/std:np.float64(0.008829659953105173) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5985735294117647) - val-aux/tooluse/reward/maj@16/std:np.float64(0.02063114285488165) - val-aux/tooluse/score/mean@16:np.float64(0.5928308823529411) - val-aux/tooluse/score/std@16:np.float64(0.03677377920354384) - val-aux/tooluse/score/best@2/mean:np.float64(0.6079411764705882) - val-aux/tooluse/score/best@2/std:np.float64(0.029904280081493432) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5772352941176471) - val-aux/tooluse/score/worst@2/std:np.float64(0.033138138814584515) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5920588235294117) - val-aux/tooluse/score/maj@2/std:np.float64(0.03660220780839206) - val-aux/tooluse/score/best@4/mean:np.float64(0.6202352941176471) - val-aux/tooluse/score/best@4/std:np.float64(0.01901702261333825) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5629852941176471) - val-aux/tooluse/score/worst@4/std:np.float64(0.02422065970246017) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5942352941176471) - val-aux/tooluse/score/maj@4/std:np.float64(0.0314536949620504) - val-aux/tooluse/score/best@8/mean:np.float64(0.6267058823529412) - val-aux/tooluse/score/best@8/std:np.float64(0.010388173629343581) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5540735294117647) - val-aux/tooluse/score/worst@8/std:np.float64(0.014092800427817442) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5963382352941177) - val-aux/tooluse/score/maj@8/std:np.float64(0.026000548995860767) - val-aux/tooluse/score/best@16/mean:np.float64(0.630529411764706) - val-aux/tooluse/score/best@16/std:np.float64(0.005294767203959937) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5494558823529412) - val-aux/tooluse/score/worst@16/std:np.float64(0.008829659953105173) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5985735294117647) - val-aux/tooluse/score/maj@16/std:np.float64(0.02063114285488165) - val-core/tooluse/acc/mean@16:np.float64(0.5928308823529411) - val-aux/tooluse/acc/std@16:np.float64(0.03677377920354384) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6079411764705882) - val-aux/tooluse/acc/best@2/std:np.float64(0.029904280081493432) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5772352941176471) - val-aux/tooluse/acc/worst@2/std:np.float64(0.033138138814584515) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5920588235294117) - val-aux/tooluse/acc/maj@2/std:np.float64(0.03660220780839206) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6202352941176471) - val-aux/tooluse/acc/best@4/std:np.float64(0.01901702261333825) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5629852941176471) - val-aux/tooluse/acc/worst@4/std:np.float64(0.02422065970246017) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5942352941176471) - val-aux/tooluse/acc/maj@4/std:np.float64(0.0314536949620504) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6267058823529412) - val-aux/tooluse/acc/best@8/std:np.float64(0.010388173629343581) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5540735294117647) - val-aux/tooluse/acc/worst@8/std:np.float64(0.014092800427817442) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5963382352941177) - val-aux/tooluse/acc/maj@8/std:np.float64(0.026000548995860767) - val-core/tooluse/acc/best@16/mean:np.float64(0.630529411764706) - val-core/tooluse/acc/best@16/std:np.float64(0.005294767203959937) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5494558823529412) - val-aux/tooluse/acc/worst@16/std:np.float64(0.008829659953105173) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5985735294117647) - val-core/tooluse/acc/maj@16/std:np.float64(0.02063114285488165) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:0 - critic/score/mean:0.26953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.26953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004399895202368498 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.004399895202368498 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:134.05859375 - response_length/max:995.0 - response_length/min:47.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:134.05859375 - response_length_non_aborted/max:995.0 - response_length_non_aborted/min:47.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:631.46875 - prompt_length/max:1505.0 - prompt_length/min:302.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.667765021324158e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6877212207764387) - timing_s/agent_loop/generate_sequences/max:np.float64(5.826155092567205) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.439073183319124) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.826155092567205) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:415 - timing_s/agent_loop/slowest/response_length:990 - timing_s/gen:11.944609822705388 - timing_s/reward:0.07720724493265152 - timing_s/old_log_prob:2.432076556608081 - timing_s/adv:0.6790953688323498 - timing_s/update_actor:15.876319533213973 - timing_s/step:31.100267726927996 - timing_s/testing:69.14823789894581 - timing_s/save_checkpoint:6.5176573805511 - timing_s/stop_profile:0.00011819414794445038 - timing_per_token_ms/update_actor:0.08101196343010064 - timing_per_token_ms/gen:0.34804655796221884 - timing_per_token_ms/adv:0.0034652142815785165 - perf/total_num_tokens:195975 - perf/time_per_step:31.100267726927996 - perf/throughput:787.6740874095278 (TaskRunner pid=2204243) Training Progress: 80%|████████ | 80/100 [49:14<21:21, 64.06s/it] (TaskRunner pid=2204243) step:81 - global_seqlen/min:17928 - global_seqlen/max:28839 - global_seqlen/minmax_diff:10911 - global_seqlen/balanced_min:22463 - global_seqlen/balanced_max:22466 - global_seqlen/mean:22464.375 - actor/entropy:0.15417571365833282 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2647586762905121 - training/rollout_probs_diff_mean:0.003974084742367268 - training/rollout_probs_diff_std:0.012667772360146046 - training/rollout_actor_probs_pearson_corr:0.9978932738304138 - self_distillation/success_group_fraction:0.5 - self_distillation/success_sample_fraction:0.49609375 - self_distillation/correct_sample_fraction:0.4453125 - self_distillation/feedback_available_fraction:0.5546875 - self_distillation/feedback_used_fraction:0.5 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9995285272598267 - rollout_corr/rollout_is_ratio_fraction_high:3.7409747164929286e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.7409747164929286e-05 - rollout_corr/rollout_is_max:2.654141902923584 - rollout_corr/rollout_is_min:0.4818713068962097 - rollout_corr/rollout_is_std:0.035701341927051544 - rollout_corr/rollout_is_eff_sample_size:0.9987257288127562 - rollout_corr/rollout_is_seq_mean:0.9995540976524353 - rollout_corr/rollout_is_seq_std:0.0036673536524176598 - rollout_corr/rollout_is_seq_max:1.0117744207382202 - rollout_corr/rollout_is_seq_min:0.9901750087738037 - rollout_corr/rollout_is_seq_max_deviation:0.011774420738220215 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1546420948579907) - rollout_corr/training_log_ppl:np.float64(0.14085338862787466) - rollout_corr/kl:np.float64(0.0012604825372477535) - rollout_corr/k3_kl:np.float64(0.0008642361134789667) - rollout_corr/rollout_ppl:np.float64(1.153144343290478) - rollout_corr/rollout_log_ppl:np.float64(0.1395929056161549) - rollout_corr/log_ppl_diff:np.float64(0.0012604830117197707) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034368490596534684) - rollout_corr/log_ppl_diff_max:np.float64(0.01777172088623047) - rollout_corr/log_ppl_diff_min:np.float64(-0.014761913567781448) - rollout_corr/ppl_ratio:np.float64(1.0012704834807664) - rollout_corr/chi2_token:np.float64(0.0008417221251875162) - rollout_corr/chi2_seq:np.float64(0.10313086141832173) - actor/pg_loss:np.float64(0.0012402702122926712) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003762469705179683) - actor/ppo_kl:np.float64(0.00017385742377484803) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4453125) - self_distillation/token_reweight_w_mean:np.float64(314717.4119199689) - self_distillation/token_reweight_w_std:np.float64(2645187.100312316) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997840190771967) - self_distillation/token_reweight_w_clip_frac:np.float64(0.028456083440687507) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.2973490469157696) - perf/mfu/actor:np.float64(0.0417033663470472) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.3630027770996) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:0 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001557180774398148 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001557180774398148 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:104.41796875 - response_length/max:296.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:104.41796875 - response_length_non_aborted/max:296.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:597.59375 - prompt_length/max:1127.0 - prompt_length/min:252.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023244135081768036 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5206291191279888) - timing_s/agent_loop/generate_sequences/max:np.float64(2.263333672657609) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1677860180352582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.263333672657609) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:453 - timing_s/agent_loop/slowest/response_length:296 - timing_s/gen:8.04948065802455 - timing_s/reward:0.07031497173011303 - timing_s/old_log_prob:2.4710217490792274 - timing_s/adv:0.6364662386476994 - timing_s/update_actor:15.254998929798603 - timing_s/step:26.5776224154979 - timing_s/stop_profile:0.0001115705817937851 - timing_per_token_ms/update_actor:0.0848843943454837 - timing_per_token_ms/gen:0.30112905084076724 - timing_per_token_ms/adv:0.003541530972081904 - perf/total_num_tokens:179715 - perf/time_per_step:26.5776224154979 - perf/throughput:845.2364417255251 (TaskRunner pid=2204243) Training Progress: 81%|████████ | 81/100 [49:40<16:43, 52.83s/it] (TaskRunner pid=2204243) step:82 - global_seqlen/min:18131 - global_seqlen/max:26066 - global_seqlen/minmax_diff:7935 - global_seqlen/balanced_min:22562 - global_seqlen/balanced_max:22734 - global_seqlen/mean:22584.125 - actor/entropy:0.16985785961151123 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3016318678855896 - training/rollout_probs_diff_mean:0.003941318020224571 - training/rollout_probs_diff_std:0.012615574523806572 - training/rollout_actor_probs_pearson_corr:0.9981062412261963 - self_distillation/success_group_fraction:0.40625 - self_distillation/success_sample_fraction:0.40625 - self_distillation/correct_sample_fraction:0.38671875 - self_distillation/feedback_available_fraction:0.61328125 - self_distillation/feedback_used_fraction:0.59375 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9999383687973022 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6127465963363647 - rollout_corr/rollout_is_min:0.5126444697380066 - rollout_corr/rollout_is_std:0.035523928701877594 - rollout_corr/rollout_is_eff_sample_size:0.998739403152397 - rollout_corr/rollout_is_seq_mean:0.9997885227203369 - rollout_corr/rollout_is_seq_std:0.0036332320887595415 - rollout_corr/rollout_is_seq_max:1.0100631713867188 - rollout_corr/rollout_is_seq_min:0.9866380095481873 - rollout_corr/rollout_is_seq_max_deviation:0.013361990451812744 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1710331719368696) - rollout_corr/training_log_ppl:np.float64(0.15347579008084722) - rollout_corr/kl:np.float64(0.0011650914289838) - rollout_corr/k3_kl:np.float64(0.0008884033978944217) - rollout_corr/rollout_ppl:np.float64(1.1696686162613332) - rollout_corr/rollout_log_ppl:np.float64(0.1523106970853405) - rollout_corr/log_ppl_diff:np.float64(0.0011650929955067113) - rollout_corr/log_ppl_abs_diff:np.float64(0.003433958365349099) - rollout_corr/log_ppl_diff_max:np.float64(0.02052687108516693) - rollout_corr/log_ppl_diff_min:np.float64(-0.015586599707603455) - rollout_corr/ppl_ratio:np.float64(1.0011764424853027) - rollout_corr/chi2_token:np.float64(0.0011556970421224833) - rollout_corr/chi2_seq:np.float64(0.27680961857549846) - actor/pg_loss:np.float64(0.0003503011539578438) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(8.499710747855715e-05) - actor/ppo_kl:np.float64(0.0003015789514329015) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.38671875) - self_distillation/token_reweight_w_mean:np.float64(79386.08817859669) - self_distillation/token_reweight_w_std:np.float64(481021.56727622903) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000321809202433) - self_distillation/token_reweight_w_clip_frac:np.float64(0.009597325028153136) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.22535676509141922) - perf/mfu/actor:np.float64(0.04182800160089407) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.27236557006836) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:0 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-2.2143489331938326e-05 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:-2.2143489331938326e-05 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:110.25390625 - response_length/max:1044.0 - response_length/min:36.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:110.25390625 - response_length_non_aborted/max:1044.0 - response_length_non_aborted/min:36.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:595.5 - prompt_length/max:1112.0 - prompt_length/min:343.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013537146151065826 - timing_s/agent_loop/generate_sequences/min:np.float64(0.43506602011621) - timing_s/agent_loop/generate_sequences/max:np.float64(5.568701019510627) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1628019850322744) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.568701019510627) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:690 - timing_s/agent_loop/slowest/response_length:1044 - timing_s/gen:11.387402337044477 - timing_s/reward:0.07148287072777748 - timing_s/old_log_prob:2.315675027668476 - timing_s/adv:0.6052231229841709 - timing_s/update_actor:15.502152184024453 - timing_s/step:29.96822457201779 - timing_s/stop_profile:0.00012261606752872467 - timing_per_token_ms/update_actor:0.08580226256288684 - timing_per_token_ms/gen:0.40345092425312584 - timing_per_token_ms/adv:0.0033498260558255575 - perf/total_num_tokens:180673 - perf/time_per_step:29.96822457201779 - perf/throughput:753.6023679256415 (TaskRunner pid=2204243) Training Progress: 82%|████████▏ | 82/100 [50:10<13:47, 45.99s/it] (TaskRunner pid=2204243) step:83 - global_seqlen/min:15129 - global_seqlen/max:27802 - global_seqlen/minmax_diff:12673 - global_seqlen/balanced_min:20568 - global_seqlen/balanced_max:20601 - global_seqlen/mean:20572.625 - actor/entropy:0.15253762900829315 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30164164304733276 - training/rollout_probs_diff_mean:0.00375850941054523 - training/rollout_probs_diff_std:0.012769849970936775 - training/rollout_actor_probs_pearson_corr:0.9978535771369934 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.48046875 - self_distillation/feedback_available_fraction:0.51953125 - self_distillation/feedback_used_fraction:0.40625 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:1.0001826286315918 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00010588359873509035 - rollout_corr/rollout_is_max:1.9181381464004517 - rollout_corr/rollout_is_min:0.23636506497859955 - rollout_corr/rollout_is_std:0.03759131580591202 - rollout_corr/rollout_is_eff_sample_size:0.9985894815008278 - rollout_corr/rollout_is_seq_mean:1.0002963542938232 - rollout_corr/rollout_is_seq_std:0.004574134014546871 - rollout_corr/rollout_is_seq_max:1.0248439311981201 - rollout_corr/rollout_is_seq_min:0.9872625470161438 - rollout_corr/rollout_is_seq_max_deviation:0.024843931198120117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.154557657893747) - rollout_corr/training_log_ppl:np.float64(0.13714414653077256) - rollout_corr/kl:np.float64(0.0011963266094241476) - rollout_corr/k3_kl:np.float64(0.00119553365255598) - rollout_corr/rollout_ppl:np.float64(1.1531633012928069) - rollout_corr/rollout_log_ppl:np.float64(0.13594781965730363) - rollout_corr/log_ppl_diff:np.float64(0.0011963268734689336) - rollout_corr/log_ppl_abs_diff:np.float64(0.0039459568906750064) - rollout_corr/log_ppl_diff_max:np.float64(0.033453747630119324) - rollout_corr/log_ppl_diff_min:np.float64(-0.02310211956501007) - rollout_corr/ppl_ratio:np.float64(1.00121411238797) - rollout_corr/chi2_token:np.float64(0.00240033189766109) - rollout_corr/chi2_seq:np.float64(0.31055444525554776) - actor/pg_loss:np.float64(0.0024829749017953873) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011531597174325725) - actor/ppo_kl:np.float64(0.000661982456762189) - actor/pg_clipfrac_lower:np.float64(4.292582525522448e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48046875) - self_distillation/token_reweight_w_mean:np.float64(698139.1198134948) - self_distillation/token_reweight_w_std:np.float64(5344575.876064017) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010894627775997) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02956085276673548) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.6029211431741714) - perf/mfu/actor:np.float64(0.03854349928456097) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.21487426757812) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:0 - critic/score/mean:0.48046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00398387061432004 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00398387061432004 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:110.67578125 - response_length/max:815.0 - response_length/min:35.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:110.67578125 - response_length_non_aborted/max:815.0 - response_length_non_aborted/min:35.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:532.21875 - prompt_length/max:974.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011205486953258514 - timing_s/agent_loop/generate_sequences/min:np.float64(0.362441748380661) - timing_s/agent_loop/generate_sequences/max:np.float64(4.812540667131543) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1279527979932027) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.812540667131543) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:681 - timing_s/agent_loop/slowest/response_length:815 - timing_s/gen:10.983776589855552 - timing_s/reward:0.06510071642696857 - timing_s/old_log_prob:2.2893887255340815 - timing_s/adv:0.5861021522432566 - timing_s/update_actor:15.324710292741656 - timing_s/step:29.348286448046565 - timing_s/stop_profile:0.00011092610657215118 - timing_per_token_ms/update_actor:0.09311348389389817 - timing_per_token_ms/gen:0.38766726396271317 - timing_per_token_ms/adv:0.00356117748855127 - perf/total_num_tokens:164581 - perf/time_per_step:29.348286448046565 - perf/throughput:700.9821522772183 (TaskRunner pid=2204243) Training Progress: 83%|████████▎ | 83/100 [50:40<11:37, 41.01s/it] (TaskRunner pid=2204243) step:84 - global_seqlen/min:17575 - global_seqlen/max:28636 - global_seqlen/minmax_diff:11061 - global_seqlen/balanced_min:24048 - global_seqlen/balanced_max:24052 - global_seqlen/mean:24050.0 - actor/entropy:0.12998901307582855 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2449260950088501 - training/rollout_probs_diff_mean:0.003552412847056985 - training/rollout_probs_diff_std:0.012100638821721077 - training/rollout_actor_probs_pearson_corr:0.9977862238883972 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.37109375 - self_distillation/correct_sample_fraction:0.30078125 - self_distillation/feedback_available_fraction:0.69921875 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9998891353607178 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.114399340935051e-05 - rollout_corr/rollout_is_max:1.695290207862854 - rollout_corr/rollout_is_min:0.46146276593208313 - rollout_corr/rollout_is_std:0.03329634293913841 - rollout_corr/rollout_is_eff_sample_size:0.9988923434521209 - rollout_corr/rollout_is_seq_mean:0.99997878074646 - rollout_corr/rollout_is_seq_std:0.0034572496078908443 - rollout_corr/rollout_is_seq_max:1.0122427940368652 - rollout_corr/rollout_is_seq_min:0.9867332577705383 - rollout_corr/rollout_is_seq_max_deviation:0.01326674222946167 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1450771004892886) - rollout_corr/training_log_ppl:np.float64(0.1328997489908943) - rollout_corr/kl:np.float64(0.002724082794657079) - rollout_corr/k3_kl:np.float64(0.0020446916239897917) - rollout_corr/rollout_ppl:np.float64(1.1419600611552596) - rollout_corr/rollout_log_ppl:np.float64(0.13017566601047292) - rollout_corr/log_ppl_diff:np.float64(0.002724082994973287) - rollout_corr/log_ppl_abs_diff:np.float64(0.004877446306636557) - rollout_corr/log_ppl_diff_max:np.float64(0.12481576949357986) - rollout_corr/log_ppl_diff_min:np.float64(-0.011138498783111572) - rollout_corr/ppl_ratio:np.float64(1.0027916976250708) - rollout_corr/chi2_token:np.float64(0.0009019074495881796) - rollout_corr/chi2_seq:np.float64(0.16505708848126233) - actor/pg_loss:np.float64(0.0012974075507372618) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006074218690628186) - actor/ppo_kl:np.float64(0.0021287094028394904) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.30078125) - self_distillation/token_reweight_w_mean:np.float64(487399.21458006324) - self_distillation/token_reweight_w_std:np.float64(3658836.1558035817) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998655966483057) - self_distillation/token_reweight_w_clip_frac:np.float64(0.018986002745805308) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.4852507822215557) - perf/mfu/actor:np.float64(0.04432775262921944) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.37873077392578) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:0 - critic/score/mean:0.30078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.30078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009373221546411514 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.009373221546411514 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:109.8125 - response_length/max:395.0 - response_length/min:44.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:109.8125 - response_length_non_aborted/max:395.0 - response_length_non_aborted/min:44.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:641.75 - prompt_length/max:1262.0 - prompt_length/min:294.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001328699290752411 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6081859972327948) - timing_s/agent_loop/generate_sequences/max:np.float64(2.8203831240534782) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2004522403658484) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.8203831240534782) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:739 - timing_s/agent_loop/slowest/response_length:395 - timing_s/gen:8.652177827432752 - timing_s/reward:0.07659208215773106 - timing_s/old_log_prob:2.269890945404768 - timing_s/adv:0.6486167665570974 - timing_s/update_actor:15.732981575652957 - timing_s/step:27.466560419648886 - timing_s/stop_profile:5.258247256278992e-05 - timing_per_token_ms/update_actor:0.08177225351170976 - timing_per_token_ms/gen:0.30777524997982186 - timing_per_token_ms/adv:0.003371189015369529 - perf/total_num_tokens:192400 - perf/time_per_step:27.466560419648886 - perf/throughput:875.610183166409 (TaskRunner pid=2204243) Training Progress: 84%|████████▍ | 84/100 [51:07<09:51, 36.96s/it] (TaskRunner pid=2204243) step:85 - global_seqlen/min:19844 - global_seqlen/max:24592 - global_seqlen/minmax_diff:4748 - global_seqlen/balanced_min:21810 - global_seqlen/balanced_max:21812 - global_seqlen/mean:21811.5 - actor/entropy:0.17794406414031982 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24004700779914856 - training/rollout_probs_diff_mean:0.004227302502840757 - training/rollout_probs_diff_std:0.012926393188536167 - training/rollout_actor_probs_pearson_corr:0.9980298280715942 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.43359375 - self_distillation/correct_sample_fraction:0.38671875 - self_distillation/feedback_available_fraction:0.61328125 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.0001064538955688 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.816565990447998 - rollout_corr/rollout_is_min:0.5352625846862793 - rollout_corr/rollout_is_std:0.03662272170186043 - rollout_corr/rollout_is_eff_sample_size:0.9986608105352269 - rollout_corr/rollout_is_seq_mean:1.0000821352005005 - rollout_corr/rollout_is_seq_std:0.003729208605363965 - rollout_corr/rollout_is_seq_max:1.0140963792800903 - rollout_corr/rollout_is_seq_min:0.9891149401664734 - rollout_corr/rollout_is_seq_max_deviation:0.014096379280090332 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1706541613675654) - rollout_corr/training_log_ppl:np.float64(0.1515700320378528) - rollout_corr/kl:np.float64(0.001192973389480656) - rollout_corr/k3_kl:np.float64(0.0010574734194124247) - rollout_corr/rollout_ppl:np.float64(1.169240111950785) - rollout_corr/rollout_log_ppl:np.float64(0.15037705790746259) - rollout_corr/log_ppl_diff:np.float64(0.001192974130390212) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034889425587607548) - rollout_corr/log_ppl_diff_max:np.float64(0.06276441365480423) - rollout_corr/log_ppl_diff_min:np.float64(-0.01376381516456604) - rollout_corr/ppl_ratio:np.float64(1.0012129179667681) - rollout_corr/chi2_token:np.float64(0.0014934807550162077) - rollout_corr/chi2_seq:np.float64(0.1903841698076576) - actor/pg_loss:np.float64(0.0005256927106529474) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00039427635238098446) - actor/ppo_kl:np.float64(0.0006357589195156166) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.38671875) - self_distillation/token_reweight_w_mean:np.float64(421217.8233769045) - self_distillation/token_reweight_w_std:np.float64(2723633.8979068603) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008101854473352) - self_distillation/token_reweight_w_clip_frac:np.float64(0.010901796689722687) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.2588997967541218) - perf/mfu/actor:np.float64(0.04031768594155601) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.3627052307129) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:0 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0031290461774915457 - critic/advantages/max:0.875 - critic/advantages/min:-0.25 - critic/returns/mean:0.0031290461774915457 - critic/returns/max:0.875 - critic/returns/min:-0.25 - response_length/mean:108.609375 - response_length/max:409.0 - response_length/min:43.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:108.609375 - response_length_non_aborted/max:409.0 - response_length_non_aborted/min:43.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:573.0 - prompt_length/max:984.0 - prompt_length/min:190.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.5478885769844055e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.53044736944139) - timing_s/agent_loop/generate_sequences/max:np.float64(2.9406584091484547) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.18859201840678) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.9406584091484547) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:736 - timing_s/agent_loop/slowest/response_length:409 - timing_s/gen:8.679400341585279 - timing_s/reward:0.07171659171581268 - timing_s/old_log_prob:2.2620538473129272 - timing_s/adv:0.6048305444419384 - timing_s/update_actor:15.313473671674728 - timing_s/step:27.017788216471672 - timing_s/stop_profile:0.00012122839689254761 - timing_per_token_ms/update_actor:0.08776031950848594 - timing_per_token_ms/gen:0.31216372973619905 - timing_per_token_ms/adv:0.0034662365291356534 - perf/total_num_tokens:174492 - perf/time_per_step:27.017788216471672 - perf/throughput:807.3014646958553 (TaskRunner pid=2204243) Training Progress: 85%|████████▌ | 85/100 [51:34<08:29, 33.99s/it] (TaskRunner pid=2204243) step:86 - global_seqlen/min:16144 - global_seqlen/max:60957 - global_seqlen/minmax_diff:44813 - global_seqlen/balanced_min:21653 - global_seqlen/balanced_max:28914 - global_seqlen/mean:25328.5 - actor/entropy:0.09163974970579147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.369753360748291 - training/rollout_probs_diff_mean:0.0024442446883767843 - training/rollout_probs_diff_std:0.011506135575473309 - training/rollout_actor_probs_pearson_corr:0.9973499178886414 - self_distillation/success_group_fraction:0.5 - self_distillation/success_sample_fraction:0.4921875 - self_distillation/correct_sample_fraction:0.4453125 - self_distillation/feedback_available_fraction:0.5546875 - self_distillation/feedback_used_fraction:0.5 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:1.0001195669174194 - rollout_corr/rollout_is_ratio_fraction_high:0.0001009897023322992 - rollout_corr/rollout_is_ratio_fraction_low:0.0001683161681285128 - rollout_corr/rollout_is_max:3.9662251472473145 - rollout_corr/rollout_is_min:0.30373305082321167 - rollout_corr/rollout_is_std:0.031024117022752762 - rollout_corr/rollout_is_eff_sample_size:0.9990384297035424 - rollout_corr/rollout_is_seq_mean:0.9999980330467224 - rollout_corr/rollout_is_seq_std:0.003821984166279435 - rollout_corr/rollout_is_seq_max:1.013857364654541 - rollout_corr/rollout_is_seq_min:0.9863834381103516 - rollout_corr/rollout_is_seq_max_deviation:0.013857364654541016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1719011110253632) - rollout_corr/training_log_ppl:np.float64(0.15430643195577431) - rollout_corr/kl:np.float64(0.0007580332686956126) - rollout_corr/k3_kl:np.float64(0.0007594637973333818) - rollout_corr/rollout_ppl:np.float64(1.170976147055626) - rollout_corr/rollout_log_ppl:np.float64(0.15354839794599684) - rollout_corr/log_ppl_diff:np.float64(0.0007580340097774751) - rollout_corr/log_ppl_abs_diff:np.float64(0.002954981282528024) - rollout_corr/log_ppl_diff_max:np.float64(0.02083895355463028) - rollout_corr/log_ppl_diff_min:np.float64(-0.009281948208808899) - rollout_corr/ppl_ratio:np.float64(1.0007657622918487) - rollout_corr/chi2_token:np.float64(0.0014637114945799112) - rollout_corr/chi2_seq:np.float64(0.23655171529389918) - actor/pg_loss:np.float64(5.919951945543289e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00032189934790949337) - actor/ppo_kl:np.float64(8.871946654664953e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4453125) - self_distillation/token_reweight_w_mean:np.float64(374764.59442137624) - self_distillation/token_reweight_w_std:np.float64(2854560.623046875) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004833433777094) - self_distillation/token_reweight_w_clip_frac:np.float64(0.007673227315535769) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.16934452578425407) - perf/mfu/actor:np.float64(0.04497720870630688) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.24027252197266) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:0 - critic/score/mean:0.4453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-9.888575004879385e-05 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:-9.888575004879385e-05 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:232.078125 - response_length/max:8192.0 - response_length/min:46.0 - response_length/clip_ratio:0.01171875 - response_length_non_aborted/mean:232.078125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:46.0 - response_length_non_aborted/clip_ratio:0.01171875 - response/aborted_ratio:0.0 - prompt_length/mean:559.4375 - prompt_length/max:937.0 - prompt_length/min:310.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010770559310913086 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6175977569073439) - timing_s/agent_loop/generate_sequences/max:np.float64(48.33113631978631) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.9512149386428064) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(48.33113631978631) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:537 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:54.00079749152064 - timing_s/reward:0.06639662384986877 - timing_s/old_log_prob:2.4838423933833838 - timing_s/adv:0.5642285142093897 - timing_s/update_actor:16.534782646223903 - timing_s/step:73.68361703306437 - timing_s/stop_profile:3.0431896448135376e-05 - timing_per_token_ms/update_actor:0.08160166732250183 - timing_per_token_ms/gen:0.9089207145277157 - timing_per_token_ms/adv:0.0027845535375633657 - perf/total_num_tokens:202628 - perf/time_per_step:73.68361703306437 - perf/throughput:343.74669729682563 (TaskRunner pid=2204243) Training Progress: 86%|████████▌ | 86/100 [52:48<10:42, 45.90s/it] (TaskRunner pid=2204243) step:87 - global_seqlen/min:17981 - global_seqlen/max:33694 - global_seqlen/minmax_diff:15713 - global_seqlen/balanced_min:23663 - global_seqlen/balanced_max:23667 - global_seqlen/mean:23666.125 - actor/entropy:0.20360888540744781 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2925991714000702 - training/rollout_probs_diff_mean:0.004420910961925983 - training/rollout_probs_diff_std:0.012825322337448597 - training/rollout_actor_probs_pearson_corr:0.9982466101646423 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.34375 - self_distillation/correct_sample_fraction:0.33984375 - self_distillation/feedback_available_fraction:0.66015625 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:0.9997774958610535 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.5995540618896484 - rollout_corr/rollout_is_min:0.5990896821022034 - rollout_corr/rollout_is_std:0.03714228793978691 - rollout_corr/rollout_is_eff_sample_size:0.9986218160671462 - rollout_corr/rollout_is_seq_mean:0.999767005443573 - rollout_corr/rollout_is_seq_std:0.003877673763781786 - rollout_corr/rollout_is_seq_max:1.0143232345581055 - rollout_corr/rollout_is_seq_min:0.9884859323501587 - rollout_corr/rollout_is_seq_max_deviation:0.014323234558105469 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2073437934741378) - rollout_corr/training_log_ppl:np.float64(0.18021402907106676) - rollout_corr/kl:np.float64(0.000991385972050729) - rollout_corr/k3_kl:np.float64(0.0007554213387326314) - rollout_corr/rollout_ppl:np.float64(1.2060586903244257) - rollout_corr/rollout_log_ppl:np.float64(0.1792226426259731) - rollout_corr/log_ppl_diff:np.float64(0.0009913864450936671) - rollout_corr/log_ppl_abs_diff:np.float64(0.002984586677484913) - rollout_corr/log_ppl_diff_max:np.float64(0.01778009533882141) - rollout_corr/log_ppl_diff_min:np.float64(-0.011145472526550293) - rollout_corr/ppl_ratio:np.float64(1.0009996725711972) - rollout_corr/chi2_token:np.float64(0.0009986464865505695) - rollout_corr/chi2_seq:np.float64(0.13895411207340658) - actor/pg_loss:np.float64(0.0001801503822207451) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(5.5803571740398183e-05) - actor/ppo_kl:np.float64(4.988508512582257e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.33984375) - self_distillation/token_reweight_w_mean:np.float64(27075.774560384452) - self_distillation/token_reweight_w_std:np.float64(224893.0407767702) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998465650714934) - self_distillation/token_reweight_w_clip_frac:np.float64(0.004404331411933526) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.07866272330284119) - perf/mfu/actor:np.float64(0.04350149547025709) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.26218795776367) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:0 - critic/score/mean:0.33984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009291116148233414 - critic/advantages/max:0.125 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009291116148233414 - critic/returns/max:0.125 - critic/returns/min:-0.875 - response_length/mean:120.34765625 - response_length/max:352.0 - response_length/min:43.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:120.34765625 - response_length_non_aborted/max:352.0 - response_length_non_aborted/min:43.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:619.21875 - prompt_length/max:1504.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.503208816051483e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5906312745064497) - timing_s/agent_loop/generate_sequences/max:np.float64(2.6868079993873835) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.33232098530425) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.6868079993873835) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:822 - timing_s/agent_loop/slowest/response_length:337 - timing_s/gen:8.730725707486272 - timing_s/reward:0.07404297031462193 - timing_s/old_log_prob:2.272407330572605 - timing_s/adv:0.6395836509764194 - timing_s/update_actor:15.722655784338713 - timing_s/step:27.528541481122375 - timing_s/stop_profile:0.00012158788740634918 - timing_per_token_ms/update_actor:0.0830440967011853 - timing_per_token_ms/gen:0.2833823138526493 - timing_per_token_ms/adv:0.00337815998064966 - perf/total_num_tokens:189329 - perf/time_per_step:27.528541481122375 - perf/throughput:859.6941111547441 (TaskRunner pid=2204243) Training Progress: 87%|████████▋ | 87/100 [53:16<08:45, 40.40s/it] (TaskRunner pid=2204243) step:88 - global_seqlen/min:16117 - global_seqlen/max:26575 - global_seqlen/minmax_diff:10458 - global_seqlen/balanced_min:22836 - global_seqlen/balanced_max:22840 - global_seqlen/mean:22838.75 - actor/entropy:0.16092704236507416 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23108381032943726 - training/rollout_probs_diff_mean:0.003980543930083513 - training/rollout_probs_diff_std:0.012280721217393875 - training/rollout_actor_probs_pearson_corr:0.9980723261833191 - self_distillation/success_group_fraction:0.34375 - self_distillation/success_sample_fraction:0.33984375 - self_distillation/correct_sample_fraction:0.31640625 - self_distillation/feedback_available_fraction:0.68359375 - self_distillation/feedback_used_fraction:0.65625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.99952632188797 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00010614960046950728 - rollout_corr/rollout_is_max:1.6479557752609253 - rollout_corr/rollout_is_min:0.24590186774730682 - rollout_corr/rollout_is_std:0.03567294776439667 - rollout_corr/rollout_is_eff_sample_size:0.9987278691194604 - rollout_corr/rollout_is_seq_mean:0.9996195435523987 - rollout_corr/rollout_is_seq_std:0.00424192612990737 - rollout_corr/rollout_is_seq_max:1.0219006538391113 - rollout_corr/rollout_is_seq_min:0.982966959476471 - rollout_corr/rollout_is_seq_max_deviation:0.021900653839111328 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1626617689616978) - rollout_corr/training_log_ppl:np.float64(0.14736086662014714) - rollout_corr/kl:np.float64(0.0014385642446335112) - rollout_corr/k3_kl:np.float64(0.0010576494764382716) - rollout_corr/rollout_ppl:np.float64(1.1609200090169907) - rollout_corr/rollout_log_ppl:np.float64(0.14592230111156823) - rollout_corr/log_ppl_diff:np.float64(0.001438565508578904) - rollout_corr/log_ppl_abs_diff:np.float64(0.0036717926122946665) - rollout_corr/log_ppl_diff_max:np.float64(0.030946820974349976) - rollout_corr/log_ppl_diff_min:np.float64(-0.018898382782936096) - rollout_corr/ppl_ratio:np.float64(1.0014523859135807) - rollout_corr/chi2_token:np.float64(0.0013002885971218348) - rollout_corr/chi2_seq:np.float64(0.05099422694183886) - actor/pg_loss:np.float64(-9.366520680487156e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015767694276291877) - actor/ppo_kl:np.float64(0.0003774436690520133) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.31640625) - self_distillation/token_reweight_w_mean:np.float64(26122.738449087832) - self_distillation/token_reweight_w_std:np.float64(224774.28886795044) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006604329682887) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0056974871549755335) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.15424800477921963) - perf/mfu/actor:np.float64(0.04256022111070913) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.17083358764648) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:0 - critic/score/mean:0.31640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.31640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000667857879307121 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:-0.000667857879307121 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:110.3984375 - response_length/max:505.0 - response_length/min:52.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:110.3984375 - response_length_non_aborted/max:505.0 - response_length_non_aborted/min:52.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:603.3125 - prompt_length/max:898.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.351223707199097e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6736796479672194) - timing_s/agent_loop/generate_sequences/max:np.float64(3.339456807821989) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.284205783093057) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.339456807821989) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:626 - timing_s/agent_loop/slowest/response_length:505 - timing_s/gen:9.218855174258351 - timing_s/reward:0.07278558053076267 - timing_s/old_log_prob:2.2618861589580774 - timing_s/adv:0.6049918532371521 - timing_s/update_actor:15.37425435334444 - timing_s/step:27.61986025609076 - timing_s/stop_profile:0.00011293590068817139 - timing_per_token_ms/update_actor:0.08414566445922193 - timing_per_token_ms/gen:0.32619259692372626 - timing_per_token_ms/adv:0.003311213689656571 - perf/total_num_tokens:182710 - perf/time_per_step:27.61986025609076 - perf/throughput:826.8959288077344 (TaskRunner pid=2204243) Training Progress: 88%|████████▊ | 88/100 [53:43<07:18, 36.58s/it] (TaskRunner pid=2204243) step:89 - global_seqlen/min:19256 - global_seqlen/max:26759 - global_seqlen/minmax_diff:7503 - global_seqlen/balanced_min:22339 - global_seqlen/balanced_max:22346 - global_seqlen/mean:22343.5 - actor/entropy:0.14621137082576752 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3519406318664551 - training/rollout_probs_diff_mean:0.003735581412911415 - training/rollout_probs_diff_std:0.012287755496799946 - training/rollout_actor_probs_pearson_corr:0.9978753924369812 - self_distillation/success_group_fraction:0.46875 - self_distillation/success_sample_fraction:0.46484375 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.56640625 - self_distillation/feedback_used_fraction:0.53125 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9999240636825562 - rollout_corr/rollout_is_ratio_fraction_high:3.530574758769944e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.377272605895996 - rollout_corr/rollout_is_min:0.5124092102050781 - rollout_corr/rollout_is_std:0.034004855901002884 - rollout_corr/rollout_is_eff_sample_size:0.9988447675782389 - rollout_corr/rollout_is_seq_mean:0.9997600317001343 - rollout_corr/rollout_is_seq_std:0.0039210692048072815 - rollout_corr/rollout_is_seq_max:1.010308027267456 - rollout_corr/rollout_is_seq_min:0.9864083528518677 - rollout_corr/rollout_is_seq_max_deviation:0.013591647148132324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1447467715479434) - rollout_corr/training_log_ppl:np.float64(0.13193432396656135) - rollout_corr/kl:np.float64(0.0010786341643296282) - rollout_corr/k3_kl:np.float64(0.0008497881394013973) - rollout_corr/rollout_ppl:np.float64(1.143433082383126) - rollout_corr/rollout_log_ppl:np.float64(0.1308556885633152) - rollout_corr/log_ppl_diff:np.float64(0.0010786354032461531) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032684893158148043) - rollout_corr/log_ppl_diff_max:np.float64(0.01591651141643524) - rollout_corr/log_ppl_diff_min:np.float64(-0.011132173240184784) - rollout_corr/ppl_ratio:np.float64(1.0010887831449509) - rollout_corr/chi2_token:np.float64(0.0011855072807520628) - rollout_corr/chi2_seq:np.float64(0.1610140281263739) - actor/pg_loss:np.float64(0.0003769798204302788) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022688171156914905) - actor/ppo_kl:np.float64(0.00024304834030530742) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(109769.26533999643) - self_distillation/token_reweight_w_std:np.float64(1158514.170177088) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000638244440779) - self_distillation/token_reweight_w_clip_frac:np.float64(0.010712702525779605) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.16431638691574335) - perf/mfu/actor:np.float64(0.04192488994817313) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.3653564453125) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:0 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0005075201042927802 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0005075201042927802 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:110.640625 - response_length/max:403.0 - response_length/min:47.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:110.640625 - response_length_non_aborted/max:403.0 - response_length_non_aborted/min:47.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:587.59375 - prompt_length/max:1263.0 - prompt_length/min:356.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00021118484437465668 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6210750788450241) - timing_s/agent_loop/generate_sequences/max:np.float64(2.8871809858828783) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2451201435178518) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.8871809858828783) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:975 - timing_s/agent_loop/slowest/response_length:403 - timing_s/gen:8.729107432067394 - timing_s/reward:0.07258789800107479 - timing_s/old_log_prob:2.276223549619317 - timing_s/adv:0.6422680336982012 - timing_s/update_actor:15.398617455735803 - timing_s/step:27.20969813875854 - timing_s/stop_profile:0.000126618891954422 - timing_per_token_ms/update_actor:0.08614707552384251 - timing_per_token_ms/gen:0.30818766530389047 - timing_per_token_ms/adv:0.0035931480838845816 - perf/total_num_tokens:178748 - perf/time_per_step:27.20969813875854 - perf/throughput:821.1594221316649 (TaskRunner pid=2204243) Training Progress: 89%|████████▉ | 89/100 [54:11<06:11, 33.79s/it] (TaskRunner pid=2204243) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2204243) validation generation end (TaskRunner pid=2204243) [prompt] user (TaskRunner pid=2204243) Your task is to answer the user's question using available tools. (TaskRunner pid=2204243) You have access to the following tools: (TaskRunner pid=2204243) Name: Axolotl (TaskRunner pid=2204243) Description: Collection of axolotl pictures and facts (TaskRunner pid=2204243) Documentation: (TaskRunner pid=2204243) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2204243) Parameters: {} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{url, source, description} (TaskRunner pid=2204243) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2204243) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2204243) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2204243) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Array[Object{fact, source}] (TaskRunner pid=2204243) (TaskRunner pid=2204243) Use the following format: (TaskRunner pid=2204243) Thought: you should always think about what to do (TaskRunner pid=2204243) Action: the action to take, should be one of the tool names. (TaskRunner pid=2204243) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2204243) (TaskRunner pid=2204243) Begin! (TaskRunner pid=2204243) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2204243) assistant (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: I need to find a random axolotl image. The available tool for this is getRandomAxolotlImage. Since the user is asking for a random picture, I don't need any specific parameters. I can directly call the getRandomAxolotlImage tool. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2204655) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2204243) step:90 - global_seqlen/min:18501 - global_seqlen/max:28331 - global_seqlen/minmax_diff:9830 - global_seqlen/balanced_min:23354 - global_seqlen/balanced_max:23357 - global_seqlen/mean:23355.625 - actor/entropy:0.13051380217075348 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17917847633361816 - training/rollout_probs_diff_mean:0.0035186894237995148 - training/rollout_probs_diff_std:0.011770475655794144 - training/rollout_actor_probs_pearson_corr:0.9978505969047546 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.43359375 - self_distillation/correct_sample_fraction:0.38671875 - self_distillation/feedback_available_fraction:0.61328125 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9999172687530518 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6893341541290283 - rollout_corr/rollout_is_min:0.5992377996444702 - rollout_corr/rollout_is_std:0.03218584507703781 - rollout_corr/rollout_is_eff_sample_size:0.9989649053862552 - rollout_corr/rollout_is_seq_mean:0.9999633431434631 - rollout_corr/rollout_is_seq_std:0.003544068429619074 - rollout_corr/rollout_is_seq_max:1.0195832252502441 - rollout_corr/rollout_is_seq_min:0.9887708425521851 - rollout_corr/rollout_is_seq_max_deviation:0.01958322525024414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1339023541659117) - rollout_corr/training_log_ppl:np.float64(0.12281538376555545) - rollout_corr/kl:np.float64(0.0010227053296603117) - rollout_corr/k3_kl:np.float64(0.0006095054993018323) - rollout_corr/rollout_ppl:np.float64(1.1327121485956013) - rollout_corr/rollout_log_ppl:np.float64(0.12179267901956337) - rollout_corr/log_ppl_diff:np.float64(0.0010227047459920868) - rollout_corr/log_ppl_abs_diff:np.float64(0.003151139200781472) - rollout_corr/log_ppl_diff_max:np.float64(0.014972761273384094) - rollout_corr/log_ppl_diff_min:np.float64(-0.01233142614364624) - rollout_corr/ppl_ratio:np.float64(1.001030828570947) - rollout_corr/chi2_token:np.float64(0.0003959059249609709) - rollout_corr/chi2_seq:np.float64(0.025008260970935225) - actor/pg_loss:np.float64(0.0012602752540260553) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001473158463340951) - actor/ppo_kl:np.float64(0.0004688246643951288) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.38671875) - self_distillation/token_reweight_w_mean:np.float64(253621.5214341036) - self_distillation/token_reweight_w_std:np.float64(2056351.2120031216) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998478267807513) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020603987904905807) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.3405766598880291) - perf/mfu/actor:np.float64(0.04384506998777208) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.3672571182251) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.53125) - val-aux/tooluse/reward/std@16:np.float64(0.05290202981906864) - val-aux/tooluse/reward/best@2/mean:np.float64(0.5533970588235294) - val-aux/tooluse/reward/best@2/std:np.float64(0.05410191956786321) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5092941176470588) - val-aux/tooluse/reward/worst@2/std:np.float64(0.03658367016914121) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5321470588235294) - val-aux/tooluse/reward/maj@2/std:np.float64(0.05347644544856224) - val-aux/tooluse/reward/best@4/mean:np.float64(0.5772352941176471) - val-aux/tooluse/reward/best@4/std:np.float64(0.04646617367882632) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.4966323529411766) - val-aux/tooluse/reward/worst@4/std:np.float64(0.019288062044388266) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.5277941176470589) - val-aux/tooluse/reward/maj@4/std:np.float64(0.04382840063916081) - val-aux/tooluse/reward/best@8/mean:np.float64(0.5969264705882353) - val-aux/tooluse/reward/best@8/std:np.float64(0.033624195864156804) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.4905147058823529) - val-aux/tooluse/reward/worst@8/std:np.float64(0.009293220406232768) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.5248235294117647) - val-aux/tooluse/reward/maj@8/std:np.float64(0.03316329238859305) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6098823529411765) - val-aux/tooluse/reward/best@16/std:np.float64(0.017716953531754984) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.48707352941176474) - val-aux/tooluse/reward/worst@16/std:np.float64(0.004795989349532633) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.5242941176470589) - val-aux/tooluse/reward/maj@16/std:np.float64(0.02231425995182375) - val-aux/tooluse/score/mean@16:np.float64(0.53125) - val-aux/tooluse/score/std@16:np.float64(0.05290202981906864) - val-aux/tooluse/score/best@2/mean:np.float64(0.5533970588235294) - val-aux/tooluse/score/best@2/std:np.float64(0.05410191956786321) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5092941176470588) - val-aux/tooluse/score/worst@2/std:np.float64(0.03658367016914121) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5321470588235294) - val-aux/tooluse/score/maj@2/std:np.float64(0.05347644544856224) - val-aux/tooluse/score/best@4/mean:np.float64(0.5772352941176471) - val-aux/tooluse/score/best@4/std:np.float64(0.04646617367882632) - val-aux/tooluse/score/worst@4/mean:np.float64(0.4966323529411766) - val-aux/tooluse/score/worst@4/std:np.float64(0.019288062044388266) - val-aux/tooluse/score/maj@4/mean:np.float64(0.5277941176470589) - val-aux/tooluse/score/maj@4/std:np.float64(0.04382840063916081) - val-aux/tooluse/score/best@8/mean:np.float64(0.5969264705882353) - val-aux/tooluse/score/best@8/std:np.float64(0.033624195864156804) - val-aux/tooluse/score/worst@8/mean:np.float64(0.4905147058823529) - val-aux/tooluse/score/worst@8/std:np.float64(0.009293220406232768) - val-aux/tooluse/score/maj@8/mean:np.float64(0.5248235294117647) - val-aux/tooluse/score/maj@8/std:np.float64(0.03316329238859305) - val-aux/tooluse/score/best@16/mean:np.float64(0.6098823529411765) - val-aux/tooluse/score/best@16/std:np.float64(0.017716953531754984) - val-aux/tooluse/score/worst@16/mean:np.float64(0.48707352941176474) - val-aux/tooluse/score/worst@16/std:np.float64(0.004795989349532633) - val-aux/tooluse/score/maj@16/mean:np.float64(0.5242941176470589) - val-aux/tooluse/score/maj@16/std:np.float64(0.02231425995182375) - val-core/tooluse/acc/mean@16:np.float64(0.53125) - val-aux/tooluse/acc/std@16:np.float64(0.05290202981906864) - val-aux/tooluse/acc/best@2/mean:np.float64(0.5533970588235294) - val-aux/tooluse/acc/best@2/std:np.float64(0.05410191956786321) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5092941176470588) - val-aux/tooluse/acc/worst@2/std:np.float64(0.03658367016914121) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5321470588235294) - val-aux/tooluse/acc/maj@2/std:np.float64(0.05347644544856224) - val-aux/tooluse/acc/best@4/mean:np.float64(0.5772352941176471) - val-aux/tooluse/acc/best@4/std:np.float64(0.04646617367882632) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.4966323529411766) - val-aux/tooluse/acc/worst@4/std:np.float64(0.019288062044388266) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.5277941176470589) - val-aux/tooluse/acc/maj@4/std:np.float64(0.04382840063916081) - val-aux/tooluse/acc/best@8/mean:np.float64(0.5969264705882353) - val-aux/tooluse/acc/best@8/std:np.float64(0.033624195864156804) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.4905147058823529) - val-aux/tooluse/acc/worst@8/std:np.float64(0.009293220406232768) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.5248235294117647) - val-aux/tooluse/acc/maj@8/std:np.float64(0.03316329238859305) - val-core/tooluse/acc/best@16/mean:np.float64(0.6098823529411765) - val-core/tooluse/acc/best@16/std:np.float64(0.017716953531754984) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.48707352941176474) - val-aux/tooluse/acc/worst@16/std:np.float64(0.004795989349532633) - val-core/tooluse/acc/maj@16/mean:np.float64(0.5242941176470589) - val-core/tooluse/acc/maj@16/std:np.float64(0.02231425995182375) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:0 - critic/score/mean:0.38671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.38671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012657631887122989 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012657631887122989 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:103.76953125 - response_length/max:254.0 - response_length/min:47.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:103.76953125 - response_length_non_aborted/max:254.0 - response_length_non_aborted/min:47.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:626.09375 - prompt_length/max:1100.0 - prompt_length/min:300.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013740919530391693 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5605221595615149) - timing_s/agent_loop/generate_sequences/max:np.float64(2.158891500905156) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.197140905962442) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.158891500905156) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:694 - timing_s/agent_loop/slowest/response_length:254 - timing_s/gen:8.09290637075901 - timing_s/reward:0.07736576348543167 - timing_s/old_log_prob:2.39638589322567 - timing_s/adv:0.6262637078762054 - timing_s/update_actor:15.072210533544421 - timing_s/step:26.3537392411381 - timing_s/testing:20.333198606967926 - timing_s/save_checkpoint:6.670652311295271 - timing_s/stop_profile:0.0001239720731973648 - timing_per_token_ms/update_actor:0.08066691928360097 - timing_per_token_ms/gen:0.3046454496803693 - timing_per_token_ms/adv:0.003351782000461374 - perf/total_num_tokens:186845 - perf/time_per_step:26.3537392411381 - perf/throughput:886.235717303522 (TaskRunner pid=2204243) Training Progress: 90%|█████████ | 90/100 [55:04<06:36, 39.67s/it] (TaskRunner pid=2204243) step:91 - global_seqlen/min:18063 - global_seqlen/max:29639 - global_seqlen/minmax_diff:11576 - global_seqlen/balanced_min:23351 - global_seqlen/balanced_max:23354 - global_seqlen/mean:23352.375 - actor/entropy:0.1644558608531952 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135532975196838 - training/rollout_probs_diff_mean:0.00420619361102581 - training/rollout_probs_diff_std:0.01313781924545765 - training/rollout_actor_probs_pearson_corr:0.9977981448173523 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.4375 - self_distillation/correct_sample_fraction:0.40625 - self_distillation/feedback_available_fraction:0.59375 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.000225305557251 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.7523045539855957 - rollout_corr/rollout_is_min:0.5224010944366455 - rollout_corr/rollout_is_std:0.03756117448210716 - rollout_corr/rollout_is_eff_sample_size:0.9985918589711463 - rollout_corr/rollout_is_seq_mean:1.0001392364501953 - rollout_corr/rollout_is_seq_std:0.0036101790610700846 - rollout_corr/rollout_is_seq_max:1.0111454725265503 - rollout_corr/rollout_is_seq_min:0.9880301356315613 - rollout_corr/rollout_is_seq_max_deviation:0.01196986436843872 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1577908601611853) - rollout_corr/training_log_ppl:np.float64(0.14243710546725197) - rollout_corr/kl:np.float64(0.0012016478976946132) - rollout_corr/k3_kl:np.float64(0.0008741293130943295) - rollout_corr/rollout_ppl:np.float64(1.156438204459846) - rollout_corr/rollout_log_ppl:np.float64(0.14123545563052176) - rollout_corr/log_ppl_diff:np.float64(0.001201649836730212) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034914914576802403) - rollout_corr/log_ppl_diff_max:np.float64(0.04410596191883087) - rollout_corr/log_ppl_diff_min:np.float64(-0.010282188653945923) - rollout_corr/ppl_ratio:np.float64(1.0012147452216595) - rollout_corr/chi2_token:np.float64(0.0009491420350968838) - rollout_corr/chi2_seq:np.float64(0.20896582235582173) - actor/pg_loss:np.float64(0.001087681739591062) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006307077401288552) - actor/ppo_kl:np.float64(0.0006723608765497602) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40625) - self_distillation/token_reweight_w_mean:np.float64(325708.91742183524) - self_distillation/token_reweight_w_std:np.float64(2131817.271234979) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999031920451671) - self_distillation/token_reweight_w_clip_frac:np.float64(0.014555271249264479) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.2527767941355705) - perf/mfu/actor:np.float64(0.04351605173052003) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.65637588500977) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:0 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00399628235027194 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.00399628235027194 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:99.82421875 - response_length/max:282.0 - response_length/min:48.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:99.82421875 - response_length_non_aborted/max:282.0 - response_length_non_aborted/min:48.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:629.9375 - prompt_length/max:1250.0 - prompt_length/min:313.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.43988561630249e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6656279005110264) - timing_s/agent_loop/generate_sequences/max:np.float64(2.1905483808368444) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1939243460728903) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.1905483808368444) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:476 - timing_s/agent_loop/slowest/response_length:282 - timing_s/gen:7.9196035992354155 - timing_s/reward:0.06918943300843239 - timing_s/old_log_prob:2.331745021045208 - timing_s/adv:0.6299956068396568 - timing_s/update_actor:15.46997002325952 - timing_s/step:26.522130550816655 - timing_s/stop_profile:0.00010880455374717712 - timing_per_token_ms/update_actor:0.0828072627690948 - timing_per_token_ms/gen:0.3099042691933248 - timing_per_token_ms/adv:0.003372224489156118 - perf/total_num_tokens:186819 - perf/time_per_step:26.522130550816655 - perf/throughput:880.4863906109136 (TaskRunner pid=2204243) Training Progress: 91%|█████████ | 91/100 [55:30<05:21, 35.74s/it] (TaskRunner pid=2204243) step:92 - global_seqlen/min:20339 - global_seqlen/max:23634 - global_seqlen/minmax_diff:3295 - global_seqlen/balanced_min:21986 - global_seqlen/balanced_max:21997 - global_seqlen/mean:21995.125 - actor/entropy:0.17001771926879883 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24490222334861755 - training/rollout_probs_diff_mean:0.004076202865689993 - training/rollout_probs_diff_std:0.012681076303124428 - training/rollout_actor_probs_pearson_corr:0.9980455636978149 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.43359375 - self_distillation/correct_sample_fraction:0.3828125 - self_distillation/feedback_available_fraction:0.6171875 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9998642206192017 - rollout_corr/rollout_is_ratio_fraction_high:3.8746173231629655e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.749234646325931e-05 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.2996797263622284 - rollout_corr/rollout_is_std:0.03740852698683739 - rollout_corr/rollout_is_eff_sample_size:0.9986022010987637 - rollout_corr/rollout_is_seq_mean:0.999732494354248 - rollout_corr/rollout_is_seq_std:0.004131541587412357 - rollout_corr/rollout_is_seq_max:1.014111042022705 - rollout_corr/rollout_is_seq_min:0.9845104813575745 - rollout_corr/rollout_is_seq_max_deviation:0.015489518642425537 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1747825471684337) - rollout_corr/training_log_ppl:np.float64(0.15525201605487382) - rollout_corr/kl:np.float64(0.0014213697064346942) - rollout_corr/k3_kl:np.float64(0.0008873683766346119) - rollout_corr/rollout_ppl:np.float64(1.1731058820150793) - rollout_corr/rollout_log_ppl:np.float64(0.15383064672641922) - rollout_corr/log_ppl_diff:np.float64(0.0014213693284546025) - rollout_corr/log_ppl_abs_diff:np.float64(0.0037850769367651083) - rollout_corr/log_ppl_diff_max:np.float64(0.030654504895210266) - rollout_corr/log_ppl_diff_min:np.float64(-0.012456655502319336) - rollout_corr/ppl_ratio:np.float64(1.0014354344457388) - rollout_corr/chi2_token:np.float64(0.0007288116030395031) - rollout_corr/chi2_seq:np.float64(0.14073164644651115) - actor/pg_loss:np.float64(0.0011522832792252302) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00042633156044757925) - actor/ppo_kl:np.float64(0.00047862250814034724) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3828125) - self_distillation/token_reweight_w_mean:np.float64(108850.37335947971) - self_distillation/token_reweight_w_std:np.float64(1028225.960011102) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007107008714229) - self_distillation/token_reweight_w_clip_frac:np.float64(0.023114861105568707) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.4250175515189767) - perf/mfu/actor:np.float64(0.0410627512181573) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.38052463531494) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:0 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017242047470062971 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017242047470062971 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:100.81640625 - response_length/max:452.0 - response_length/min:42.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:100.81640625 - response_length_non_aborted/max:452.0 - response_length_non_aborted/min:42.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:586.53125 - prompt_length/max:912.0 - prompt_length/min:292.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010212138295173645 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6064733229577541) - timing_s/agent_loop/generate_sequences/max:np.float64(3.0024579726159573) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2015974550231476) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.0024579726159573) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:869 - timing_s/agent_loop/slowest/response_length:452 - timing_s/gen:8.922092705965042 - timing_s/reward:0.07139282859861851 - timing_s/old_log_prob:2.2432087529450655 - timing_s/adv:0.5813731998205185 - timing_s/update_actor:15.456907227635384 - timing_s/step:27.364940766245127 - timing_s/stop_profile:0.00014621764421463013 - timing_per_token_ms/update_actor:0.08784280168693849 - timing_per_token_ms/gen:0.3456969547818607 - timing_per_token_ms/adv:0.0033039889510773322 - perf/total_num_tokens:175961 - perf/time_per_step:27.364940766245127 - perf/throughput:803.7702397343086 (TaskRunner pid=2204243) Training Progress: 92%|█████████▏| 92/100 [55:58<04:25, 33.24s/it] (TaskRunner pid=2204243) step:93 - global_seqlen/min:17211 - global_seqlen/max:26339 - global_seqlen/minmax_diff:9128 - global_seqlen/balanced_min:21169 - global_seqlen/balanced_max:21171 - global_seqlen/mean:21169.625 - actor/entropy:0.1592872589826584 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.17936190962791443 - training/rollout_probs_diff_mean:0.004115244839340448 - training/rollout_probs_diff_std:0.012719552963972092 - training/rollout_actor_probs_pearson_corr:0.9978631138801575 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.37109375 - self_distillation/correct_sample_fraction:0.3125 - self_distillation/feedback_available_fraction:0.6875 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9998572468757629 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.5892466257791966e-05 - rollout_corr/rollout_is_max:1.609740972518921 - rollout_corr/rollout_is_min:0.3864908516407013 - rollout_corr/rollout_is_std:0.036453064531087875 - rollout_corr/rollout_is_eff_sample_size:0.9986724619123001 - rollout_corr/rollout_is_seq_mean:0.9999569654464722 - rollout_corr/rollout_is_seq_std:0.0035286122001707554 - rollout_corr/rollout_is_seq_max:1.0103294849395752 - rollout_corr/rollout_is_seq_min:0.9889457821846008 - rollout_corr/rollout_is_seq_max_deviation:0.01105421781539917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.162720002233982) - rollout_corr/training_log_ppl:np.float64(0.1469354477885645) - rollout_corr/kl:np.float64(0.0018152437487373163) - rollout_corr/k3_kl:np.float64(0.001766292733201169) - rollout_corr/rollout_ppl:np.float64(1.1605583704076707) - rollout_corr/rollout_log_ppl:np.float64(0.14512020467373077) - rollout_corr/log_ppl_diff:np.float64(0.00181524311483372) - rollout_corr/log_ppl_abs_diff:np.float64(0.004289531105314381) - rollout_corr/log_ppl_diff_max:np.float64(0.0966716855764389) - rollout_corr/log_ppl_diff_min:np.float64(-0.012332402169704437) - rollout_corr/ppl_ratio:np.float64(1.0018754643388093) - rollout_corr/chi2_token:np.float64(0.0018281545490026474) - rollout_corr/chi2_seq:np.float64(0.16897357907146215) - actor/pg_loss:np.float64(0.001049531390890479) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008920867130655097) - actor/ppo_kl:np.float64(0.0010840785810053433) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3125) - self_distillation/token_reweight_w_mean:np.float64(242902.90372828813) - self_distillation/token_reweight_w_std:np.float64(1826677.196471394) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000198530498892) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01693812574376352) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.30447177216410637) - perf/mfu/actor:np.float64(0.039272406861337816) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.57890701293945) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:0 - critic/score/mean:0.3125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00016600266098976135 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:-0.00016600266098976135 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:108.83203125 - response_length/max:319.0 - response_length/min:40.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:108.83203125 - response_length_non_aborted/max:319.0 - response_length_non_aborted/min:40.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:552.71875 - prompt_length/max:862.0 - prompt_length/min:309.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.387491106987e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5658710487186909) - timing_s/agent_loop/generate_sequences/max:np.float64(2.352899741381407) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.219376548891887) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.352899741381407) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:649 - timing_s/agent_loop/slowest/response_length:319 - timing_s/gen:8.463766392320395 - timing_s/reward:0.07135904766619205 - timing_s/old_log_prob:2.2968989238142967 - timing_s/adv:0.5780770163983107 - timing_s/update_actor:15.484928153455257 - timing_s/step:26.981482112780213 - timing_s/stop_profile:0.00011260062456130981 - timing_per_token_ms/update_actor:0.09143364699100277 - timing_per_token_ms/gen:0.3037854489185742 - timing_per_token_ms/adv:0.0034133635834261984 - perf/total_num_tokens:169357 - perf/time_per_step:26.981482112780213 - perf/throughput:784.5983001049697 (TaskRunner pid=2204243) Training Progress: 93%|█████████▎| 93/100 [56:25<03:39, 31.38s/it] (TaskRunner pid=2204243) step:94 - global_seqlen/min:19609 - global_seqlen/max:27502 - global_seqlen/minmax_diff:7893 - global_seqlen/balanced_min:23298 - global_seqlen/balanced_max:23303 - global_seqlen/mean:23299.75 - actor/entropy:0.19475434720516205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3619335889816284 - training/rollout_probs_diff_mean:0.004458364564925432 - training/rollout_probs_diff_std:0.012608465738594532 - training/rollout_actor_probs_pearson_corr:0.9982342720031738 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.43359375 - self_distillation/correct_sample_fraction:0.3671875 - self_distillation/feedback_available_fraction:0.6328125 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9997863173484802 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.645844743819907e-05 - rollout_corr/rollout_is_max:1.6351319551467896 - rollout_corr/rollout_is_min:0.17326880991458893 - rollout_corr/rollout_is_std:0.03640807047486305 - rollout_corr/rollout_is_eff_sample_size:0.998675434245894 - rollout_corr/rollout_is_seq_mean:0.9996681809425354 - rollout_corr/rollout_is_seq_std:0.003934528212994337 - rollout_corr/rollout_is_seq_max:1.0094499588012695 - rollout_corr/rollout_is_seq_min:0.9825236201286316 - rollout_corr/rollout_is_seq_max_deviation:0.017476379871368408 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1977363326586783) - rollout_corr/training_log_ppl:np.float64(0.1754945992397552) - rollout_corr/kl:np.float64(0.0011967971605884031) - rollout_corr/k3_kl:np.float64(0.0009977852648859198) - rollout_corr/rollout_ppl:np.float64(1.196248003281653) - rollout_corr/rollout_log_ppl:np.float64(0.17429780179736554) - rollout_corr/log_ppl_diff:np.float64(0.0011967974423896521) - rollout_corr/log_ppl_abs_diff:np.float64(0.0034962853824254125) - rollout_corr/log_ppl_diff_max:np.float64(0.06141384690999985) - rollout_corr/log_ppl_diff_min:np.float64(-0.020983338356018066) - rollout_corr/ppl_ratio:np.float64(1.0012146916706115) - rollout_corr/chi2_token:np.float64(0.0013997186906635761) - rollout_corr/chi2_seq:np.float64(0.0676977182738483) - actor/pg_loss:np.float64(0.0007157016079872847) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000650725518426043) - actor/ppo_kl:np.float64(0.00019640693034728152) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3671875) - self_distillation/token_reweight_w_mean:np.float64(207901.21812492097) - self_distillation/token_reweight_w_std:np.float64(2032185.534226591) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005562133155763) - self_distillation/token_reweight_w_clip_frac:np.float64(0.020680241752415895) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.38690751045942307) - perf/mfu/actor:np.float64(0.04314620838851893) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.4060935974121) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:0 - critic/score/mean:0.3671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0035170884802937508 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0035170884802937508 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:102.1796875 - response_length/max:310.0 - response_length/min:36.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:102.1796875 - response_length_non_aborted/max:310.0 - response_length_non_aborted/min:36.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:625.9375 - prompt_length/max:975.0 - prompt_length/min:247.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011400505900382996 - timing_s/agent_loop/generate_sequences/min:np.float64(0.4132590312510729) - timing_s/agent_loop/generate_sequences/max:np.float64(2.341607976704836) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1057334259530762) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.341607976704836) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:793 - timing_s/agent_loop/slowest/response_length:310 - timing_s/gen:8.27325558848679 - timing_s/reward:0.0741738174110651 - timing_s/old_log_prob:2.2711919993162155 - timing_s/adv:0.5859129186719656 - timing_s/update_actor:15.534094601869583 - timing_s/step:26.825815303251147 - timing_s/stop_profile:0.00011529959738254547 - timing_per_token_ms/update_actor:0.08333831157989668 - timing_per_token_ms/gen:0.3162801280100463 - timing_per_token_ms/adv:0.003143343376387974 - perf/total_num_tokens:186398 - perf/time_per_step:26.825815303251147 - perf/throughput:868.5570125869089 (TaskRunner pid=2204243) Training Progress: 94%|█████████▍| 94/100 [56:52<03:00, 30.03s/it] (TaskRunner pid=2204243) step:95 - global_seqlen/min:17990 - global_seqlen/max:24964 - global_seqlen/minmax_diff:6974 - global_seqlen/balanced_min:21549 - global_seqlen/balanced_max:21551 - global_seqlen/mean:21550.0 - actor/entropy:0.19248172640800476 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.1728518009185791 - training/rollout_probs_diff_mean:0.004438828211277723 - training/rollout_probs_diff_std:0.01277910266071558 - training/rollout_actor_probs_pearson_corr:0.9982007145881653 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.4375 - self_distillation/correct_sample_fraction:0.4140625 - self_distillation/feedback_available_fraction:0.5859375 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000168085098267 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9580543041229248 - rollout_corr/rollout_is_min:0.5543252229690552 - rollout_corr/rollout_is_std:0.037397369742393494 - rollout_corr/rollout_is_eff_sample_size:0.9986032709862611 - rollout_corr/rollout_is_seq_mean:1.0000882148742676 - rollout_corr/rollout_is_seq_std:0.003683962859213352 - rollout_corr/rollout_is_seq_max:1.0093775987625122 - rollout_corr/rollout_is_seq_min:0.9911869764328003 - rollout_corr/rollout_is_seq_max_deviation:0.009377598762512207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1941617969423532) - rollout_corr/training_log_ppl:np.float64(0.17134780643391423) - rollout_corr/kl:np.float64(0.0009291269474241415) - rollout_corr/k3_kl:np.float64(0.00113581047133815) - rollout_corr/rollout_ppl:np.float64(1.1929360306821764) - rollout_corr/rollout_log_ppl:np.float64(0.17041867987427395) - rollout_corr/log_ppl_diff:np.float64(0.0009291265596402809) - rollout_corr/log_ppl_abs_diff:np.float64(0.0035810804256470874) - rollout_corr/log_ppl_diff_max:np.float64(0.022804126143455505) - rollout_corr/log_ppl_diff_min:np.float64(-0.010400041937828064) - rollout_corr/ppl_ratio:np.float64(1.000940319383517) - rollout_corr/chi2_token:np.float64(0.002698986092582345) - rollout_corr/chi2_seq:np.float64(0.15336574474349618) - actor/pg_loss:np.float64(0.0009283754043281078) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005071759806014597) - actor/ppo_kl:np.float64(0.0003353679610471616) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4140625) - self_distillation/token_reweight_w_mean:np.float64(257400.8506274596) - self_distillation/token_reweight_w_std:np.float64(1628479.1144143306) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000633473973721) - self_distillation/token_reweight_w_clip_frac:np.float64(0.01134518306935206) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.33654243499040604) - perf/mfu/actor:np.float64(0.040259980230792164) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.4330291748047) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:0 - critic/score/mean:0.4140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000727857812307775 - critic/advantages/max:0.375 - critic/advantages/min:-0.625 - critic/returns/mean:0.000727857812307775 - critic/returns/max:0.375 - critic/returns/min:-0.625 - response_length/mean:101.96875 - response_length/max:486.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:101.96875 - response_length_non_aborted/max:486.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:571.46875 - prompt_length/max:884.0 - prompt_length/min:310.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.856931865215302e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.3991419542580843) - timing_s/agent_loop/generate_sequences/max:np.float64(3.218200411647558) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1195261132743326) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(3.218200411647558) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:382 - timing_s/agent_loop/slowest/response_length:486 - timing_s/gen:9.077309928834438 - timing_s/reward:0.0719029325991869 - timing_s/old_log_prob:2.240558125078678 - timing_s/adv:0.5955379344522953 - timing_s/update_actor:15.445282116532326 - timing_s/step:27.517477422952652 - timing_s/stop_profile:0.00011677481234073639 - timing_per_token_ms/update_actor:0.0895898034601643 - timing_per_token_ms/gen:0.3477363595171023 - timing_per_token_ms/adv:0.0034543963715330354 - perf/total_num_tokens:172400 - perf/time_per_step:27.517477422952652 - perf/throughput:783.138645623995 (TaskRunner pid=2204243) Training Progress: 95%|█████████▌| 95/100 [57:19<02:26, 29.29s/it] (TaskRunner pid=2204243) step:96 - global_seqlen/min:17060 - global_seqlen/max:33569 - global_seqlen/minmax_diff:16509 - global_seqlen/balanced_min:23748 - global_seqlen/balanced_max:23755 - global_seqlen/mean:23750.75 - actor/entropy:0.14711116254329681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105764389038086 - training/rollout_probs_diff_mean:0.0036259142216295004 - training/rollout_probs_diff_std:0.012286318466067314 - training/rollout_actor_probs_pearson_corr:0.9979830384254456 - self_distillation/success_group_fraction:0.4375 - self_distillation/success_sample_fraction:0.43359375 - self_distillation/correct_sample_fraction:0.35546875 - self_distillation/feedback_available_fraction:0.64453125 - self_distillation/feedback_used_fraction:0.5625 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.0000460147857666 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.8720544576644897 - rollout_corr/rollout_is_min:0.5459107160568237 - rollout_corr/rollout_is_std:0.0341622456908226 - rollout_corr/rollout_is_eff_sample_size:0.9988341825626527 - rollout_corr/rollout_is_seq_mean:1.0001858472824097 - rollout_corr/rollout_is_seq_std:0.0038772309198975563 - rollout_corr/rollout_is_seq_max:1.0130894184112549 - rollout_corr/rollout_is_seq_min:0.9858325719833374 - rollout_corr/rollout_is_seq_max_deviation:0.014167428016662598 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1958652851171792) - rollout_corr/training_log_ppl:np.float64(0.17219999796361662) - rollout_corr/kl:np.float64(0.0007366621619162572) - rollout_corr/k3_kl:np.float64(0.0011125711810121564) - rollout_corr/rollout_ppl:np.float64(1.1949280900880694) - rollout_corr/rollout_log_ppl:np.float64(0.1714633349547512) - rollout_corr/log_ppl_diff:np.float64(0.0007366630088654347) - rollout_corr/log_ppl_abs_diff:np.float64(0.003971026846556924) - rollout_corr/log_ppl_diff_max:np.float64(0.025582537055015564) - rollout_corr/log_ppl_diff_min:np.float64(-0.021554626524448395) - rollout_corr/ppl_ratio:np.float64(1.000751665327698) - rollout_corr/chi2_token:np.float64(0.003100023837760091) - rollout_corr/chi2_seq:np.float64(0.2239897008985281) - actor/pg_loss:np.float64(0.00182237196713686) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000535805091203656) - actor/ppo_kl:np.float64(0.0002197729370081447) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.35546875) - self_distillation/token_reweight_w_mean:np.float64(202959.42615928524) - self_distillation/token_reweight_w_std:np.float64(1820101.857986011) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010167702566832) - self_distillation/token_reweight_w_clip_frac:np.float64(0.02425311499973759) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.4228031411767006) - perf/mfu/actor:np.float64(0.04438805772068445) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.43952178955078) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003981021698564291 - critic/advantages/max:0.875 - critic/advantages/min:-0.5 - critic/returns/mean:-0.003981021698564291 - critic/returns/max:0.875 - critic/returns/min:-0.5 - response_length/mean:119.5859375 - response_length/max:997.0 - response_length/min:36.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:119.5859375 - response_length_non_aborted/max:997.0 - response_length_non_aborted/min:36.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:622.625 - prompt_length/max:1266.0 - prompt_length/min:264.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013344921171665192 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5614762883633375) - timing_s/agent_loop/generate_sequences/max:np.float64(5.672690438106656) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.2571222885308089) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.672690438106656) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:604 - timing_s/agent_loop/slowest/response_length:997 - timing_s/gen:11.539333550259471 - timing_s/reward:0.07460989616811275 - timing_s/old_log_prob:2.3063456024974585 - timing_s/adv:0.6280427556484938 - timing_s/update_actor:15.507926749065518 - timing_s/step:30.143403159454465 - timing_s/stop_profile:0.00011569075286388397 - timing_per_token_ms/update_actor:0.08161808968698629 - timing_per_token_ms/gen:0.37692995199122853 - timing_per_token_ms/adv:0.003305383807082375 - perf/total_num_tokens:190006 - perf/time_per_step:30.143403159454465 - perf/throughput:787.9253007486179 (TaskRunner pid=2204243) Training Progress: 96%|█████████▌| 96/100 [57:50<01:58, 29.56s/it] (TaskRunner pid=2204243) step:97 - global_seqlen/min:13259 - global_seqlen/max:29201 - global_seqlen/minmax_diff:15942 - global_seqlen/balanced_min:21756 - global_seqlen/balanced_max:21760 - global_seqlen/mean:21758.625 - actor/entropy:0.18894581496715546 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5010836720466614 - training/rollout_probs_diff_mean:0.004465639125555754 - training/rollout_probs_diff_std:0.013495288789272308 - training/rollout_actor_probs_pearson_corr:0.9979329109191895 - self_distillation/success_group_fraction:0.375 - self_distillation/success_sample_fraction:0.375 - self_distillation/correct_sample_fraction:0.35546875 - self_distillation/feedback_available_fraction:0.64453125 - self_distillation/feedback_used_fraction:0.625 - self_distillation/reprompt_sample_fraction:1.0 - rollout_corr/rollout_is_mean:1.0000944137573242 - rollout_corr/rollout_is_ratio_fraction_high:7.707426266279072e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.853713133139536e-05 - rollout_corr/rollout_is_max:2.894789457321167 - rollout_corr/rollout_is_min:0.4797880947589874 - rollout_corr/rollout_is_std:0.03960800915956497 - rollout_corr/rollout_is_eff_sample_size:0.998433900682309 - rollout_corr/rollout_is_seq_mean:0.9999880790710449 - rollout_corr/rollout_is_seq_std:0.003919485490769148 - rollout_corr/rollout_is_seq_max:1.015187382698059 - rollout_corr/rollout_is_seq_min:0.9894315004348755 - rollout_corr/rollout_is_seq_max_deviation:0.015187382698059082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1813332173042) - rollout_corr/training_log_ppl:np.float64(0.16064590763562592) - rollout_corr/kl:np.float64(0.0010799969986834412) - rollout_corr/k3_kl:np.float64(0.0008794531678333328) - rollout_corr/rollout_ppl:np.float64(1.1800120291300118) - rollout_corr/rollout_log_ppl:np.float64(0.15956590972200502) - rollout_corr/log_ppl_diff:np.float64(0.0010799979136209004) - rollout_corr/log_ppl_abs_diff:np.float64(0.003594762813008856) - rollout_corr/log_ppl_diff_max:np.float64(0.017611786723136902) - rollout_corr/log_ppl_diff_min:np.float64(-0.013430103659629822) - rollout_corr/ppl_ratio:np.float64(1.0010905740782619) - rollout_corr/chi2_token:np.float64(0.0013402211479842663) - rollout_corr/chi2_seq:np.float64(0.17083631292916834) - actor/pg_loss:np.float64(0.00042288796976208687) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000403566773456987) - actor/ppo_kl:np.float64(0.00038324507055520485) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(1.0) - self_distillation/token_reweight_context_sample_fraction:np.float64(1.0) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.35546875) - self_distillation/token_reweight_w_mean:np.float64(157405.40360419755) - self_distillation/token_reweight_w_std:np.float64(1085899.6162218554) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999427755828947) - self_distillation/token_reweight_w_clip_frac:np.float64(0.004944167798385024) - self_distillation/empty_target_batch:np.float64(0.0) - actor/grad_norm:np.float64(0.1787949800491333) - perf/mfu/actor:np.float64(0.04061249599672334) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.43125534057617) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0001252456713700667 - critic/advantages/max:0.625 - critic/advantages/min:-0.375 - critic/returns/mean:-0.0001252456713700667 - critic/returns/max:0.625 - critic/returns/min:-0.375 - response_length/mean:101.36328125 - response_length/max:336.0 - response_length/min:32.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:101.36328125 - response_length_non_aborted/max:336.0 - response_length_non_aborted/min:32.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:578.59375 - prompt_length/max:1007.0 - prompt_length/min:222.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001971554011106491 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5052474681288004) - timing_s/agent_loop/generate_sequences/max:np.float64(2.570444405078888) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1800286036886973) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.570444405078888) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:1007 - timing_s/agent_loop/slowest/response_length:336 - timing_s/gen:8.470093375071883 - timing_s/reward:0.0726681724190712 - timing_s/old_log_prob:2.2498553320765495 - timing_s/adv:0.60207106359303 - timing_s/update_actor:15.33978072181344 - timing_s/step:26.822263691574335 - timing_s/stop_profile:0.00015156902372837067 - timing_per_token_ms/update_actor:0.08812471331376316 - timing_per_token_ms/gen:0.3264130939562944 - timing_per_token_ms/adv:0.0034588069305449562 - perf/total_num_tokens:174069 - perf/time_per_step:26.822263691574335 - perf/throughput:811.2150879657121 (TaskRunner pid=2204243) Training Progress: 97%|█████████▋| 97/100 [58:16<01:26, 28.76s/it] (TaskRunner pid=2204243) step:98 - global_seqlen/min:16939 - global_seqlen/max:28797 - global_seqlen/minmax_diff:11858 - global_seqlen/balanced_min:23295 - global_seqlen/balanced_max:23300 - global_seqlen/mean:23297.0 - actor/entropy:0.18151973187923431 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.19610834121704102 - training/rollout_probs_diff_mean:0.004390710033476353 - training/rollout_probs_diff_std:0.012627762742340565 - training/rollout_actor_probs_pearson_corr:0.9981019496917725 - self_distillation/success_group_fraction:0.40625 - self_distillation/success_sample_fraction:0.40234375 - self_distillation/correct_sample_fraction:0.35546875 - self_distillation/feedback_available_fraction:0.64453125 - self_distillation/feedback_used_fraction:0.59375 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:0.9998459219932556 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.777576421271078e-05 - rollout_corr/rollout_is_max:1.600449562072754 - rollout_corr/rollout_is_min:0.4510498642921448 - rollout_corr/rollout_is_std:0.03535743057727814 - rollout_corr/rollout_is_eff_sample_size:0.9987510563634333 - rollout_corr/rollout_is_seq_mean:0.999804675579071 - rollout_corr/rollout_is_seq_std:0.0036281393840909004 - rollout_corr/rollout_is_seq_max:1.012976050376892 - rollout_corr/rollout_is_seq_min:0.984198808670044 - rollout_corr/rollout_is_seq_max_deviation:0.015801191329956055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.18151760334149) - rollout_corr/training_log_ppl:np.float64(0.16213003455050057) - rollout_corr/kl:np.float64(0.0006451969248360001) - rollout_corr/k3_kl:np.float64(0.0008145639176224684) - rollout_corr/rollout_ppl:np.float64(1.1807032800279558) - rollout_corr/rollout_log_ppl:np.float64(0.16148483657161705) - rollout_corr/log_ppl_diff:np.float64(0.000645197978883516) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030326917112688534) - rollout_corr/log_ppl_diff_max:np.float64(0.017492949962615967) - rollout_corr/log_ppl_diff_min:np.float64(-0.009312212467193604) - rollout_corr/ppl_ratio:np.float64(1.000653215451166) - rollout_corr/chi2_token:np.float64(0.0021187232341617346) - rollout_corr/chi2_seq:np.float64(0.1499750402290374) - actor/pg_loss:np.float64(0.0009213185403496027) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020896588284813333) - actor/ppo_kl:np.float64(-0.00017378739831030998) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.35546875) - self_distillation/token_reweight_w_mean:np.float64(111930.10576668778) - self_distillation/token_reweight_w_std:np.float64(1192437.1679238342) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004100892692804) - self_distillation/token_reweight_w_clip_frac:np.float64(0.017748428974300623) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.2336823670193553) - perf/mfu/actor:np.float64(0.04363927880203005) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.44715118408203) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:0 - critic/score/mean:0.35546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.35546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0017754608998075128 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0017754608998075128 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:103.40625 - response_length/max:303.0 - response_length/min:37.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:103.40625 - response_length_non_aborted/max:303.0 - response_length_non_aborted/min:37.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:624.625 - prompt_length/max:1258.0 - prompt_length/min:229.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001833345741033554 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5240087099373341) - timing_s/agent_loop/generate_sequences/max:np.float64(2.4301755260676146) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.1862644308130257) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.4301755260676146) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:701 - timing_s/agent_loop/slowest/response_length:303 - timing_s/gen:8.23422322422266 - timing_s/reward:0.07538575492799282 - timing_s/old_log_prob:2.272018427029252 - timing_s/adv:0.6203307136893272 - timing_s/update_actor:15.27922528795898 - timing_s/step:26.568082386627793 - timing_s/stop_profile:0.0001768842339515686 - timing_per_token_ms/update_actor:0.08198064819482648 - timing_per_token_ms/gen:0.31105406558713583 - timing_per_token_ms/adv:0.003328383019752153 - perf/total_num_tokens:186376 - perf/time_per_step:26.568082386627793 - perf/throughput:876.8792440859718 (TaskRunner pid=2204243) Training Progress: 98%|█████████▊| 98/100 [58:43<00:56, 28.12s/it] (TaskRunner pid=2204243) step:99 - global_seqlen/min:15412 - global_seqlen/max:28396 - global_seqlen/minmax_diff:12984 - global_seqlen/balanced_min:21141 - global_seqlen/balanced_max:21143 - global_seqlen/mean:21142.375 - actor/entropy:0.13693051040172577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2576254606246948 - training/rollout_probs_diff_mean:0.003940090537071228 - training/rollout_probs_diff_std:0.012845836579799652 - training/rollout_actor_probs_pearson_corr:0.997589111328125 - self_distillation/success_group_fraction:0.46875 - self_distillation/success_sample_fraction:0.46484375 - self_distillation/correct_sample_fraction:0.44140625 - self_distillation/feedback_available_fraction:0.55859375 - self_distillation/feedback_used_fraction:0.53125 - self_distillation/reprompt_sample_fraction:0.99609375 - rollout_corr/rollout_is_mean:1.000468134880066 - rollout_corr/rollout_is_ratio_fraction_high:4.5071439672028646e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.1618735790252686 - rollout_corr/rollout_is_min:0.6069955825805664 - rollout_corr/rollout_is_std:0.036101456731557846 - rollout_corr/rollout_is_eff_sample_size:0.9986995702498151 - rollout_corr/rollout_is_seq_mean:1.0004678964614868 - rollout_corr/rollout_is_seq_std:0.003795790718868375 - rollout_corr/rollout_is_seq_max:1.0119315385818481 - rollout_corr/rollout_is_seq_min:0.991576611995697 - rollout_corr/rollout_is_seq_max_deviation:0.011931538581848145 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1395642152056098) - rollout_corr/training_log_ppl:np.float64(0.12678580552892527) - rollout_corr/kl:np.float64(0.00047369026876253884) - rollout_corr/k3_kl:np.float64(0.0008205241951912967) - rollout_corr/rollout_ppl:np.float64(1.1389737036079168) - rollout_corr/rollout_log_ppl:np.float64(0.12631211427651579) - rollout_corr/log_ppl_diff:np.float64(0.0004736912524094805) - rollout_corr/log_ppl_abs_diff:np.float64(0.003678554858197458) - rollout_corr/log_ppl_diff_max:np.float64(0.015213608741760254) - rollout_corr/log_ppl_diff_min:np.float64(-0.010651998221874237) - rollout_corr/ppl_ratio:np.float64(1.000484419753775) - rollout_corr/chi2_token:np.float64(0.0023528370074927807) - rollout_corr/chi2_seq:np.float64(0.3323637186549604) - actor/pg_loss:np.float64(0.00011820322833955288) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00014191106492944527) - actor/ppo_kl:np.float64(0.00034064576455250517) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.99609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.99609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44140625) - self_distillation/token_reweight_w_mean:np.float64(91160.89013671083) - self_distillation/token_reweight_w_std:np.float64(924321.06640625) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004756471607834) - self_distillation/token_reweight_w_clip_frac:np.float64(0.005306928127538413) - self_distillation/empty_target_batch:np.float64(0.00390625) - actor/grad_norm:np.float64(0.08416355215013027) - perf/mfu/actor:np.float64(0.0388341946252759) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.39789581298828) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00023099112149793655 - critic/advantages/max:0.875 - critic/advantages/min:-0.125 - critic/returns/mean:0.00023099112149793655 - critic/returns/max:0.875 - critic/returns/min:-0.125 - response_length/mean:86.66796875 - response_length/max:351.0 - response_length/min:33.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:86.66796875 - response_length_non_aborted/max:351.0 - response_length_non_aborted/min:33.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:574.03125 - prompt_length/max:1127.0 - prompt_length/min:261.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010357238352298737 - timing_s/agent_loop/generate_sequences/min:np.float64(0.49293369613587856) - timing_s/agent_loop/generate_sequences/max:np.float64(2.452165137976408) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.037320585346606) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.452165137976408) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:606 - timing_s/agent_loop/slowest/response_length:351 - timing_s/gen:8.192044220864773 - timing_s/reward:0.07275773584842682 - timing_s/old_log_prob:2.2582249213010073 - timing_s/adv:0.5899266004562378 - timing_s/update_actor:15.597165573388338 - timing_s/step:26.79719146154821 - timing_s/stop_profile:0.00012882240116596222 - timing_per_token_ms/update_actor:0.09221507501752013 - timing_per_token_ms/gen:0.3692272150748083 - timing_per_token_ms/adv:0.0034878212621349174 - perf/total_num_tokens:169139 - perf/time_per_step:26.79719146154821 - perf/throughput:788.9772713807561 (TaskRunner pid=2204243) Training Progress: 99%|█████████▉| 99/100 [59:10<00:27, 27.74s/it] (TaskRunner pid=2204243) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2204243) validation generation end (TaskRunner pid=2204243) [prompt] user (TaskRunner pid=2204243) Your task is to answer the user's question using available tools. (TaskRunner pid=2204243) You have access to the following tools: (TaskRunner pid=2204243) Name: Axolotl (TaskRunner pid=2204243) Description: Collection of axolotl pictures and facts (TaskRunner pid=2204243) Documentation: (TaskRunner pid=2204243) getRandomAxolotlImage: Retrieve a random axolotl image with information on the image source. (TaskRunner pid=2204243) Parameters: {} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{url, source, description} (TaskRunner pid=2204243) searchAxolotlImages: Search for axolotl images based on specific criteria such as color, gender, and size. (TaskRunner pid=2204243) Parameters: {"color": "string. One of: [wild, leucistic, albino]. The color of the axolotl (e.g., 'wild', 'leucistic', 'albino', etc.).", "gender": "string. One of: [male, female]. The gender of the axolotl ('male', 'female').", "size": "string. One of: [small, medium, large]. The size of the axolotl ('small', 'medium', 'large').", "page": "integer. The page number for pagination purposes."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Object{results: Array[Object{url, source, description}], pagination: Object{current_page, total_pages, total_results}} (TaskRunner pid=2204243) getAxolotlFacts: Retrieve interesting facts about axolotls such as their habits, habitats, and physical characteristics. (TaskRunner pid=2204243) Parameters: {"category": "string. One of: [habits, habitat, physical characteristics]. The category of facts to retrieve (e.g., 'habits', 'habitat', 'physical characteristics').", "limit": "integer. The maximum number of facts to return."} (TaskRunner pid=2204243) Output: Successful response. (TaskRunner pid=2204243) - Format: application/json (TaskRunner pid=2204243) - Structure: Array[Object{fact, source}] (TaskRunner pid=2204243) (TaskRunner pid=2204243) Use the following format: (TaskRunner pid=2204243) Thought: you should always think about what to do (TaskRunner pid=2204243) Action: the action to take, should be one of the tool names. (TaskRunner pid=2204243) Action Input: the input to the action, must be in JSON format. All of the action input must be realistic and from the user. (TaskRunner pid=2204243) (TaskRunner pid=2204243) Begin! (TaskRunner pid=2204243) Question: Hey, can you show me a random picture of an axolotl? (TaskRunner pid=2204243) assistant (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) (TaskRunner pid=2204243) [response] Thought: I need to find a random axolotl image. The available tool for this is getRandomAxolotlImage. Since the user asked for a random picture, I should use that tool. (TaskRunner pid=2204243) Action: getRandomAxolotlImage (TaskRunner pid=2204243) Action Input: {} (TaskRunner pid=2204243) [ground_truth] [{"Action": "getRandomAxolotlImage", "Action_Input": "{}"}] (TaskRunner pid=2204243) [score] 1.0 (TaskRunner pid=2204243) [acc] 1.0 (TaskRunner pid=2204243) [pred] Actions: ['getRandomAxolotlImage'], Inputs: {} (TaskRunner pid=2204243) [incorrect_format] 0 (TaskRunner pid=2204243) [feedback] (TaskRunner pid=2204243) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2204655) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2204243) step:100 - global_seqlen/min:18236 - global_seqlen/max:26968 - global_seqlen/minmax_diff:8732 - global_seqlen/balanced_min:21525 - global_seqlen/balanced_max:21530 - global_seqlen/mean:21528.0 - actor/entropy:0.15581010282039642 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.41022178530693054 - training/rollout_probs_diff_mean:0.0043131280690431595 - training/rollout_probs_diff_std:0.013580762781202793 - training/rollout_actor_probs_pearson_corr:0.9975739121437073 - self_distillation/success_group_fraction:0.5 - self_distillation/success_sample_fraction:0.4921875 - self_distillation/correct_sample_fraction:0.43359375 - self_distillation/feedback_available_fraction:0.56640625 - self_distillation/feedback_used_fraction:0.5 - self_distillation/reprompt_sample_fraction:0.9921875 - rollout_corr/rollout_is_mean:1.0004314184188843 - rollout_corr/rollout_is_ratio_fraction_high:8.322236681124195e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:2.820589303970337 - rollout_corr/rollout_is_min:0.5896446108818054 - rollout_corr/rollout_is_std:0.03966965898871422 - rollout_corr/rollout_is_eff_sample_size:0.9984299791032849 - rollout_corr/rollout_is_seq_mean:1.0004057884216309 - rollout_corr/rollout_is_seq_std:0.004751189611852169 - rollout_corr/rollout_is_seq_max:1.0307097434997559 - rollout_corr/rollout_is_seq_min:0.9896077513694763 - rollout_corr/rollout_is_seq_max_deviation:0.03070974349975586 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.150276112370193) - rollout_corr/training_log_ppl:np.float64(0.13597888073957165) - rollout_corr/kl:np.float64(0.0009126573687510131) - rollout_corr/k3_kl:np.float64(0.0007612622766463763) - rollout_corr/rollout_ppl:np.float64(1.1492430758662522) - rollout_corr/rollout_log_ppl:np.float64(0.13506622198246987) - rollout_corr/log_ppl_diff:np.float64(0.0009126587571017808) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032595000756145964) - rollout_corr/log_ppl_diff_max:np.float64(0.014425680041313171) - rollout_corr/log_ppl_diff_min:np.float64(-0.01413802057504654) - rollout_corr/ppl_ratio:np.float64(1.000922082690522) - rollout_corr/chi2_token:np.float64(0.0012444618623703718) - rollout_corr/chi2_seq:np.float64(0.2823219452984631) - actor/pg_loss:np.float64(0.0003957020817324519) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003417537127461401) - actor/ppo_kl:np.float64(0.0005588802749123545) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.9921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.9921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.43359375) - self_distillation/token_reweight_w_mean:np.float64(532096.5092833575) - self_distillation/token_reweight_w_std:np.float64(3821444.055784081) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005437803920358) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0204948254104238) - self_distillation/empty_target_batch:np.float64(0.0078125) - actor/grad_norm:np.float64(0.2933488469570875) - perf/mfu/actor:np.float64(0.03961015113542468) - perf/max_memory_allocated_gb:np.float64(125.62322854995728) - perf/max_memory_reserved_gb:np.float64(132.13671875) - perf/cpu_memory_used_gb:np.float64(146.36181640625) - actor/lr:np.float64(1e-06) - val-aux/tooluse/reward/mean@16:np.float64(0.59375) - val-aux/tooluse/reward/std@16:np.float64(0.0397091660129265) - val-aux/tooluse/reward/best@2/mean:np.float64(0.6084705882352941) - val-aux/tooluse/reward/best@2/std:np.float64(0.021252889471013774) - val-aux/tooluse/reward/worst@2/mean:np.float64(0.5792499999999999) - val-aux/tooluse/reward/worst@2/std:np.float64(0.04534421397949325) - val-aux/tooluse/reward/maj@2/mean:np.float64(0.5937794117647059) - val-aux/tooluse/reward/maj@2/std:np.float64(0.04051488817390775) - val-aux/tooluse/reward/best@4/mean:np.float64(0.6149852941176471) - val-aux/tooluse/reward/best@4/std:np.float64(0.008533867314311838) - val-aux/tooluse/reward/worst@4/mean:np.float64(0.5617647058823529) - val-aux/tooluse/reward/worst@4/std:np.float64(0.04688580916450452) - val-aux/tooluse/reward/maj@4/mean:np.float64(0.6007058823529412) - val-aux/tooluse/reward/maj@4/std:np.float64(0.028165057960484432) - val-aux/tooluse/reward/best@8/mean:np.float64(0.6173676470588235) - val-aux/tooluse/reward/best@8/std:np.float64(0.0024199717661105916) - val-aux/tooluse/reward/worst@8/mean:np.float64(0.5418382352941178) - val-aux/tooluse/reward/worst@8/std:np.float64(0.04295702328078124) - val-aux/tooluse/reward/maj@8/mean:np.float64(0.6039705882352941) - val-aux/tooluse/reward/maj@8/std:np.float64(0.016895760456278525) - val-aux/tooluse/reward/best@16/mean:np.float64(0.6176323529411765) - val-aux/tooluse/reward/best@16/std:np.float64(0.00046480825380232674) - val-aux/tooluse/reward/worst@16/mean:np.float64(0.5220294117647059) - val-aux/tooluse/reward/worst@16/std:np.float64(0.03331555519855917) - val-aux/tooluse/reward/maj@16/mean:np.float64(0.6051029411764706) - val-aux/tooluse/reward/maj@16/std:np.float64(0.010892157021494882) - val-aux/tooluse/score/mean@16:np.float64(0.59375) - val-aux/tooluse/score/std@16:np.float64(0.0397091660129265) - val-aux/tooluse/score/best@2/mean:np.float64(0.6084705882352941) - val-aux/tooluse/score/best@2/std:np.float64(0.021252889471013774) - val-aux/tooluse/score/worst@2/mean:np.float64(0.5792499999999999) - val-aux/tooluse/score/worst@2/std:np.float64(0.04534421397949325) - val-aux/tooluse/score/maj@2/mean:np.float64(0.5937794117647059) - val-aux/tooluse/score/maj@2/std:np.float64(0.04051488817390775) - val-aux/tooluse/score/best@4/mean:np.float64(0.6149852941176471) - val-aux/tooluse/score/best@4/std:np.float64(0.008533867314311838) - val-aux/tooluse/score/worst@4/mean:np.float64(0.5617647058823529) - val-aux/tooluse/score/worst@4/std:np.float64(0.04688580916450452) - val-aux/tooluse/score/maj@4/mean:np.float64(0.6007058823529412) - val-aux/tooluse/score/maj@4/std:np.float64(0.028165057960484432) - val-aux/tooluse/score/best@8/mean:np.float64(0.6173676470588235) - val-aux/tooluse/score/best@8/std:np.float64(0.0024199717661105916) - val-aux/tooluse/score/worst@8/mean:np.float64(0.5418382352941178) - val-aux/tooluse/score/worst@8/std:np.float64(0.04295702328078124) - val-aux/tooluse/score/maj@8/mean:np.float64(0.6039705882352941) - val-aux/tooluse/score/maj@8/std:np.float64(0.016895760456278525) - val-aux/tooluse/score/best@16/mean:np.float64(0.6176323529411765) - val-aux/tooluse/score/best@16/std:np.float64(0.00046480825380232674) - val-aux/tooluse/score/worst@16/mean:np.float64(0.5220294117647059) - val-aux/tooluse/score/worst@16/std:np.float64(0.03331555519855917) - val-aux/tooluse/score/maj@16/mean:np.float64(0.6051029411764706) - val-aux/tooluse/score/maj@16/std:np.float64(0.010892157021494882) - val-core/tooluse/acc/mean@16:np.float64(0.59375) - val-aux/tooluse/acc/std@16:np.float64(0.0397091660129265) - val-aux/tooluse/acc/best@2/mean:np.float64(0.6084705882352941) - val-aux/tooluse/acc/best@2/std:np.float64(0.021252889471013774) - val-aux/tooluse/acc/worst@2/mean:np.float64(0.5792499999999999) - val-aux/tooluse/acc/worst@2/std:np.float64(0.04534421397949325) - val-aux/tooluse/acc/maj@2/mean:np.float64(0.5937794117647059) - val-aux/tooluse/acc/maj@2/std:np.float64(0.04051488817390775) - val-aux/tooluse/acc/best@4/mean:np.float64(0.6149852941176471) - val-aux/tooluse/acc/best@4/std:np.float64(0.008533867314311838) - val-aux/tooluse/acc/worst@4/mean:np.float64(0.5617647058823529) - val-aux/tooluse/acc/worst@4/std:np.float64(0.04688580916450452) - val-aux/tooluse/acc/maj@4/mean:np.float64(0.6007058823529412) - val-aux/tooluse/acc/maj@4/std:np.float64(0.028165057960484432) - val-aux/tooluse/acc/best@8/mean:np.float64(0.6173676470588235) - val-aux/tooluse/acc/best@8/std:np.float64(0.0024199717661105916) - val-aux/tooluse/acc/worst@8/mean:np.float64(0.5418382352941178) - val-aux/tooluse/acc/worst@8/std:np.float64(0.04295702328078124) - val-aux/tooluse/acc/maj@8/mean:np.float64(0.6039705882352941) - val-aux/tooluse/acc/maj@8/std:np.float64(0.016895760456278525) - val-core/tooluse/acc/best@16/mean:np.float64(0.6176323529411765) - val-core/tooluse/acc/best@16/std:np.float64(0.00046480825380232674) - val-aux/tooluse/acc/worst@16/mean:np.float64(0.5220294117647059) - val-aux/tooluse/acc/worst@16/std:np.float64(0.03331555519855917) - val-core/tooluse/acc/maj@16/mean:np.float64(0.6051029411764706) - val-core/tooluse/acc/maj@16/std:np.float64(0.010892157021494882) - val-aux/tooluse/incorrect_format/mean@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/std@16:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/best@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/mean:np.float64(0.0) - val-aux/tooluse/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:0 - critic/score/mean:0.43359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.43359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008946404792368412 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008946404792368412 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:93.875 - response_length/max:450.0 - response_length/min:38.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:93.875 - response_length_non_aborted/max:450.0 - response_length_non_aborted/min:38.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:578.875 - prompt_length/max:934.0 - prompt_length/min:251.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013494491577148438 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5163722541183233) - timing_s/agent_loop/generate_sequences/max:np.float64(2.9513198621571064) - timing_s/agent_loop/generate_sequences/mean:np.float64(1.0657720906820032) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(2.9513198621571064) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:565 - timing_s/agent_loop/slowest/response_length:450 - timing_s/gen:8.898955885320902 - timing_s/reward:0.0715668685734272 - timing_s/old_log_prob:2.2224635891616344 - timing_s/adv:0.5949337817728519 - timing_s/update_actor:15.542495667934418 - timing_s/step:27.41770182736218 - timing_s/testing:21.686641938984394 - timing_s/save_checkpoint:6.582349684089422 - timing_s/stop_profile:0.00013438984751701355 - timing_per_token_ms/update_actor:0.09024581746989048 - timing_per_token_ms/gen:0.37029610042114275 - timing_per_token_ms/adv:0.0034544185582314423 - perf/total_num_tokens:172224 - perf/time_per_step:27.41770182736218 - perf/throughput:785.1861594947975 (TaskRunner pid=2204243) ("Final validation metrics: {'val-aux/tooluse/reward/mean@16': " (TaskRunner pid=2204243) "np.float64(0.59375), 'val-aux/tooluse/reward/std@16': " (TaskRunner pid=2204243) "np.float64(0.0397091660129265), 'val-aux/tooluse/reward/best@2/mean': " (TaskRunner pid=2204243) "np.float64(0.6084705882352941), 'val-aux/tooluse/reward/best@2/std': " (TaskRunner pid=2204243) "np.float64(0.021252889471013774), 'val-aux/tooluse/reward/worst@2/mean': " (TaskRunner pid=2204243) "np.float64(0.5792499999999999), 'val-aux/tooluse/reward/worst@2/std': " (TaskRunner pid=2204243) "np.float64(0.04534421397949325), 'val-aux/tooluse/reward/maj@2/mean': " (TaskRunner pid=2204243) "np.float64(0.5937794117647059), 'val-aux/tooluse/reward/maj@2/std': " (TaskRunner pid=2204243) "np.float64(0.04051488817390775), 'val-aux/tooluse/reward/best@4/mean': " (TaskRunner pid=2204243) "np.float64(0.6149852941176471), 'val-aux/tooluse/reward/best@4/std': " (TaskRunner pid=2204243) "np.float64(0.008533867314311838), 'val-aux/tooluse/reward/worst@4/mean': " (TaskRunner pid=2204243) "np.float64(0.5617647058823529), 'val-aux/tooluse/reward/worst@4/std': " (TaskRunner pid=2204243) "np.float64(0.04688580916450452), 'val-aux/tooluse/reward/maj@4/mean': " (TaskRunner pid=2204243) "np.float64(0.6007058823529412), 'val-aux/tooluse/reward/maj@4/std': " (TaskRunner pid=2204243) "np.float64(0.028165057960484432), 'val-aux/tooluse/reward/best@8/mean': " (TaskRunner pid=2204243) "np.float64(0.6173676470588235), 'val-aux/tooluse/reward/best@8/std': " (TaskRunner pid=2204243) "np.float64(0.0024199717661105916), 'val-aux/tooluse/reward/worst@8/mean': " (TaskRunner pid=2204243) "np.float64(0.5418382352941178), 'val-aux/tooluse/reward/worst@8/std': " (TaskRunner pid=2204243) "np.float64(0.04295702328078124), 'val-aux/tooluse/reward/maj@8/mean': " (TaskRunner pid=2204243) "np.float64(0.6039705882352941), 'val-aux/tooluse/reward/maj@8/std': " (TaskRunner pid=2204243) "np.float64(0.016895760456278525), 'val-aux/tooluse/reward/best@16/mean': " (TaskRunner pid=2204243) "np.float64(0.6176323529411765), 'val-aux/tooluse/reward/best@16/std': " (TaskRunner pid=2204243) "np.float64(0.00046480825380232674), 'val-aux/tooluse/reward/worst@16/mean': " (TaskRunner pid=2204243) "np.float64(0.5220294117647059), 'val-aux/tooluse/reward/worst@16/std': " (TaskRunner pid=2204243) "np.float64(0.03331555519855917), 'val-aux/tooluse/reward/maj@16/mean': " (TaskRunner pid=2204243) "np.float64(0.6051029411764706), 'val-aux/tooluse/reward/maj@16/std': " (TaskRunner pid=2204243) "np.float64(0.010892157021494882), 'val-aux/tooluse/score/mean@16': " (TaskRunner pid=2204243) "np.float64(0.59375), 'val-aux/tooluse/score/std@16': " (TaskRunner pid=2204243) "np.float64(0.0397091660129265), 'val-aux/tooluse/score/best@2/mean': " (TaskRunner pid=2204243) "np.float64(0.6084705882352941), 'val-aux/tooluse/score/best@2/std': " (TaskRunner pid=2204243) "np.float64(0.021252889471013774), 'val-aux/tooluse/score/worst@2/mean': " (TaskRunner pid=2204243) "np.float64(0.5792499999999999), 'val-aux/tooluse/score/worst@2/std': " (TaskRunner pid=2204243) "np.float64(0.04534421397949325), 'val-aux/tooluse/score/maj@2/mean': " (TaskRunner pid=2204243) "np.float64(0.5937794117647059), 'val-aux/tooluse/score/maj@2/std': " (TaskRunner pid=2204243) "np.float64(0.04051488817390775), 'val-aux/tooluse/score/best@4/mean': " (TaskRunner pid=2204243) "np.float64(0.6149852941176471), 'val-aux/tooluse/score/best@4/std': " (TaskRunner pid=2204243) "np.float64(0.008533867314311838), 'val-aux/tooluse/score/worst@4/mean': " (TaskRunner pid=2204243) "np.float64(0.5617647058823529), 'val-aux/tooluse/score/worst@4/std': " (TaskRunner pid=2204243) "np.float64(0.04688580916450452), 'val-aux/tooluse/score/maj@4/mean': " (TaskRunner pid=2204243) "np.float64(0.6007058823529412), 'val-aux/tooluse/score/maj@4/std': " (TaskRunner pid=2204243) "np.float64(0.028165057960484432), 'val-aux/tooluse/score/best@8/mean': " (TaskRunner pid=2204243) "np.float64(0.6173676470588235), 'val-aux/tooluse/score/best@8/std': " (TaskRunner pid=2204243) "np.float64(0.0024199717661105916), 'val-aux/tooluse/score/worst@8/mean': " (TaskRunner pid=2204243) "np.float64(0.5418382352941178), 'val-aux/tooluse/score/worst@8/std': " (TaskRunner pid=2204243) "np.float64(0.04295702328078124), 'val-aux/tooluse/score/maj@8/mean': " (TaskRunner pid=2204243) "np.float64(0.6039705882352941), 'val-aux/tooluse/score/maj@8/std': " (TaskRunner pid=2204243) "np.float64(0.016895760456278525), 'val-aux/tooluse/score/best@16/mean': " (TaskRunner pid=2204243) "np.float64(0.6176323529411765), 'val-aux/tooluse/score/best@16/std': " (TaskRunner pid=2204243) "np.float64(0.00046480825380232674), 'val-aux/tooluse/score/worst@16/mean': " (TaskRunner pid=2204243) "np.float64(0.5220294117647059), 'val-aux/tooluse/score/worst@16/std': " (TaskRunner pid=2204243) "np.float64(0.03331555519855917), 'val-aux/tooluse/score/maj@16/mean': " (TaskRunner pid=2204243) "np.float64(0.6051029411764706), 'val-aux/tooluse/score/maj@16/std': " (TaskRunner pid=2204243) "np.float64(0.010892157021494882), 'val-core/tooluse/acc/mean@16': " (TaskRunner pid=2204243) "np.float64(0.59375), 'val-aux/tooluse/acc/std@16': " (TaskRunner pid=2204243) "np.float64(0.0397091660129265), 'val-aux/tooluse/acc/best@2/mean': " (TaskRunner pid=2204243) "np.float64(0.6084705882352941), 'val-aux/tooluse/acc/best@2/std': " (TaskRunner pid=2204243) "np.float64(0.021252889471013774), 'val-aux/tooluse/acc/worst@2/mean': " (TaskRunner pid=2204243) "np.float64(0.5792499999999999), 'val-aux/tooluse/acc/worst@2/std': " (TaskRunner pid=2204243) "np.float64(0.04534421397949325), 'val-aux/tooluse/acc/maj@2/mean': " (TaskRunner pid=2204243) "np.float64(0.5937794117647059), 'val-aux/tooluse/acc/maj@2/std': " (TaskRunner pid=2204243) "np.float64(0.04051488817390775), 'val-aux/tooluse/acc/best@4/mean': " (TaskRunner pid=2204243) "np.float64(0.6149852941176471), 'val-aux/tooluse/acc/best@4/std': " (TaskRunner pid=2204243) "np.float64(0.008533867314311838), 'val-aux/tooluse/acc/worst@4/mean': " (TaskRunner pid=2204243) "np.float64(0.5617647058823529), 'val-aux/tooluse/acc/worst@4/std': " (TaskRunner pid=2204243) "np.float64(0.04688580916450452), 'val-aux/tooluse/acc/maj@4/mean': " (TaskRunner pid=2204243) "np.float64(0.6007058823529412), 'val-aux/tooluse/acc/maj@4/std': " (TaskRunner pid=2204243) "np.float64(0.028165057960484432), 'val-aux/tooluse/acc/best@8/mean': " (TaskRunner pid=2204243) "np.float64(0.6173676470588235), 'val-aux/tooluse/acc/best@8/std': " (TaskRunner pid=2204243) "np.float64(0.0024199717661105916), 'val-aux/tooluse/acc/worst@8/mean': " (TaskRunner pid=2204243) "np.float64(0.5418382352941178), 'val-aux/tooluse/acc/worst@8/std': " (TaskRunner pid=2204243) "np.float64(0.04295702328078124), 'val-aux/tooluse/acc/maj@8/mean': " (TaskRunner pid=2204243) "np.float64(0.6039705882352941), 'val-aux/tooluse/acc/maj@8/std': " (TaskRunner pid=2204243) "np.float64(0.016895760456278525), 'val-core/tooluse/acc/best@16/mean': " (TaskRunner pid=2204243) "np.float64(0.6176323529411765), 'val-core/tooluse/acc/best@16/std': " (TaskRunner pid=2204243) "np.float64(0.00046480825380232674), 'val-aux/tooluse/acc/worst@16/mean': " (TaskRunner pid=2204243) "np.float64(0.5220294117647059), 'val-aux/tooluse/acc/worst@16/std': " (TaskRunner pid=2204243) "np.float64(0.03331555519855917), 'val-core/tooluse/acc/maj@16/mean': " (TaskRunner pid=2204243) "np.float64(0.6051029411764706), 'val-core/tooluse/acc/maj@16/std': " (TaskRunner pid=2204243) 'np.float64(0.010892157021494882), ' (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/mean@16': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/std@16': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/best@2/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/best@2/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/worst@2/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/worst@2/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/maj@2/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/maj@2/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/best@4/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/worst@4/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/worst@4/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/maj@4/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/maj@4/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/best@8/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/worst@8/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/worst@8/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/maj@8/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/best@16/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/worst@16/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/worst@16/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/maj@16/mean': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/tooluse/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2204243) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2204243) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2204243) Training Progress: 100%|██████████| 100/100 [1:00:06<00:00, 36.15s/it] Training Progress: 100%|██████████| 100/100 [1:00:06<00:00, 36.06s/it] (TaskRunner pid=2204243) wandb: updating run metadata (TaskRunner pid=2204243) wandb: updating run metadata; uploading output.log; uploading wandb-summary.json (TaskRunner pid=2204243) wandb: uploading output.log (TaskRunner pid=2204243) wandb: uploading config.yaml (TaskRunner pid=2204243) wandb: uploading history steps 98-99, summary, console lines 575-682 (TaskRunner pid=2204243) wandb: (TaskRunner pid=2204243) wandb: Run history: (TaskRunner pid=2204243) wandb: actor/entropy ▃▂▂▂▁▁▁▂▂▂▂▁▂▁▁▂▁▁▂▁█▁▁▁▁▁▂▂▂▂▂▂▂▂▂▂▂▂▂▂ (TaskRunner pid=2204243) wandb: actor/grad_norm ▅▆▆▃▆▆▁█▁█▂▆▄▇▄▂▆▂█▃▃▃▇▄▁▄█▄▂▃▃▅▃▄▃▂▂▃▅▃ (TaskRunner pid=2204243) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2204243) wandb: actor/lr ▁▂▃▄▅▆▇█████████████████████████████████ (TaskRunner pid=2204243) wandb: actor/pg_clipfrac ▁▃▁▂▁▆▁▄▂▃▃▂▅▃█▂▁▂▃▅▄▂▅▁▂▃▃▆▃▃▃▇▁▇▄▃▄▄▄▃ (TaskRunner pid=2204243) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2204243) wandb: actor/pg_loss ▂▃▃▂▄▄▃▂▂▁▄▅▂▁▃▄▂▄▅▂▄▅▄▃▄▃▇▆▂▇▃█▅▂▁▅▄▄▄▃ (TaskRunner pid=2204243) wandb: actor/ppo_kl ▂▁▁▃▁▂▂▂▁▂▂▂▃█▂▂▂▂▃▂▅▃▃▁▂▃▂▂▂▁▂▂▂▂▂▂▂▂▁▃ (TaskRunner pid=2204243) wandb: critic/advantages/max █▅▄█▅▂█▂▄▅█▂▂▆█▁▇▂█▆▇███▇▇▇▇████▂████▄██ (TaskRunner pid=2204243) wandb: critic/advantages/mean ▇▇▁▆▆▇▆▇▇█▂▇▇▆▆▅▇▇█▇▇▆▆▇▇▇▇▇▇▅▆▅▇▇▇▇▇▆▇▇ (TaskRunner pid=2204243) wandb: +220 ... (TaskRunner pid=2204243) wandb: (TaskRunner pid=2204243) wandb: Run summary: (TaskRunner pid=2204243) wandb: actor/entropy 0.15581 (TaskRunner pid=2204243) wandb: actor/grad_norm 0.29335 (TaskRunner pid=2204243) wandb: actor/kl_loss 0 (TaskRunner pid=2204243) wandb: actor/lr 0.0 (TaskRunner pid=2204243) wandb: actor/pg_clipfrac 0.00034 (TaskRunner pid=2204243) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2204243) wandb: actor/pg_loss 0.0004 (TaskRunner pid=2204243) wandb: actor/ppo_kl 0.00056 (TaskRunner pid=2204243) wandb: critic/advantages/max 0.875 (TaskRunner pid=2204243) wandb: critic/advantages/mean -0.00895 (TaskRunner pid=2204243) wandb: +220 ... (TaskRunner pid=2204243) wandb: (TaskRunner pid=2204243) wandb: 🚀 View run qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/jrdzd11b (TaskRunner pid=2204243) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2204243) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2204243) wandb: Find logs at: ./wandb/run-20260702_154716-jrdzd11b/logs (TaskRunner pid=2204243) Exception ignored in atexit callback: .teardown_atexit at 0x732593dd3880> (TaskRunner pid=2204243) Traceback (most recent call last): (TaskRunner pid=2204243) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2204243) conn.teardown(hooks.exit_code) (TaskRunner pid=2204243) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2204243) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2204243) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2204243) return future.result() (TaskRunner pid=2204243) ^^^^^^^^^^^^^^^ (TaskRunner pid=2204243) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2204243) return self.__get_result() (TaskRunner pid=2204243) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2204243) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2204243) (TaskRunner pid=2204243) raise self._exception (TaskRunner pid=2204243) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2204243) return await fn() (TaskRunner pid=2204243) ^^^^^^^^^^ (TaskRunner pid=2204243) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2204243) await self._client.close() (TaskRunner pid=2204243) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2204243) await self._writer.wait_closed() (TaskRunner pid=2204243) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2204243) await self._protocol._get_close_waiter(self) (TaskRunner pid=2204243) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 16:47:31] Finished tooluse rlsd_tr /mnt/mole/SDPO/L2T/scripts/run_l2t_batch32_grpo_rlsd_tr_dataset_first_queue.sh: line 70: ation.sh: command not found